Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor(client): simplify keyupdate testcase implementation #1808

Merged
merged 1 commit into from
Apr 10, 2024

Conversation

mxinden
Copy link
Collaborator

@mxinden mxinden commented Apr 9, 2024

The QUIC Interop Runner keyupdate testcase has a client establish a connection to the server and then trigger a key update.

https://github.com/quic-interop/quic-interop-runner/blob/2a2534a1284d50d99ff92884d4f1ecf98fb41e4c/testcases.py#L889

This testcase always uses the http09 client and server implementation.

This commit simplifies the testcase implementation:

  • Given that it is only used with http09, move it to http09.rs.
  • Reduce the KeyUpdateState struct to a single bool.
  • Mark the --key-update command line argument as hidden, given that it is only set indirectly through the -t keyupdate flag.
  • Try to run client.initiate_key_update on events only, not on every new received datagram.

In addition it enables the keyupdate test on the Neqo qns.yml CI workflow.

The QUIC Interop Runner `keyupdate` testcase has a client establish a connection
to the server and then trigger a key update.

https://github.com/quic-interop/quic-interop-runner/blob/2a2534a1284d50d99ff92884d4f1ecf98fb41e4c/testcases.py#L889

This testcase always uses the `http09` client and server implementation.

This commit simplifies the testcase implementation:

- Given that it is only used with `http09`, move it to `http09.rs`.
- Reduce the `KeyUpdateState` `struct` to a single `bool`.
- Mark the `--key-update` command line argument as hidden, given that it is only
  set indirectly through the `-t keyupdate` flag.
- Try to run `client.initiate_key_update` on events only, not on ever new
  received datagram.

In addition it enables the `keyupdate` test on the Neqo `qns.yml` CI workflow.
Copy link

codecov bot commented Apr 9, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 93.13%. Comparing base (166b84c) to head (a83b972).
Report is 1 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #1808   +/-   ##
=======================================
  Coverage   93.13%   93.13%           
=======================================
  Files         117      117           
  Lines       36363    36339   -24     
=======================================
- Hits        33865    33843   -22     
+ Misses       2498     2496    -2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@mxinden
Copy link
Collaborator Author

mxinden commented Apr 9, 2024

keyupdate testcase succeeding (as it did before) across our test matrix.

neqo-latest quic-go ngtcp2 neqo msquic
neqo-latest ✓(H,U) ✓(H,U) ✓(H,U) ✓(H,U) ✓(H,U)
?() ?() ?() ?() ?()
✕() ✕() ✕() ✕() ✕()
:---------: :---------: :-----: :----: :----: :----:
quic-go ✓(H,U)
?()
✕()
:---------: :---------: :-----: :----: :----: :----:
ngtcp2 ✓(H,U)
?()
✕()
:---------: :---------: :-----: :----: :----: :----:
neqo ✓(H,U)
?()
✕()
:---------: :---------: :-----: :----: :----: :----:
msquic ✓(H,U)
?()
✕()
:---------: :---------: :-----: :----: :----: :----:

https://github.com/mozilla/neqo/actions/runs/8615252271/job/23610385581?pr=1808

@mxinden mxinden marked this pull request as ready for review April 9, 2024 13:54
Copy link

github-actions bot commented Apr 9, 2024

Benchmark results

Performance differences relative to 329af2f.

  • drain a timer quickly time: [360.09 ns 367.82 ns 375.25 ns]
    change: [-1.8724% +0.0209% +2.0240%] (p = 0.98 > 0.05)
    No change in performance detected.

  • coalesce_acked_from_zero 1+1 entries
    time: [194.65 ns 195.10 ns 195.57 ns]
    change: [-0.0091% +2.7422% +10.069%] (p = 0.25 > 0.05)
    No change in performance detected.

  • coalesce_acked_from_zero 3+1 entries
    time: [238.64 ns 239.22 ns 239.87 ns]
    change: [-0.3720% +0.3155% +1.0455%] (p = 0.41 > 0.05)
    No change in performance detected.

  • coalesce_acked_from_zero 10+1 entries
    time: [236.67 ns 237.54 ns 238.55 ns]
    change: [-0.6077% -0.0707% +0.5377%] (p = 0.81 > 0.05)
    No change in performance detected.

  • coalesce_acked_from_zero 1000+1 entries
    time: [218.86 ns 219.05 ns 219.25 ns]
    change: [-1.2930% -0.6098% +0.0891%] (p = 0.08 > 0.05)
    No change in performance detected.

  • RxStreamOrderer::inbound_frame()
    time: [118.61 ms 118.76 ms 119.00 ms]
    change: [-0.2877% -0.0509% +0.1811%] (p = 0.71 > 0.05)
    No change in performance detected.

  • transfer/Run multiple transfers with varying seeds
    time: [119.71 ms 120.00 ms 120.29 ms]
    thrpt: [33.253 MiB/s 33.334 MiB/s 33.414 MiB/s]
    change:
    time: [+0.9814% +1.2866% +1.6137%] (p = 0.00 < 0.05)
    thrpt: [-1.5881% -1.2702% -0.9718%]
    Change within noise threshold.

  • transfer/Run multiple transfers with the same seed
    time: [120.45 ms 120.65 ms 120.86 ms]
    thrpt: [33.097 MiB/s 33.154 MiB/s 33.209 MiB/s]
    change:
    time: [+0.4748% +0.7020% +0.9136%] (p = 0.00 < 0.05)
    thrpt: [-0.9053% -0.6971% -0.4725%]
    Change within noise threshold.

  • 1-conn/1-100mb-resp (aka. Download)/client
    time: [1.1129 s 1.1167 s 1.1211 s]
    thrpt: [89.198 MiB/s 89.553 MiB/s 89.855 MiB/s]
    change:
    time: [-0.0832% +0.5697% +1.1667%] (p = 0.11 > 0.05)
    thrpt: [-1.1532% -0.5664% +0.0833%]
    No change in performance detected.

  • 1-conn/10_000-parallel-1b-resp (aka. RPS)/client
    time: [427.55 ms 429.78 ms 432.06 ms]
    thrpt: [23.145 Kelem/s 23.268 Kelem/s 23.389 Kelem/s]
    change:
    time: [-0.5981% +0.1057% +0.7970%] (p = 0.77 > 0.05)
    thrpt: [-0.7907% -0.1056% +0.6017%]
    No change in performance detected.

  • 1-conn/1-1b-resp (aka. HPS)/client
    time: [51.780 ms 52.084 ms 52.487 ms]
    thrpt: [19.052 elem/s 19.200 elem/s 19.312 elem/s]
    change:
    time: [+1.6380% +2.7697% +3.9125%] (p = 0.00 < 0.05)
    thrpt: [-3.7652% -2.6950% -1.6116%]
    💔 Performance has regressed.

Client/server transfer results

Transfer of 134217728 bytes over loopback.

Client Server CC Pacing Mean [ms] Min [ms] Max [ms] Relative
msquic msquic 773.9 ± 258.0 531.2 1218.3 1.00
neqo msquic reno on 852.1 ± 83.8 742.6 1017.1 1.00
neqo msquic reno 909.3 ± 229.2 735.2 1376.2 1.00
neqo msquic cubic on 930.0 ± 260.8 736.6 1434.0 1.00
neqo msquic cubic 860.3 ± 261.8 766.3 1604.7 1.00
msquic neqo reno on 4498.9 ± 182.6 4230.4 4859.0 1.00
msquic neqo reno 4400.8 ± 297.7 4054.7 5104.2 1.00
msquic neqo cubic on 4552.2 ± 212.2 4286.7 4986.7 1.00
msquic neqo cubic 4501.3 ± 273.6 4133.6 5090.6 1.00
neqo neqo reno on 3617.8 ± 282.0 3176.7 4158.7 1.00
neqo neqo reno 3898.0 ± 250.5 3588.5 4351.2 1.00
neqo neqo cubic on 4368.4 ± 311.1 4016.3 4950.9 1.00
neqo neqo cubic 4360.9 ± 223.1 4029.3 4794.3 1.00

⬇️ Download logs

@larseggert larseggert added this pull request to the merge queue Apr 10, 2024
Merged via the queue into mozilla:main with commit c44e536 Apr 10, 2024
16 checks passed
mxinden added a commit to mxinden/neqo that referenced this pull request May 4, 2024
There are two server implementations based on neqo:

1. https://github.com/mozilla/neqo/tree/main/neqo-bin/src/server
  - http3 and http09 implementation
  - used for manual testing and QUIC Interop

2. https://searchfox.org/mozilla-central/source/netwerk/test/http3server/src/main.rs
  - used to test Firefox

I assume one was once an exact copy of the other. Both implement their own I/O,
event loop, ... Since then, the two implementations diverged significantly.
Especially (1) saw a lot of improvements in recent months:

- mozilla#1564
- mozilla#1569
- mozilla#1578
- mozilla#1581
- mozilla#1604
- mozilla#1612
- mozilla#1676
- mozilla#1692
- mozilla#1707
- mozilla#1708
- mozilla#1727
- mozilla#1753
- mozilla#1756
- mozilla#1766
- mozilla#1772
- mozilla#1786
- mozilla#1787
- mozilla#1788
- mozilla#1794
- mozilla#1806
- mozilla#1808
- mozilla#1848
- mozilla#1866

At this point, bugs in (2) are hard to fix, see e.g.
mozilla#1801.

This commit merges (2) into (1), thus removing all duplicate logic and
having (2) benefit from all the recent improvements to (1).
KershawChang pushed a commit to KershawChang/neqo that referenced this pull request May 7, 2024
There are two server implementations based on neqo:

1. https://github.com/mozilla/neqo/tree/main/neqo-bin/src/server
  - http3 and http09 implementation
  - used for manual testing and QUIC Interop

2. https://searchfox.org/mozilla-central/source/netwerk/test/http3server/src/main.rs
  - used to test Firefox

I assume one was once an exact copy of the other. Both implement their own I/O,
event loop, ... Since then, the two implementations diverged significantly.
Especially (1) saw a lot of improvements in recent months:

- mozilla#1564
- mozilla#1569
- mozilla#1578
- mozilla#1581
- mozilla#1604
- mozilla#1612
- mozilla#1676
- mozilla#1692
- mozilla#1707
- mozilla#1708
- mozilla#1727
- mozilla#1753
- mozilla#1756
- mozilla#1766
- mozilla#1772
- mozilla#1786
- mozilla#1787
- mozilla#1788
- mozilla#1794
- mozilla#1806
- mozilla#1808
- mozilla#1848
- mozilla#1866

At this point, bugs in (2) are hard to fix, see e.g.
mozilla#1801.

This commit merges (2) into (1), thus removing all duplicate logic and
having (2) benefit from all the recent improvements to (1).
github-merge-queue bot pushed a commit that referenced this pull request May 8, 2024
* refactor(bin): introduce server/http3.rs and server/http09.rs

The QUIC Interop Runner requires an http3 and http09 implementation for both
client and server. The client code is already structured into an http3 and an
http09 implementation since #1727.

This commit does the same for the server side, i.e. splits the http3 and http09
implementation into separate Rust modules.

* refactor: merge mozilla-central http3 server into neqo-bin

There are two server implementations based on neqo:

1. https://github.com/mozilla/neqo/tree/main/neqo-bin/src/server
  - http3 and http09 implementation
  - used for manual testing and QUIC Interop

2. https://searchfox.org/mozilla-central/source/netwerk/test/http3server/src/main.rs
  - used to test Firefox

I assume one was once an exact copy of the other. Both implement their own I/O,
event loop, ... Since then, the two implementations diverged significantly.
Especially (1) saw a lot of improvements in recent months:

- #1564
- #1569
- #1578
- #1581
- #1604
- #1612
- #1676
- #1692
- #1707
- #1708
- #1727
- #1753
- #1756
- #1766
- #1772
- #1786
- #1787
- #1788
- #1794
- #1806
- #1808
- #1848
- #1866

At this point, bugs in (2) are hard to fix, see e.g.
#1801.

This commit merges (2) into (1), thus removing all duplicate logic and
having (2) benefit from all the recent improvements to (1).

* Move firefox.rs to mozilla-central

* Reduce HttpServer trait functions

* Extract constructor

* Remove unused deps

* Remove clap color feature

Nice to have. Adds multiple dependencies. Hard to justify for mozilla-central.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants