Skip to content

Conversation

mxinden
Copy link
Member

@mxinden mxinden commented Aug 15, 2025

Also to stay consistent with upcoming MASQUE connect-udp implementation. See Martin's comment in
#2796 (comment).

Also to stay consistent with upcoming MASQUE connect-udp implementation.
See Martin's comment in
mozilla#2796 (comment).
Copy link

codecov bot commented Aug 15, 2025

Codecov Report

❌ Patch coverage is 76.08696% with 11 lines in your changes missing coverage. Please review.
✅ Project coverage is 95.47%. Comparing base (c535de9) to head (1493d14).
⚠️ Report is 3 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2860      +/-   ##
==========================================
- Coverage   95.49%   95.47%   -0.03%     
==========================================
  Files         115      115              
  Lines       34439    34437       -2     
  Branches    34439    34437       -2     
==========================================
- Hits        32889    32879      -10     
- Misses       1543     1551       +8     
  Partials        7        7              
Components Coverage Δ
neqo-common 97.84% <ø> (ø)
neqo-crypto 89.77% <ø> (ø)
neqo-http3 94.46% <76.08%> (+<0.01%) ⬆️
neqo-qpack 95.90% <ø> (ø)
neqo-transport 96.49% <ø> (-0.06%) ⬇️
neqo-udp 91.17% <ø> (ø)

Copy link

github-actions bot commented Aug 15, 2025

Failed Interop Tests

QUIC Interop Runner, client vs. server, differences relative to 5723e5c.

neqo-latest as client

neqo-latest as server

All results

Succeeded Interop Tests

QUIC Interop Runner, client vs. server

neqo-latest as client

neqo-latest as server

Unsupported Interop Tests

QUIC Interop Runner, client vs. server

neqo-latest as client

neqo-latest as server

Copy link

github-actions bot commented Aug 15, 2025

Benchmark results

Performance differences relative to c535de9.

1-conn/1-100mb-resp/mtu-1504 (aka. Download)/client: 💚 Performance has improved.
       time:   [199.63 ms 199.94 ms 200.26 ms]
       thrpt:  [499.36 MiB/s 500.16 MiB/s 500.93 MiB/s]
change:
       time:   [−1.6416% −1.3400% −1.0658%] (p = 0.00 < 0.05)
       thrpt:  [+1.0773% +1.3582% +1.6690%]

Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high mild

1-conn/10_000-parallel-1b-resp/mtu-1504 (aka. RPS)/client: No change in performance detected.
       time:   [303.29 ms 304.85 ms 306.39 ms]
       thrpt:  [32.638 Kelem/s 32.803 Kelem/s 32.972 Kelem/s]
change:
       time:   [−0.9885% −0.2818% +0.4744%] (p = 0.46 > 0.05)
       thrpt:  [−0.4722% +0.2826% +0.9984%]
1-conn/1-1b-resp/mtu-1504 (aka. HPS)/client: No change in performance detected.
       time:   [28.126 ms 28.220 ms 28.323 ms]
       thrpt:  [35.307   B/s 35.436   B/s 35.554   B/s]
change:
       time:   [−0.7743% −0.1574% +0.4170%] (p = 0.60 > 0.05)
       thrpt:  [−0.4153% +0.1576% +0.7804%]

Found 2 outliers among 100 measurements (2.00%)
2 (2.00%) high severe

1-conn/1-100mb-req/mtu-1504 (aka. Upload)/client: 💚 Performance has improved.
       time:   [203.22 ms 203.52 ms 203.80 ms]
       thrpt:  [490.67 MiB/s 491.35 MiB/s 492.08 MiB/s]
change:
       time:   [−1.6825% −1.4945% −1.3079%] (p = 0.00 < 0.05)
       thrpt:  [+1.3252% +1.5172% +1.7113%]

Found 9 outliers among 100 measurements (9.00%)
2 (2.00%) low severe
4 (4.00%) low mild
3 (3.00%) high mild

decode 4096 bytes, mask ff: No change in performance detected.
       time:   [11.582 µs 11.607 µs 11.639 µs]
       change: [−1.8835% −0.5388% +0.5024%] (p = 0.46 > 0.05)

Found 13 outliers among 100 measurements (13.00%)
1 (1.00%) low severe
4 (4.00%) low mild
4 (4.00%) high mild
4 (4.00%) high severe

decode 1048576 bytes, mask ff: No change in performance detected.
       time:   [3.0607 ms 3.0706 ms 3.0818 ms]
       change: [−0.4935% −0.0184% +0.4973%] (p = 0.94 > 0.05)

Found 8 outliers among 100 measurements (8.00%)
8 (8.00%) high severe

decode 4096 bytes, mask 7f: No change in performance detected.
       time:   [19.381 µs 19.439 µs 19.506 µs]
       change: [−0.9762% −0.3251% +0.1888%] (p = 0.30 > 0.05)

Found 18 outliers among 100 measurements (18.00%)
3 (3.00%) low mild
1 (1.00%) high mild
14 (14.00%) high severe

decode 1048576 bytes, mask 7f: No change in performance detected.
       time:   [5.0915 ms 5.1049 ms 5.1200 ms]
       change: [−0.0943% +0.2619% +0.6231%] (p = 0.16 > 0.05)

Found 20 outliers among 100 measurements (20.00%)
1 (1.00%) high mild
19 (19.00%) high severe

decode 4096 bytes, mask 3f: No change in performance detected.
       time:   [5.5159 µs 5.5393 µs 5.5679 µs]
       change: [−0.8275% −0.3048% +0.1656%] (p = 0.25 > 0.05)

Found 13 outliers among 100 measurements (13.00%)
6 (6.00%) low mild
2 (2.00%) high mild
5 (5.00%) high severe

decode 1048576 bytes, mask 3f: Change within noise threshold.
       time:   [1.7580 ms 1.7627 ms 1.7689 ms]
       change: [+0.0061% +0.2801% +0.6269%] (p = 0.04 < 0.05)

Found 4 outliers among 100 measurements (4.00%)
1 (1.00%) high mild
3 (3.00%) high severe

coalesce_acked_from_zero 1+1 entries: No change in performance detected.
       time:   [88.309 ns 88.597 ns 88.875 ns]
       change: [−0.4975% −0.0486% +0.3878%] (p = 0.83 > 0.05)

Found 7 outliers among 100 measurements (7.00%)
6 (6.00%) high mild
1 (1.00%) high severe

coalesce_acked_from_zero 3+1 entries: No change in performance detected.
       time:   [106.09 ns 106.42 ns 106.78 ns]
       change: [−0.2183% +0.1550% +0.5700%] (p = 0.45 > 0.05)

Found 11 outliers among 100 measurements (11.00%)
11 (11.00%) high severe

coalesce_acked_from_zero 10+1 entries: No change in performance detected.
       time:   [105.60 ns 106.02 ns 106.53 ns]
       change: [−0.2590% +0.3262% +0.8630%] (p = 0.27 > 0.05)

Found 18 outliers among 100 measurements (18.00%)
5 (5.00%) low severe
1 (1.00%) low mild
3 (3.00%) high mild
9 (9.00%) high severe

coalesce_acked_from_zero 1000+1 entries: No change in performance detected.
       time:   [88.901 ns 96.999 ns 109.41 ns]
       change: [−0.6549% +4.2014% +11.049%] (p = 0.20 > 0.05)

Found 8 outliers among 100 measurements (8.00%)
1 (1.00%) high mild
7 (7.00%) high severe

RxStreamOrderer::inbound_frame(): Change within noise threshold.
       time:   [107.44 ms 107.50 ms 107.57 ms]
       change: [−0.4955% −0.3870% −0.2836%] (p = 0.00 < 0.05)

Found 11 outliers among 100 measurements (11.00%)
9 (9.00%) low mild
1 (1.00%) high mild
1 (1.00%) high severe

sent::Packets::take_ranges: :green_heart: Performance has improved.
       time:   [5.0975 µs 5.1669 µs 5.2274 µs]
       change: [−41.860% −35.738% −24.075%] (p = 0.00 < 0.05)

Found 2 outliers among 100 measurements (2.00%)
2 (2.00%) high severe

transfer/pacing-false/varying-seeds: No change in performance detected.
       time:   [36.501 ms 36.581 ms 36.661 ms]
       change: [−0.1396% +0.1602% +0.4719%] (p = 0.29 > 0.05)

Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high mild

transfer/pacing-true/varying-seeds: No change in performance detected.
       time:   [37.275 ms 37.380 ms 37.488 ms]
       change: [−0.5252% −0.1587% +0.2262%] (p = 0.42 > 0.05)

Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high mild

transfer/pacing-false/same-seed: Change within noise threshold.
       time:   [36.019 ms 36.091 ms 36.169 ms]
       change: [−2.5313% −2.2827% −1.9997%] (p = 0.00 < 0.05)

Found 2 outliers among 100 measurements (2.00%)
1 (1.00%) high mild
1 (1.00%) high severe

transfer/pacing-true/same-seed: No change in performance detected.
       time:   [37.811 ms 37.901 ms 37.997 ms]
       change: [−0.6114% −0.2679% +0.0854%] (p = 0.14 > 0.05)

Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high severe

Download data for profiler.firefox.com or download performance comparison data.

Copy link

github-actions bot commented Aug 15, 2025

Client/server transfer results

Performance differences relative to c535de9.

Transfer of 33554432 bytes over loopback, min. 100 runs. All unit-less numbers are in milliseconds.

Client vs. server (params) Mean ± σ Min Max MiB/s ± σ Δ main Δ main
google vs. google 452.8 ± 3.6 447.9 469.4 70.7 ± 8.9
google vs. neqo (cubic, paced) 271.9 ± 4.0 266.7 278.7 117.7 ± 8.0 💚 -2.2 -0.8%
msquic vs. msquic 132.3 ± 37.0 108.5 343.2 241.9 ± 0.9
msquic vs. neqo (cubic, paced) 154.7 ± 42.9 128.1 341.5 206.8 ± 0.7 -2.6 -1.6%
neqo vs. google (cubic, paced) 762.7 ± 5.3 752.2 778.1 42.0 ± 6.0 💔 4.1 0.5%
neqo vs. msquic (cubic, paced) 158.1 ± 5.6 149.7 182.0 202.3 ± 5.7 1.5 1.0%
neqo vs. neqo (cubic) 93.1 ± 4.0 85.1 101.5 343.8 ± 8.0 -1.1 -1.1%
neqo vs. neqo (cubic, paced) 91.0 ± 3.9 84.7 104.1 351.6 ± 8.2 💚 -3.2 -3.4%
neqo vs. neqo (reno) 92.4 ± 4.3 83.8 105.5 346.3 ± 7.4 💔 1.5 1.7%
neqo vs. neqo (reno, paced) 93.7 ± 4.9 86.2 113.5 341.7 ± 6.5 -0.3 -0.3%
neqo vs. quiche (cubic, paced) 193.8 ± 3.8 188.7 203.4 165.1 ± 8.4 💚 -1.4 -0.7%
neqo vs. s2n (cubic, paced) 221.4 ± 4.3 213.2 230.0 144.5 ± 7.4 0.2 0.1%
quiche vs. neqo (cubic, paced) 146.8 ± 5.0 136.6 157.8 217.9 ± 6.4 -1.0 -0.7%
quiche vs. quiche 146.5 ± 4.8 137.1 165.6 218.5 ± 6.7
s2n vs. neqo (cubic, paced) 168.2 ± 3.9 162.5 180.7 190.2 ± 8.2 💚 -2.7 -1.6%
s2n vs. s2n 248.4 ± 29.0 231.0 347.6 128.8 ± 1.1

Download data for profiler.firefox.com or download performance comparison data.

@mxinden mxinden enabled auto-merge August 16, 2025 12:21
@larseggert
Copy link
Collaborator

@mxinden is there anything we can do to improve coverage?

@larseggert larseggert disabled auto-merge August 18, 2025 05:29
@martinthomson
Copy link
Member

It looks like webtransport_session.rs is in dire need of some coverage attention.

@larseggert
Copy link
Collaborator

It looks like webtransport_session.rs is in dire need of some coverage attention.

I created #2879 for this. I will merge this one now.

@larseggert larseggert enabled auto-merge August 19, 2025 08:36
@mxinden
Copy link
Member Author

mxinden commented Aug 22, 2025

@larseggert whenever you have a minute, can you manually merge here?

@larseggert larseggert merged commit 1b16f83 into mozilla:main Aug 23, 2025
85 of 87 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants