Skip to content

WIP: experimental testing; do not merge! #9436

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 1 commit into from
Closed

WIP: experimental testing; do not merge! #9436

wants to merge 1 commit into from

Conversation

mcculls
Copy link
Contributor

@mcculls mcculls commented Aug 28, 2025

No description provided.

@mcculls mcculls added the tag: do not merge Do not merge changes label Aug 28, 2025
@datadog-datadog-prod-us1
Copy link
Contributor

datadog-datadog-prod-us1 bot commented Aug 28, 2025 •
edited
Loading

🎯 Code Coverage
• Patch Coverage: 0.00%
• Total Coverage: 59.16% (+1.52%)

View detailed report

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 531f83e | Docs | Was this helpful? Give us feedback!

@pr-commenter
Copy link

pr-commenter bot commented Aug 28, 2025 •
edited
Loading

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master mcculls/experiments
git_commit_date 1756404933 1756416797
git_commit_sha d07fee9 531f83e
release_version 1.53.0-SNAPSHOT~d07fee98fb 1.53.0-SNAPSHOT~531f83e0d3
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1756418684 1756418684
ci_job_id 1104525113 1104525113
ci_pipeline_id 75035086 75035086
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-0-qvq9zyqy 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-0-qvq9zyqy 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module Agent Agent
parent None None

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 45 metrics, 14 unstable metrics.

Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.53.0-SNAPSHOT~531f83e0d3, baseline=1.53.0-SNAPSHOT~d07fee98fb

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.046 s) : 0, 1045673
Total [baseline] (10.807 s) : 0, 10806511
Agent [candidate] (1.061 s) : 0, 1060842
Total [candidate] (10.771 s) : 0, 10770689
section appsec
Agent [baseline] (1.234 s) : 0, 1233567
Total [baseline] (10.907 s) : 0, 10907496
Agent [candidate] (1.242 s) : 0, 1242044
Total [candidate] (11.021 s) : 0, 11021397
section iast
Agent [baseline] (1.183 s) : 0, 1182793
Total [baseline] (10.993 s) : 0, 10993227
Agent [candidate] (1.204 s) : 0, 1203759
Total [candidate] (11.039 s) : 0, 11038871
section profiling
Agent [baseline] (1.202 s) : 0, 1201586
Total [baseline] (10.986 s) : 0, 10986002
Agent [candidate] (1.22 s) : 0, 1219727
Total [candidate] (11.058 s) : 0, 11058109
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.046 s -
Agent appsec 1.234 s 187.894 ms (18.0%)
Agent iast 1.183 s 137.12 ms (13.1%)
Agent profiling 1.202 s 155.914 ms (14.9%)
Total tracing 10.807 s -
Total appsec 10.907 s 100.986 ms (0.9%)
Total iast 10.993 s 186.716 ms (1.7%)
Total profiling 10.986 s 179.491 ms (1.7%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.061 s -
Agent appsec 1.242 s 181.203 ms (17.1%)
Agent iast 1.204 s 142.917 ms (13.5%)
Agent profiling 1.22 s 158.885 ms (15.0%)
Total tracing 10.771 s -
Total appsec 11.021 s 250.708 ms (2.3%)
Total iast 11.039 s 268.182 ms (2.5%)
Total profiling 11.058 s 287.42 ms (2.7%) 
gantt
    title petclinic - break down per module: candidate=1.53.0-SNAPSHOT~531f83e0d3, baseline=1.53.0-SNAPSHOT~d07fee98fb

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.451 ms) : 0, 1451
crashtracking [candidate] (1.47 ms) : 0, 1470
BytebuddyAgent [baseline] (733.371 ms) : 0, 733371
BytebuddyAgent [candidate] (744.496 ms) : 0, 744496
GlobalTracer [baseline] (242.894 ms) : 0, 242894
GlobalTracer [candidate] (242.338 ms) : 0, 242338
AppSec [baseline] (30.1 ms) : 0, 30100
AppSec [candidate] (30.257 ms) : 0, 30257
Debugger [baseline] (6.093 ms) : 0, 6093
Debugger [candidate] (6.093 ms) : 0, 6093
Remote Config [baseline] (669.715 µs) : 0, 670
Remote Config [candidate] (660.612 µs) : 0, 661
Telemetry [baseline] (10.017 ms) : 0, 10017
Telemetry [candidate] (14.509 ms) : 0, 14509
section appsec
crashtracking [baseline] (1.461 ms) : 0, 1461
crashtracking [candidate] (1.458 ms) : 0, 1458
BytebuddyAgent [baseline] (761.886 ms) : 0, 761886
BytebuddyAgent [candidate] (771.257 ms) : 0, 771257
GlobalTracer [baseline] (237.517 ms) : 0, 237517
GlobalTracer [candidate] (236.288 ms) : 0, 236288
AppSec [baseline] (170.994 ms) : 0, 170994
AppSec [candidate] (172.34 ms) : 0, 172340
Debugger [baseline] (7.366 ms) : 0, 7366
Debugger [candidate] (5.942 ms) : 0, 5942
Remote Config [baseline] (640.512 µs) : 0, 641
Remote Config [candidate] (712.815 µs) : 0, 713
Telemetry [baseline] (8.505 ms) : 0, 8505
Telemetry [candidate] (8.641 ms) : 0, 8641
IAST [baseline] (23.961 ms) : 0, 23961
IAST [candidate] (23.825 ms) : 0, 23825
section iast
crashtracking [baseline] (1.453 ms) : 0, 1453
crashtracking [candidate] (1.459 ms) : 0, 1459
BytebuddyAgent [baseline] (853.154 ms) : 0, 853154
BytebuddyAgent [candidate] (873.736 ms) : 0, 873736
GlobalTracer [baseline] (234.336 ms) : 0, 234336
GlobalTracer [candidate] (236.408 ms) : 0, 236408
AppSec [baseline] (28.666 ms) : 0, 28666
AppSec [candidate] (27.797 ms) : 0, 27797
Debugger [baseline] (6.602 ms) : 0, 6602
Debugger [candidate] (5.765 ms) : 0, 5765
Remote Config [baseline] (600.897 µs) : 0, 601
Remote Config [candidate] (602.685 µs) : 0, 603
Telemetry [baseline] (8.356 ms) : 0, 8356
Telemetry [candidate] (8.185 ms) : 0, 8185
IAST [baseline] (28.541 ms) : 0, 28541
IAST [candidate] (28.593 ms) : 0, 28593
section profiling
crashtracking [baseline] (1.472 ms) : 0, 1472
crashtracking [candidate] (1.473 ms) : 0, 1473
BytebuddyAgent [baseline] (763.981 ms) : 0, 763981
BytebuddyAgent [candidate] (780.821 ms) : 0, 780821
GlobalTracer [baseline] (222.805 ms) : 0, 222805
GlobalTracer [candidate] (223.403 ms) : 0, 223403
AppSec [baseline] (30.433 ms) : 0, 30433
AppSec [candidate] (30.402 ms) : 0, 30402
Debugger [baseline] (6.348 ms) : 0, 6348
Debugger [candidate] (10.856 ms) : 0, 10856
Remote Config [baseline] (730.475 µs) : 0, 730
Remote Config [candidate] (707.349 µs) : 0, 707
Telemetry [baseline] (16.462 ms) : 0, 16462
Telemetry [candidate] (10.251 ms) : 0, 10251
ProfilingAgent [baseline] (108.999 ms) : 0, 108999
ProfilingAgent [candidate] (111.597 ms) : 0, 111597
Profiling [baseline] (109.649 ms) : 0, 109649
Profiling [candidate] (112.253 ms) : 0, 112253
Loading
Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.53.0-SNAPSHOT~531f83e0d3, baseline=1.53.0-SNAPSHOT~d07fee98fb

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.063 s) : 0, 1063211
Total [baseline] (8.671 s) : 0, 8671401
Agent [candidate] (1.065 s) : 0, 1065496
Total [candidate] (8.69 s) : 0, 8689883
section iast
Agent [baseline] (1.202 s) : 0, 1202197
Total [baseline] (9.433 s) : 0, 9433027
Agent [candidate] (1.206 s) : 0, 1206063
Total [candidate] (9.409 s) : 0, 9408918
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.063 s -
Agent iast 1.202 s 138.987 ms (13.1%)
Total tracing 8.671 s -
Total iast 9.433 s 761.626 ms (8.8%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.065 s -
Agent iast 1.206 s 140.568 ms (13.2%)
Total tracing 8.69 s -
Total iast 9.409 s 719.035 ms (8.3%) 
gantt
    title insecure-bank - break down per module: candidate=1.53.0-SNAPSHOT~531f83e0d3, baseline=1.53.0-SNAPSHOT~d07fee98fb

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.498 ms) : 0, 1498
crashtracking [candidate] (1.461 ms) : 0, 1461
BytebuddyAgent [baseline] (743.592 ms) : 0, 743592
BytebuddyAgent [candidate] (750.521 ms) : 0, 750521
GlobalTracer [baseline] (246.024 ms) : 0, 246024
GlobalTracer [candidate] (242.318 ms) : 0, 242318
AppSec [baseline] (30.648 ms) : 0, 30648
AppSec [candidate] (30.419 ms) : 0, 30419
Debugger [baseline] (6.14 ms) : 0, 6140
Debugger [candidate] (6.117 ms) : 0, 6117
Remote Config [baseline] (688.992 µs) : 0, 689
Remote Config [candidate] (667.402 µs) : 0, 667
Telemetry [baseline] (13.248 ms) : 0, 13248
Telemetry [candidate] (12.975 ms) : 0, 12975
section iast
crashtracking [baseline] (1.488 ms) : 0, 1488
crashtracking [candidate] (1.471 ms) : 0, 1471
BytebuddyAgent [baseline] (870.517 ms) : 0, 870517
BytebuddyAgent [candidate] (877.66 ms) : 0, 877660
GlobalTracer [baseline] (237.482 ms) : 0, 237482
GlobalTracer [candidate] (234.998 ms) : 0, 234998
AppSec [baseline] (28.431 ms) : 0, 28431
AppSec [candidate] (27.033 ms) : 0, 27033
Debugger [baseline] (5.82 ms) : 0, 5820
Debugger [candidate] (5.743 ms) : 0, 5743
Remote Config [baseline] (639.415 µs) : 0, 639
Remote Config [candidate] (610.544 µs) : 0, 611
Telemetry [baseline] (8.511 ms) : 0, 8511
Telemetry [candidate] (8.137 ms) : 0, 8137
IAST [baseline] (28.019 ms) : 0, 28019
IAST [candidate] (28.378 ms) : 0, 28378
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master mcculls/experiments
git_commit_date 1756404933 1756416797
git_commit_sha d07fee9 531f83e
release_version 1.53.0-SNAPSHOT~d07fee98fb 1.53.0-SNAPSHOT~531f83e0d3
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1756418346 1756418346
ci_job_id 1104525115 1104525115
ci_pipeline_id 75035086 75035086
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-0-xy6bs0d2 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-0-xy6bs0d2 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 3 performance improvements and 2 performance regressions! Performance is the same for 7 metrics, 12 unstable metrics.

scenario Δ mean http_req_duration Δ mean throughput candidate mean http_req_duration candidate mean throughput baseline mean http_req_duration baseline mean throughput
scenario:load:insecure-bank:profiling:high_load better
[-520.940µs; -203.774µs] or [-5.712%; -2.234%]		 unstable
[-44.189op/s; +85.626op/s] or [-8.684%; +16.827%]		 8.758ms 529.594op/s 9.120ms 508.875op/s
scenario:load:insecure-bank:no_agent:high_load better
[-273.800µs; -167.695µs] or [-6.192%; -3.792%]		 unstable
[-61.849op/s; +167.787op/s] or [-5.957%; +16.160%]		 4.201ms 1091.250op/s 4.422ms 1038.281op/s
scenario:load:insecure-bank:iast_GLOBAL:high_load worse
[+662.181µs; +1054.023µs] or [+6.655%; +10.593%]		 unstable
[-88.436op/s; +15.124op/s] or [-18.946%; +3.240%]		 10.808ms 430.125op/s 9.950ms 466.781op/s
scenario:load:petclinic:appsec:high_load worse
[+1.536ms; +2.444ms] or [+3.347%; +5.325%]		 unstable
[-11.219op/s; +2.744op/s] or [-11.003%; +2.691%]		 47.884ms 97.725op/s 45.894ms 101.963op/s
scenario:load:petclinic:profiling:high_load better
[-2.386ms; -1.440ms] or [-4.757%; -2.872%]		 unstable
[-2.807op/s; +10.132op/s] or [-3.006%; +10.850%]		 48.231ms 97.037op/s 50.144ms 93.375op/s
Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.53.0-SNAPSHOT~531f83e0d3, baseline=1.53.0-SNAPSHOT~d07fee98fb
    dateFormat X
    axisFormat %s
section baseline
no_agent (4.422 ms) : 4373, 4471
.   : milestone, 4422,
iast (9.33 ms) : 9175, 9484
.   : milestone, 9330,
iast_FULL (14.096 ms) : 13815, 14377
.   : milestone, 14096,
iast_GLOBAL (9.95 ms) : 9778, 10121
.   : milestone, 9950,
profiling (9.12 ms) : 8960, 9280
.   : milestone, 9120,
tracing (7.695 ms) : 7585, 7805
.   : milestone, 7695,
section candidate
no_agent (4.201 ms) : 4152, 4250
.   : milestone, 4201,
iast (9.278 ms) : 9121, 9434
.   : milestone, 9278,
iast_FULL (14.129 ms) : 13849, 14409
.   : milestone, 14129,
iast_GLOBAL (10.808 ms) : 10616, 11000
.   : milestone, 10808,
profiling (8.758 ms) : 8624, 8892
.   : milestone, 8758,
tracing (7.907 ms) : 7793, 8021
.   : milestone, 7907,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 4.422 ms [4.373 ms, 4.471 ms] -
iast 9.33 ms [9.175 ms, 9.484 ms] 4.908 ms (111.0%)
iast_FULL 14.096 ms [13.815 ms, 14.377 ms] 9.674 ms (218.8%)
iast_GLOBAL 9.95 ms [9.778 ms, 10.121 ms] 5.528 ms (125.0%)
profiling 9.12 ms [8.96 ms, 9.28 ms] 4.698 ms (106.2%)
tracing 7.695 ms [7.585 ms, 7.805 ms] 3.273 ms (74.0%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 4.201 ms [4.152 ms, 4.25 ms] -
iast 9.278 ms [9.121 ms, 9.434 ms] 5.077 ms (120.8%)
iast_FULL 14.129 ms [13.849 ms, 14.409 ms] 9.927 ms (236.3%)
iast_GLOBAL 10.808 ms [10.616 ms, 11.0 ms] 6.607 ms (157.3%)
profiling 8.758 ms [8.624 ms, 8.892 ms] 4.556 ms (108.5%)
tracing 7.907 ms [7.793 ms, 8.021 ms] 3.706 ms (88.2%)
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.53.0-SNAPSHOT~531f83e0d3, baseline=1.53.0-SNAPSHOT~d07fee98fb
    dateFormat X
    axisFormat %s
section baseline
no_agent (37.525 ms) : 37224, 37826
.   : milestone, 37525,
appsec (45.894 ms) : 45476, 46313
.   : milestone, 45894,
code_origins (45.5 ms) : 45110, 45889
.   : milestone, 45500,
iast (44.733 ms) : 44357, 45110
.   : milestone, 44733,
profiling (50.144 ms) : 49685, 50604
.   : milestone, 50144,
tracing (43.957 ms) : 43587, 44328
.   : milestone, 43957,
section candidate
no_agent (36.722 ms) : 36418, 37027
.   : milestone, 36722,
appsec (47.884 ms) : 47459, 48310
.   : milestone, 47884,
code_origins (45.954 ms) : 45554, 46353
.   : milestone, 45954,
iast (45.249 ms) : 44867, 45630
.   : milestone, 45249,
profiling (48.231 ms) : 47813, 48649
.   : milestone, 48231,
tracing (44.442 ms) : 44067, 44818
.   : milestone, 44442,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 37.525 ms [37.224 ms, 37.826 ms] -
appsec 45.894 ms [45.476 ms, 46.313 ms] 8.37 ms (22.3%)
code_origins 45.5 ms [45.11 ms, 45.889 ms] 7.975 ms (21.3%)
iast 44.733 ms [44.357 ms, 45.11 ms] 7.209 ms (19.2%)
profiling 50.144 ms [49.685 ms, 50.604 ms] 12.62 ms (33.6%)
tracing 43.957 ms [43.587 ms, 44.328 ms] 6.432 ms (17.1%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 36.722 ms [36.418 ms, 37.027 ms] -
appsec 47.884 ms [47.459 ms, 48.31 ms] 11.162 ms (30.4%)
code_origins 45.954 ms [45.554 ms, 46.353 ms] 9.231 ms (25.1%)
iast 45.249 ms [44.867 ms, 45.63 ms] 8.526 ms (23.2%)
profiling 48.231 ms [47.813 ms, 48.649 ms] 11.509 ms (31.3%)
tracing 44.442 ms [44.067 ms, 44.818 ms] 7.72 ms (21.0%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master mcculls/experiments
git_commit_date 1756404933 1756416797
git_commit_sha d07fee9 531f83e
release_version 1.53.0-SNAPSHOT~d07fee98fb 1.53.0-SNAPSHOT~531f83e0d3
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1756418819 1756418819
ci_job_id 1104525117 1104525117
ci_pipeline_id 75035086 75035086
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-0-j1n88z3m 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-0-j1n88z3m 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics.

Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.53.0-SNAPSHOT~531f83e0d3, baseline=1.53.0-SNAPSHOT~d07fee98fb
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.473 ms) : 1462, 1485
.   : milestone, 1473,
appsec (3.602 ms) : 3388, 3816
.   : milestone, 3602,
iast (2.189 ms) : 2126, 2252
.   : milestone, 2189,
iast_GLOBAL (2.24 ms) : 2177, 2303
.   : milestone, 2240,
profiling (2.032 ms) : 1982, 2083
.   : milestone, 2032,
tracing (2.016 ms) : 1967, 2065
.   : milestone, 2016,
section candidate
no_agent (1.475 ms) : 1464, 1487
.   : milestone, 1475,
appsec (3.621 ms) : 3405, 3837
.   : milestone, 3621,
iast (2.195 ms) : 2132, 2257
.   : milestone, 2195,
iast_GLOBAL (2.24 ms) : 2177, 2303
.   : milestone, 2240,
profiling (2.04 ms) : 1989, 2091
.   : milestone, 2040,
tracing (2.015 ms) : 1967, 2064
.   : milestone, 2015,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.473 ms [1.462 ms, 1.485 ms] -
appsec 3.602 ms [3.388 ms, 3.816 ms] 2.128 ms (144.4%)
iast 2.189 ms [2.126 ms, 2.252 ms] 715.751 µs (48.6%)
iast_GLOBAL 2.24 ms [2.177 ms, 2.303 ms] 766.355 µs (52.0%)
profiling 2.032 ms [1.982 ms, 2.083 ms] 559.068 µs (37.9%)
tracing 2.016 ms [1.967 ms, 2.065 ms] 542.198 µs (36.8%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.475 ms [1.464 ms, 1.487 ms] -
appsec 3.621 ms [3.405 ms, 3.837 ms] 2.146 ms (145.4%)
iast 2.195 ms [2.132 ms, 2.257 ms] 719.432 µs (48.8%)
iast_GLOBAL 2.24 ms [2.177 ms, 2.303 ms] 764.402 µs (51.8%)
profiling 2.04 ms [1.989 ms, 2.091 ms] 564.708 µs (38.3%)
tracing 2.015 ms [1.967 ms, 2.064 ms] 540.051 µs (36.6%)
Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.53.0-SNAPSHOT~531f83e0d3, baseline=1.53.0-SNAPSHOT~d07fee98fb
    dateFormat X
    axisFormat %s
section baseline
no_agent (14.985 s) : 14985000, 14985000
.   : milestone, 14985000,
appsec (14.736 s) : 14736000, 14736000
.   : milestone, 14736000,
iast (18.23 s) : 18230000, 18230000
.   : milestone, 18230000,
iast_GLOBAL (17.778 s) : 17778000, 17778000
.   : milestone, 17778000,
profiling (15.206 s) : 15206000, 15206000
.   : milestone, 15206000,
tracing (14.981 s) : 14981000, 14981000
.   : milestone, 14981000,
section candidate
no_agent (15.388 s) : 15388000, 15388000
.   : milestone, 15388000,
appsec (14.817 s) : 14817000, 14817000
.   : milestone, 14817000,
iast (18.437 s) : 18437000, 18437000
.   : milestone, 18437000,
iast_GLOBAL (18.145 s) : 18145000, 18145000
.   : milestone, 18145000,
profiling (15.036 s) : 15036000, 15036000
.   : milestone, 15036000,
tracing (15.06 s) : 15060000, 15060000
.   : milestone, 15060000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 14.985 s [14.985 s, 14.985 s] -
appsec 14.736 s [14.736 s, 14.736 s] -249.0 ms (-1.7%)
iast 18.23 s [18.23 s, 18.23 s] 3.245 s (21.7%)
iast_GLOBAL 17.778 s [17.778 s, 17.778 s] 2.793 s (18.6%)
profiling 15.206 s [15.206 s, 15.206 s] 221.0 ms (1.5%)
tracing 14.981 s [14.981 s, 14.981 s] -4.0 ms (-0.0%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.388 s [15.388 s, 15.388 s] -
appsec 14.817 s [14.817 s, 14.817 s] -571.0 ms (-3.7%)
iast 18.437 s [18.437 s, 18.437 s] 3.049 s (19.8%)
iast_GLOBAL 18.145 s [18.145 s, 18.145 s] 2.757 s (17.9%)
profiling 15.036 s [15.036 s, 15.036 s] -352.0 ms (-2.3%)
tracing 15.06 s [15.06 s, 15.06 s] -328.0 ms (-2.1%)

@mcculls mcculls force-pushed the mcculls/experiments branch 3 times, most recently from acb2659 to e3275ff Compare August 28, 2025 19:20
@mcculls mcculls force-pushed the mcculls/experiments branch from e3275ff to 531f83e Compare August 28, 2025 21:35
@mcculls
Copy link
Contributor Author

mcculls commented Aug 29, 2025

Closing now that we've verified the generated glue approach works

@mcculls mcculls closed this Aug 29, 2025
@mcculls mcculls deleted the mcculls/experiments branch August 29, 2025 08:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
tag: do not merge Do not merge changes
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@mcculls