Skip to content

Use versioned docker images in CI #5963

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
smola merged 3 commits into from
Oct 4, 2023
Merged

Use versioned docker images in CI #5963

smola merged 3 commits into from
Oct 4, 2023

Conversation

@smola
Copy link
Member

@smola smola commented Sep 29, 2023

What Does This Do

Nightly builds use latest versions, everything else use a pinned version.

Motivation

  • Ensure we can build old release branches in CI.
  • Avoid unexpected problems with new JDK versions.

Additional Notes

For the image build and tagging side, see DataDog/dd-trace-java-docker-build#72

@smola
Use versioned docker images in CI
35cd3a9 
Nightly builds use latest versions, everything else use a pinned
version.
@smola smola added comp: testing Testing tag: no release notes Changes to exclude from release notes labels Sep 29, 2023
@smola smola requested a review from a team as a code owner September 29, 2023 10:50
@pr-commenter
Copy link

pr-commenter bot commented Sep 29, 2023 •
edited
Loading

Benchmarks

Startup

Parameters

Baseline Candidate
commit 1.22.0-SNAPSHOT~d4ca13f3e4 1.22.0-SNAPSHOT~851d375989
config baseline candidate
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
module Agent Agent
parent None None
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 62 cases.

Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.22.0-SNAPSHOT~851d375989, baseline=1.22.0-SNAPSHOT~d4ca13f3e4

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.019 s) : 0, 1018514
Total [baseline] (8.687 s) : 0, 8686977
Agent [candidate] (1.013 s) : 0, 1012666
Total [candidate] (8.665 s) : 0, 8664606
section appsec
Agent [baseline] (1.101 s) : 0, 1100711
Total [baseline] (8.752 s) : 0, 8752377
Agent [candidate] (1.101 s) : 0, 1101460
Total [candidate] (8.783 s) : 0, 8783054
section iast
Agent [baseline] (1.122 s) : 0, 1122267
Total [baseline] (9.206 s) : 0, 9205913
Agent [candidate] (1.123 s) : 0, 1122879
Total [candidate] (9.214 s) : 0, 9214162
section profiling
Agent [baseline] (1.182 s) : 0, 1182228
Total [baseline] (8.902 s) : 0, 8902344
Agent [candidate] (1.183 s) : 0, 1183141
Total [candidate] (8.865 s) : 0, 8865013
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.019 s -
Agent appsec 1.101 s 82.198 ms (8.1%)
Agent iast 1.122 s 103.754 ms (10.2%)
Agent profiling 1.182 s 163.715 ms (16.1%)
Total tracing 8.687 s -
Total appsec 8.752 s 65.401 ms (0.8%)
Total iast 9.206 s 518.937 ms (6.0%)
Total profiling 8.902 s 215.367 ms (2.5%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.013 s -
Agent appsec 1.101 s 88.794 ms (8.8%)
Agent iast 1.123 s 110.213 ms (10.9%)
Agent profiling 1.183 s 170.475 ms (16.8%)
Total tracing 8.665 s -
Total appsec 8.783 s 118.448 ms (1.4%)
Total iast 9.214 s 549.556 ms (6.3%)
Total profiling 8.865 s 200.408 ms (2.3%) 
gantt
    title insecure-bank - break down per module: candidate=1.22.0-SNAPSHOT~851d375989, baseline=1.22.0-SNAPSHOT~d4ca13f3e4

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (634.012 ms) : 0, 634012
BytebuddyAgent [candidate] (629.779 ms) : 0, 629779
GlobalTracer [baseline] (294.047 ms) : 0, 294047
GlobalTracer [candidate] (293.13 ms) : 0, 293130
AppSec [baseline] (49.145 ms) : 0, 49145
AppSec [candidate] (48.935 ms) : 0, 48935
Remote Config [baseline] (665.35 µs) : 0, 665
Remote Config [candidate] (656.199 µs) : 0, 656
Telemetry [baseline] (6.051 ms) : 0, 6051
Telemetry [candidate] (6.023 ms) : 0, 6023
section appsec
BytebuddyAgent [baseline] (629.928 ms) : 0, 629928
BytebuddyAgent [candidate] (629.98 ms) : 0, 629980
GlobalTracer [baseline] (292.504 ms) : 0, 292504
GlobalTracer [candidate] (293.145 ms) : 0, 293145
AppSec [baseline] (137.494 ms) : 0, 137494
AppSec [candidate] (137.723 ms) : 0, 137723
Remote Config [baseline] (634.475 µs) : 0, 634
Remote Config [candidate] (634.432 µs) : 0, 634
Telemetry [baseline] (5.733 ms) : 0, 5733
Telemetry [candidate] (5.738 ms) : 0, 5738
section iast
BytebuddyAgent [baseline] (743.682 ms) : 0, 743682
BytebuddyAgent [candidate] (743.987 ms) : 0, 743987
GlobalTracer [baseline] (277.086 ms) : 0, 277086
GlobalTracer [candidate] (277.445 ms) : 0, 277445
AppSec [baseline] (46.31 ms) : 0, 46310
AppSec [candidate] (46.235 ms) : 0, 46235
Remote Config [baseline] (559.028 µs) : 0, 559
Remote Config [candidate] (582.078 µs) : 0, 582
Telemetry [baseline] (5.792 ms) : 0, 5792
Telemetry [candidate] (5.839 ms) : 0, 5839
IAST [baseline] (14.475 ms) : 0, 14475
IAST [candidate] (14.509 ms) : 0, 14509
section profiling
BytebuddyAgent [baseline] (639.171 ms) : 0, 639171
BytebuddyAgent [candidate] (640.843 ms) : 0, 640843
GlobalTracer [baseline] (353.904 ms) : 0, 353904
GlobalTracer [candidate] (353.519 ms) : 0, 353519
AppSec [baseline] (49.219 ms) : 0, 49219
AppSec [candidate] (49.108 ms) : 0, 49108
Remote Config [baseline] (670.099 µs) : 0, 670
Remote Config [candidate] (663.348 µs) : 0, 663
Telemetry [baseline] (6.055 ms) : 0, 6055
Telemetry [candidate] (6.04 ms) : 0, 6040
ProfilingAgent [baseline] (80.195 ms) : 0, 80195
ProfilingAgent [candidate] (79.895 ms) : 0, 79895
Profiling [baseline] (80.219 ms) : 0, 80219
Profiling [candidate] (79.919 ms) : 0, 79919
Loading
Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.22.0-SNAPSHOT~851d375989, baseline=1.22.0-SNAPSHOT~d4ca13f3e4

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.019 s) : 0, 1019116
Total [baseline] (9.255 s) : 0, 9255060
Agent [candidate] (1.014 s) : 0, 1014082
Total [candidate] (9.235 s) : 0, 9235079
section appsec
Agent [baseline] (1.099 s) : 0, 1099227
Total [baseline] (9.256 s) : 0, 9256467
Agent [candidate] (1.117 s) : 0, 1117399
Total [candidate] (9.336 s) : 0, 9336259
section iast
Agent [baseline] (1.119 s) : 0, 1119480
Total [baseline] (9.373 s) : 0, 9373406
Agent [candidate] (1.121 s) : 0, 1121154
Total [candidate] (9.374 s) : 0, 9374426
section profiling
Agent [baseline] (1.184 s) : 0, 1183766
Total [baseline] (9.44 s) : 0, 9439611
Agent [candidate] (1.186 s) : 0, 1185933
Total [candidate] (9.498 s) : 0, 9497845
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.019 s -
Agent appsec 1.099 s 80.111 ms (7.9%)
Agent iast 1.119 s 100.364 ms (9.8%)
Agent profiling 1.184 s 164.65 ms (16.2%)
Total tracing 9.255 s -
Total appsec 9.256 s 1.407 ms (0.0%)
Total iast 9.373 s 118.345 ms (1.3%)
Total profiling 9.44 s 184.55 ms (2.0%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.014 s -
Agent appsec 1.117 s 103.317 ms (10.2%)
Agent iast 1.121 s 107.072 ms (10.6%)
Agent profiling 1.186 s 171.851 ms (16.9%)
Total tracing 9.235 s -
Total appsec 9.336 s 101.18 ms (1.1%)
Total iast 9.374 s 139.347 ms (1.5%)
Total profiling 9.498 s 262.766 ms (2.8%) 
gantt
    title petclinic - break down per module: candidate=1.22.0-SNAPSHOT~851d375989, baseline=1.22.0-SNAPSHOT~d4ca13f3e4

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (634.296 ms) : 0, 634296
BytebuddyAgent [candidate] (630.557 ms) : 0, 630557
GlobalTracer [baseline] (294.374 ms) : 0, 294374
GlobalTracer [candidate] (293.333 ms) : 0, 293333
AppSec [baseline] (49.244 ms) : 0, 49244
AppSec [candidate] (49.183 ms) : 0, 49183
Remote Config [baseline] (671.095 µs) : 0, 671
Remote Config [candidate] (661.965 µs) : 0, 662
Telemetry [baseline] (6.032 ms) : 0, 6032
Telemetry [candidate] (6.017 ms) : 0, 6017
section appsec
BytebuddyAgent [baseline] (629.478 ms) : 0, 629478
BytebuddyAgent [candidate] (640.191 ms) : 0, 640191
GlobalTracer [baseline] (291.528 ms) : 0, 291528
GlobalTracer [candidate] (296.684 ms) : 0, 296684
AppSec [baseline] (137.582 ms) : 0, 137582
AppSec [candidate] (139.196 ms) : 0, 139196
Remote Config [baseline] (636.081 µs) : 0, 636
Remote Config [candidate] (658.265 µs) : 0, 658
Telemetry [baseline] (5.741 ms) : 0, 5741
Telemetry [candidate] (5.847 ms) : 0, 5847
section iast
BytebuddyAgent [baseline] (741.578 ms) : 0, 741578
BytebuddyAgent [candidate] (742.983 ms) : 0, 742983
GlobalTracer [baseline] (276.728 ms) : 0, 276728
GlobalTracer [candidate] (277.065 ms) : 0, 277065
AppSec [baseline] (46.231 ms) : 0, 46231
AppSec [candidate] (46.237 ms) : 0, 46237
Remote Config [baseline] (558.041 µs) : 0, 558
Remote Config [candidate] (547.739 µs) : 0, 548
Telemetry [baseline] (5.755 ms) : 0, 5755
Telemetry [candidate] (5.733 ms) : 0, 5733
IAST [baseline] (14.319 ms) : 0, 14319
IAST [candidate] (14.269 ms) : 0, 14269
section profiling
BytebuddyAgent [baseline] (640.801 ms) : 0, 640801
BytebuddyAgent [candidate] (642.033 ms) : 0, 642033
GlobalTracer [baseline] (353.243 ms) : 0, 353243
GlobalTracer [candidate] (354.183 ms) : 0, 354183
AppSec [baseline] (49.448 ms) : 0, 49448
AppSec [candidate] (49.478 ms) : 0, 49478
Remote Config [baseline] (680.833 µs) : 0, 681
Remote Config [candidate] (657.208 µs) : 0, 657
Telemetry [baseline] (6.075 ms) : 0, 6075
Telemetry [candidate] (6.11 ms) : 0, 6110
ProfilingAgent [baseline] (80.509 ms) : 0, 80509
ProfilingAgent [candidate] (80.325 ms) : 0, 80325
Profiling [baseline] (80.534 ms) : 0, 80534
Profiling [candidate] (80.349 ms) : 0, 80349
Loading

Load

Parameters

Baseline Candidate
commit 1.22.0-SNAPSHOT~d4ca13f3e4 1.22.0-SNAPSHOT~851d375989
config baseline candidate
end_time 2023-10-03T06:57:01 2023-10-03T07:15:10
start_time 2023-10-03T06:56:43 2023-10-03T07:14:52
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 24 cases.

Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.22.0-SNAPSHOT~851d375989, baseline=1.22.0-SNAPSHOT~d4ca13f3e4
    dateFormat X
    axisFormat %s
section baseline
no_agent (365.076 µs) : 344, 386
.   : milestone, 365,
appsec (692.383 µs) : 672, 713
.   : milestone, 692,
iast (461.356 µs) : 440, 483
.   : milestone, 461,
iast_FULL (516.94 µs) : 496, 538
.   : milestone, 517,
iast_INACTIVE (432.068 µs) : 411, 453
.   : milestone, 432,
profiling (440.542 µs) : 420, 461
.   : milestone, 441,
tracing (433.81 µs) : 413, 455
.   : milestone, 434,
section candidate
no_agent (363.428 µs) : 344, 383
.   : milestone, 363,
appsec (699.404 µs) : 679, 720
.   : milestone, 699,
iast (464.75 µs) : 444, 486
.   : milestone, 465,
iast_FULL (527.414 µs) : 506, 549
.   : milestone, 527,
iast_INACTIVE (435.96 µs) : 415, 457
.   : milestone, 436,
profiling (447.558 µs) : 426, 469
.   : milestone, 448,
tracing (435.92 µs) : 415, 457
.   : milestone, 436,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 365.076 µs [344.24 µs, 385.911 µs] -
appsec 692.383 µs [671.948 µs, 712.818 µs] 327.307 µs (89.7%)
iast 461.356 µs [440.097 µs, 482.614 µs] 96.28 µs (26.4%)
iast_FULL 516.94 µs [496.236 µs, 537.644 µs] 151.865 µs (41.6%)
iast_INACTIVE 432.068 µs [411.069 µs, 453.066 µs] 66.992 µs (18.4%)
profiling 440.542 µs [420.283 µs, 460.801 µs] 75.467 µs (20.7%)
tracing 433.81 µs [412.737 µs, 454.883 µs] 68.735 µs (18.8%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 363.428 µs [343.688 µs, 383.168 µs] -
appsec 699.404 µs [679.127 µs, 719.681 µs] 335.976 µs (92.4%)
iast 464.75 µs [443.804 µs, 485.697 µs] 101.322 µs (27.9%)
iast_FULL 527.414 µs [506.286 µs, 548.542 µs] 163.986 µs (45.1%)
iast_INACTIVE 435.96 µs [414.539 µs, 457.382 µs] 72.533 µs (20.0%)
profiling 447.558 µs [425.943 µs, 469.173 µs] 84.13 µs (23.1%)
tracing 435.92 µs [414.579 µs, 457.262 µs] 72.492 µs (19.9%)
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.22.0-SNAPSHOT~851d375989, baseline=1.22.0-SNAPSHOT~d4ca13f3e4
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.314 ms) : 1295, 1334
.   : milestone, 1314,
appsec (1.711 ms) : 1687, 1735
.   : milestone, 1711,
iast (1.48 ms) : 1457, 1504
.   : milestone, 1480,
profiling (1.46 ms) : 1435, 1484
.   : milestone, 1460,
tracing (1.464 ms) : 1440, 1488
.   : milestone, 1464,
section candidate
no_agent (1.335 ms) : 1316, 1355
.   : milestone, 1335,
appsec (1.696 ms) : 1672, 1721
.   : milestone, 1696,
iast (1.454 ms) : 1430, 1478
.   : milestone, 1454,
profiling (1.464 ms) : 1439, 1488
.   : milestone, 1464,
tracing (1.458 ms) : 1433, 1482
.   : milestone, 1458,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.314 ms [1.295 ms, 1.334 ms] -
appsec 1.711 ms [1.687 ms, 1.735 ms] 396.912 µs (30.2%)
iast 1.48 ms [1.457 ms, 1.504 ms] 166.157 µs (12.6%)
profiling 1.46 ms [1.435 ms, 1.484 ms] 145.267 µs (11.1%)
tracing 1.464 ms [1.44 ms, 1.488 ms] 149.387 µs (11.4%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.335 ms [1.316 ms, 1.355 ms] -
appsec 1.696 ms [1.672 ms, 1.721 ms] 361.282 µs (27.1%)
iast 1.454 ms [1.43 ms, 1.478 ms] 118.973 µs (8.9%)
profiling 1.464 ms [1.439 ms, 1.488 ms] 128.744 µs (9.6%)
tracing 1.458 ms [1.433 ms, 1.482 ms] 122.723 µs (9.2%)

smola
smola commented Sep 29, 2023
View reviewed changes
@smola smola merged commit 153fd96 into master Oct 4, 2023
@smola smola deleted the smola/versioned-docker-ci-images branch October 4, 2023 07:47
@github-actions github-actions bot added this to the 1.22.0 milestone Oct 4, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@mcculls mcculls mcculls approved these changes

@nayeem-kamal nayeem-kamal Awaiting requested review from nayeem-kamal nayeem-kamal was automatically assigned from DataDog/apm-java

Assignees

No one assigned

Labels

comp: testing Testing tag: no release notes Changes to exclude from release notes

Projects

None yet

Milestone

1.22.0

Development

Successfully merging this pull request may close these issues.

3 participants

@smola @mcculls