Antithesis poc #9993
Draft
Antithesis poc #9993
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
BenchmarksStartupParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 54 metrics, 11 unstable metrics. Startup time reports for insecure-bankgantt
title insecure-bank - global startup overhead: candidate=1.56.0-SNAPSHOT~18d42b5fc3, baseline=1.57.0-SNAPSHOT~cebe04f7d3
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.086 s) : 0, 1085964
Total [baseline] (8.721 s) : 0, 8721041
Agent [candidate] (1.081 s) : 0, 1080601
Total [candidate] (8.905 s) : 0, 8904609
section iast
Agent [baseline] (1.23 s) : 0, 1230291
Total [baseline] (9.368 s) : 0, 9367668
Agent [candidate] (1.218 s) : 0, 1217805
Total [candidate] (9.502 s) : 0, 9502254
gantt
title insecure-bank - break down per module: candidate=1.56.0-SNAPSHOT~18d42b5fc3, baseline=1.57.0-SNAPSHOT~cebe04f7d3
dateFormat X
axisFormat %s
section tracing
crashtracking [baseline] (1.197 ms) : 0, 1197
crashtracking [candidate] (1.194 ms) : 0, 1194
BytebuddyAgent [baseline] (651.688 ms) : 0, 651688
BytebuddyAgent [candidate] (647.47 ms) : 0, 647470
GlobalTracer [baseline] (284.295 ms) : 0, 284295
GlobalTracer [candidate] (282.678 ms) : 0, 282678
AppSec [baseline] (32.583 ms) : 0, 32583
AppSec [candidate] (32.332 ms) : 0, 32332
Debugger [baseline] (67.193 ms) : 0, 67193
Debugger [candidate] (67.636 ms) : 0, 67636
Remote Config [baseline] (654.588 µs) : 0, 655
Remote Config [candidate] (644.728 µs) : 0, 645
Telemetry [baseline] (8.964 ms) : 0, 8964
Telemetry [candidate] (9.179 ms) : 0, 9179
Flare Poller [baseline] (3.704 ms) : 0, 3704
Flare Poller [candidate] (3.841 ms) : 0, 3841
section iast
crashtracking [baseline] (1.21 ms) : 0, 1210
crashtracking [candidate] (1.215 ms) : 0, 1215
BytebuddyAgent [baseline] (795.701 ms) : 0, 795701
BytebuddyAgent [candidate] (787.346 ms) : 0, 787346
GlobalTracer [baseline] (257.23 ms) : 0, 257230
GlobalTracer [candidate] (255.541 ms) : 0, 255541
IAST [baseline] (27.205 ms) : 0, 27205
IAST [candidate] (28.707 ms) : 0, 28707
AppSec [baseline] (33.227 ms) : 0, 33227
AppSec [candidate] (33.199 ms) : 0, 33199
Debugger [baseline] (67.617 ms) : 0, 67617
Debugger [candidate] (63.764 ms) : 0, 63764
Remote Config [baseline] (599.868 µs) : 0, 600
Remote Config [candidate] (641.531 µs) : 0, 642
Telemetry [baseline] (8.397 ms) : 0, 8397
Telemetry [candidate] (8.375 ms) : 0, 8375
Flare Poller [baseline] (3.477 ms) : 0, 3477
Flare Poller [candidate] (3.509 ms) : 0, 3509
Startup time reports for petclinicgantt
title petclinic - global startup overhead: candidate=1.56.0-SNAPSHOT~18d42b5fc3, baseline=1.57.0-SNAPSHOT~cebe04f7d3
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.081 s) : 0, 1081386
Total [baseline] (10.799 s) : 0, 10799293
Agent [candidate] (1.084 s) : 0, 1083863
Total [candidate] (10.911 s) : 0, 10911273
section appsec
Agent [baseline] (1.27 s) : 0, 1270263
Total [baseline] (11.128 s) : 0, 11127730
Agent [candidate] (1.27 s) : 0, 1269911
Total [candidate] (11.28 s) : 0, 11280262
section iast
Agent [baseline] (1.217 s) : 0, 1217495
Total [baseline] (11.283 s) : 0, 11283141
Agent [candidate] (1.22 s) : 0, 1220270
Total [candidate] (11.239 s) : 0, 11238865
section profiling
Agent [baseline] (1.205 s) : 0, 1204709
Total [baseline] (10.994 s) : 0, 10993960
Agent [candidate] (1.203 s) : 0, 1203487
Total [candidate] (11.132 s) : 0, 11131887
gantt
title petclinic - break down per module: candidate=1.56.0-SNAPSHOT~18d42b5fc3, baseline=1.57.0-SNAPSHOT~cebe04f7d3
dateFormat X
axisFormat %s
section tracing
crashtracking [baseline] (1.189 ms) : 0, 1189
crashtracking [candidate] (1.191 ms) : 0, 1191
BytebuddyAgent [baseline] (648.197 ms) : 0, 648197
BytebuddyAgent [candidate] (648.412 ms) : 0, 648412
GlobalTracer [baseline] (282.494 ms) : 0, 282494
GlobalTracer [candidate] (284.273 ms) : 0, 284273
AppSec [baseline] (32.426 ms) : 0, 32426
AppSec [candidate] (32.628 ms) : 0, 32628
Debugger [baseline] (67.415 ms) : 0, 67415
Debugger [candidate] (68.303 ms) : 0, 68303
Remote Config [baseline] (635.576 µs) : 0, 636
Remote Config [candidate] (624.358 µs) : 0, 624
Telemetry [baseline] (9.749 ms) : 0, 9749
Telemetry [candidate] (9.055 ms) : 0, 9055
Flare Poller [baseline] (3.71 ms) : 0, 3710
Flare Poller [candidate] (3.726 ms) : 0, 3726
section appsec
crashtracking [baseline] (1.19 ms) : 0, 1190
crashtracking [candidate] (1.197 ms) : 0, 1197
BytebuddyAgent [baseline] (693.289 ms) : 0, 693289
BytebuddyAgent [candidate] (692.328 ms) : 0, 692328
GlobalTracer [baseline] (260.119 ms) : 0, 260119
GlobalTracer [candidate] (259.945 ms) : 0, 259945
IAST [baseline] (24.835 ms) : 0, 24835
IAST [candidate] (24.848 ms) : 0, 24848
AppSec [baseline] (171.556 ms) : 0, 171556
AppSec [candidate] (175.645 ms) : 0, 175645
Debugger [baseline] (69.865 ms) : 0, 69865
Debugger [candidate] (66.522 ms) : 0, 66522
Remote Config [baseline] (709.462 µs) : 0, 709
Remote Config [candidate] (723.137 µs) : 0, 723
Telemetry [baseline] (9.109 ms) : 0, 9109
Telemetry [candidate] (8.97 ms) : 0, 8970
Flare Poller [baseline] (3.849 ms) : 0, 3849
Flare Poller [candidate] (3.861 ms) : 0, 3861
section iast
crashtracking [baseline] (1.176 ms) : 0, 1176
crashtracking [candidate] (1.196 ms) : 0, 1196
BytebuddyAgent [baseline] (786.325 ms) : 0, 786325
BytebuddyAgent [candidate] (788.648 ms) : 0, 788648
GlobalTracer [baseline] (254.807 ms) : 0, 254807
GlobalTracer [candidate] (255.748 ms) : 0, 255748
IAST [baseline] (27.051 ms) : 0, 27051
IAST [candidate] (30.433 ms) : 0, 30433
AppSec [baseline] (33.766 ms) : 0, 33766
AppSec [candidate] (31.623 ms) : 0, 31623
Debugger [baseline] (66.609 ms) : 0, 66609
Debugger [candidate] (64.771 ms) : 0, 64771
Remote Config [baseline] (596.71 µs) : 0, 597
Remote Config [candidate] (586.819 µs) : 0, 587
Telemetry [baseline] (8.478 ms) : 0, 8478
Telemetry [candidate] (8.412 ms) : 0, 8412
Flare Poller [baseline] (3.454 ms) : 0, 3454
Flare Poller [candidate] (3.458 ms) : 0, 3458
section profiling
crashtracking [baseline] (1.221 ms) : 0, 1221
crashtracking [candidate] (1.251 ms) : 0, 1251
BytebuddyAgent [baseline] (701.106 ms) : 0, 701106
BytebuddyAgent [candidate] (701.048 ms) : 0, 701048
GlobalTracer [baseline] (220.905 ms) : 0, 220905
GlobalTracer [candidate] (221.598 ms) : 0, 221598
AppSec [baseline] (32.01 ms) : 0, 32010
AppSec [candidate] (32.042 ms) : 0, 32042
Debugger [baseline] (68.996 ms) : 0, 68996
Debugger [candidate] (67.751 ms) : 0, 67751
Remote Config [baseline] (633.402 µs) : 0, 633
Remote Config [candidate] (635.44 µs) : 0, 635
Telemetry [baseline] (8.892 ms) : 0, 8892
Telemetry [candidate] (8.602 ms) : 0, 8602
Flare Poller [baseline] (3.718 ms) : 0, 3718
Flare Poller [candidate] (3.712 ms) : 0, 3712
ProfilingAgent [baseline] (97.482 ms) : 0, 97482
ProfilingAgent [candidate] (96.915 ms) : 0, 96915
Profiling [baseline] (98.065 ms) : 0, 98065
Profiling [candidate] (97.516 ms) : 0, 97516
LoadParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 20 metrics, 16 unstable metrics. Request duration reports for insecure-bankgantt
title insecure-bank - request duration [CI 0.99] : candidate=1.56.0-SNAPSHOT~18d42b5fc3, baseline=1.57.0-SNAPSHOT~cebe04f7d3
dateFormat X
axisFormat %s
section baseline
no_agent (1.191 ms) : 1179, 1203
. : milestone, 1191,
iast (3.096 ms) : 3054, 3138
. : milestone, 3096,
iast_FULL (5.757 ms) : 5699, 5814
. : milestone, 5757,
iast_GLOBAL (3.559 ms) : 3507, 3610
. : milestone, 3559,
profiling (2.05 ms) : 2032, 2068
. : milestone, 2050,
tracing (1.746 ms) : 1732, 1760
. : milestone, 1746,
section candidate
no_agent (1.282 ms) : 1268, 1297
. : milestone, 1282,
iast (3.19 ms) : 3150, 3229
. : milestone, 3190,
iast_FULL (5.846 ms) : 5788, 5903
. : milestone, 5846,
iast_GLOBAL (3.502 ms) : 3451, 3554
. : milestone, 3502,
profiling (1.988 ms) : 1972, 2005
. : milestone, 1988,
tracing (1.792 ms) : 1778, 1806
. : milestone, 1792,
Request duration reports for petclinicgantt
title petclinic - request duration [CI 0.99] : candidate=1.56.0-SNAPSHOT~18d42b5fc3, baseline=1.57.0-SNAPSHOT~cebe04f7d3
dateFormat X
axisFormat %s
section baseline
no_agent (18.378 ms) : 18190, 18566
. : milestone, 18378,
appsec (18.617 ms) : 18425, 18810
. : milestone, 18617,
code_origins (17.799 ms) : 17625, 17974
. : milestone, 17799,
iast (17.869 ms) : 17695, 18043
. : milestone, 17869,
profiling (18.764 ms) : 18579, 18948
. : milestone, 18764,
tracing (18.034 ms) : 17856, 18212
. : milestone, 18034,
section candidate
no_agent (19.186 ms) : 18990, 19383
. : milestone, 19186,
appsec (18.65 ms) : 18461, 18839
. : milestone, 18650,
code_origins (17.721 ms) : 17546, 17896
. : milestone, 17721,
iast (17.744 ms) : 17564, 17923
. : milestone, 17744,
profiling (18.557 ms) : 18370, 18743
. : milestone, 18557,
tracing (18.286 ms) : 18097, 18475
. : milestone, 18286,
DacapoParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 10 metrics, 2 unstable metrics. Execution time for biojavagantt
title biojava - execution time [CI 0.99] : candidate=1.56.0-SNAPSHOT~18d42b5fc3, baseline=1.57.0-SNAPSHOT~cebe04f7d3
dateFormat X
axisFormat %s
section baseline
no_agent (15.003 s) : 15003000, 15003000
. : milestone, 15003000,
appsec (14.467 s) : 14467000, 14467000
. : milestone, 14467000,
iast (18.109 s) : 18109000, 18109000
. : milestone, 18109000,
iast_GLOBAL (17.64 s) : 17640000, 17640000
. : milestone, 17640000,
profiling (15.302 s) : 15302000, 15302000
. : milestone, 15302000,
tracing (14.611 s) : 14611000, 14611000
. : milestone, 14611000,
section candidate
no_agent (14.747 s) : 14747000, 14747000
. : milestone, 14747000,
appsec (14.877 s) : 14877000, 14877000
. : milestone, 14877000,
iast (18.203 s) : 18203000, 18203000
. : milestone, 18203000,
iast_GLOBAL (17.725 s) : 17725000, 17725000
. : milestone, 17725000,
profiling (14.676 s) : 14676000, 14676000
. : milestone, 14676000,
tracing (14.509 s) : 14509000, 14509000
. : milestone, 14509000,
Execution time for tomcatgantt
title tomcat - execution time [CI 0.99] : candidate=1.56.0-SNAPSHOT~18d42b5fc3, baseline=1.57.0-SNAPSHOT~cebe04f7d3
dateFormat X
axisFormat %s
section baseline
no_agent (1.473 ms) : 1462, 1485
. : milestone, 1473,
appsec (3.714 ms) : 3493, 3935
. : milestone, 3714,
iast (2.21 ms) : 2146, 2274
. : milestone, 2210,
iast_GLOBAL (2.251 ms) : 2186, 2316
. : milestone, 2251,
profiling (2.47 ms) : 2257, 2684
. : milestone, 2470,
tracing (2.037 ms) : 1986, 2088
. : milestone, 2037,
section candidate
no_agent (1.475 ms) : 1464, 1487
. : milestone, 1475,
appsec (3.681 ms) : 3466, 3896
. : milestone, 3681,
iast (2.212 ms) : 2147, 2277
. : milestone, 2212,
iast_GLOBAL (2.262 ms) : 2197, 2327
. : milestone, 2262,
profiling (2.054 ms) : 2002, 2106
. : milestone, 2054,
tracing (2.05 ms) : 1999, 2101
. : milestone, 2050,
|
6 tasks
This comment has been minimized.
This comment has been minimized.
bric3
reviewed
Nov 25, 2025
Comment on lines
152
to
158
| ObjectNode httpResponseDetails = JsonNodeFactory.instance.objectNode(); | ||
| httpResponseDetails.put("trace_count", payload.traceCount()); | ||
| httpResponseDetails.put("payload_size_bytes", sizeInBytes); | ||
| httpResponseDetails.put("http_status", response.code()); | ||
| httpResponseDetails.put("http_message", response.message()); | ||
| httpResponseDetails.put("success", response.code() == 200); | ||
| httpResponseDetails.put("agent_url", tracesUrl.toString()); |
Contributor
There was a problem hiding this comment.
thought: I'm very uncomfortable with using jackson API in these parts. And the Assert methods rely on it.
Comment on lines
175
to
178
| log.debug("ANTITHESIS_ASSERT: Non-200 HTTP response (unreachable) - code: {}, message: {}, traces: {}", response.code(), response.message(), payload.traceCount()); | ||
| Assert.unreachable( | ||
| "Non-200 HTTP response from agent indicates API failure - traces may be lost", | ||
| errorDetails); |
Contributor
There was a problem hiding this comment.
thought: Another thing is the human cost to maintain these assertions.
bric3
reviewed
Nov 25, 2025
| implementation(project(":internal-api")) | ||
|
|
||
| // Antithesis SDK for assertions and property testing - bundled in tracer JAR | ||
| implementation(group = "com.antithesis", name = "sdk", version = "1.4.5") |
Contributor
There was a problem hiding this comment.
thought: Another concern, is how much more weight it adds to the jar.
Contributor
There was a problem hiding this comment.
Right now this PR adds ~20mb to the final jar... but a lot of that is because it embeds antithesis, its native FFI wrapper, and various jackson dependencies multiple times in each of the product directories in the agent jar.
If we move it to the shared section in dd-java-agent/build.gradle and add the necessary excludes to gradle/dependencies.gradle like we do for other shared dependencies then the overhead is 3mb compressed and 8mb uncompressed.
That still feels too big to have in the general deliverable for something only used for testing purposes.
One option might be to only include the direct dependency in the release (i.e. without jackson or the ffi wrapper.) - in other words just enough to allow the classes to load. We'd then have to look at how to combine the other parts for testing, whether that's via -Xbootclasspath/a: or something similar.
Contributor
Author
There was a problem hiding this comment.
Thank you very much for your analysis. Right now, the size of the Jar "doesn't matter much" because it's just a PoC, but when I write the final evaluation report, I think the data you provide is really very important, and I will include it in that report as something critical to consider if it is decided to adopt this Antithesis API.
Debugger benchmarksParameters
See matching parameters
SummaryFound 3 performance improvements and 5 performance regressions! Performance is the same for 2 metrics, 5 unstable metrics.
See unchanged results
Request duration reports for reportsgantt
title reports - request duration [CI 0.99] : candidate=None, baseline=None
dateFormat X
axisFormat %s
section baseline
noprobe (304.624 µs) : 281, 328
. : milestone, 305,
basic (287.669 µs) : 282, 293
. : milestone, 288,
loop (9.534 ms) : 9404, 9664
. : milestone, 9534,
section candidate
noprobe (313.793 µs) : 289, 339
. : milestone, 314,
basic (277.983 µs) : 272, 284
. : milestone, 278,
loop (10.054 ms) : 9945, 10163
. : milestone, 10054,
|
- Add assertions to CoreTracer.write() to track sampling decisions - Add assertions to RemoteWriter.write() to track buffer drops - Add assertions to PayloadDispatcherImpl.accept() to track send success/failure This provides clear tracking of: - trace_accepted_by_sampling: Traces that passed sampling - trace_dropped_by_sampling: Traces dropped by sampling - trace_enqueued_for_send: Traces accepted into buffer - trace_dropped_buffer_overflow: Traces dropped due to full buffer - trace_dropped_by_policy: Traces dropped by policy - trace_dropped_writer_closed: Traces dropped during shutdown - trace_payloads_being_sent: All send attempts - traces_sent_successfully: Traces successfully sent to agent - traces_failed_to_send: Traces that failed HTTP send
robertomonteromiguel
force-pushed
the
robertomonteromiguel/antithesis_poc_java_tracer
branch
from
ac89cad to
136a766
Compare
robertomonteromiguel
temporarily deployed
to
ci-build
GitHub Actions
Inactive
…-config, and telemetry Removed assertions from: - telemetry/DependencyResolver.java - profiling-uploader/ProfileUploader.java - profiling/ProfilingAgent.java - profiling-ddprof/DatadogProfiler.java - profiling-controller-openjdk/OpenJdkController.java - profiling-controller/ProfilingSystem.java - agent-debugger/ConfigurationUpdater.java - remote-config-core/DefaultConfigurationPoller.java Also removed obsolete documentation and scripts. Now only the new trace loss tracking assertions remain in: - CoreTracer.write() - RemoteWriter.write() - PayloadDispatcherImpl.accept()
Replaced all java.util.Map.of() calls with HashMap initialization to support Java 8 compilation. Map.of() was introduced in Java 9. Fixed in: - CoreTracer.java - RemoteWriter.java - PayloadDispatcherImpl.java
robertomonteromiguel
temporarily deployed
to
ci-build
GitHub Actions
Inactive
The Antithesis SDK expects Jackson ObjectNode, not Map. Replaced HashMap with ObjectNode using JsonNodeFactory. Fixed in: - CoreTracer.java - RemoteWriter.java - PayloadDispatcherImpl.java
robertomonteromiguel
temporarily deployed
to
ci-build
GitHub Actions
Inactive
robertomonteromiguel
temporarily deployed
to
ci-build
GitHub Actions
Inactive
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What Does This Do
Add Antithesis assertions in the tracer
Motivation
Additional Notes
Contributor Checklist
type:and (comp:orinst:) labels in addition to any useful labelsclose,fixor any linking keywords when referencing an issue.Use
solvesinstead, and assign the PR milestone to the issueJira ticket: [PROJ-IDENT]