Add an API to create a suspended process #5015

kevingosse · 2024-01-04T11:33:07Z

Summary of changes

On Windows, start the sample apps suspended to give time to attach procdump.

Reason for change

There was a small window of time where the sample could crash before procdump was launched.

Implementation details

There is a flag in the CreateProcess win32 API to create a process suspended, but .NET does not expose it. As a workaround, I vendored the code to create a process (and injected the flag). Then I use internal APIs though reflection to recreate an instance of System.Diagnostics.Process that maps the process that I manually started.

When vendoring the code, I removed two parts:

The stuff to start a process with a different user
The stuff to set the window style
So if we ever try to do that, it will fail silently and some people will be sad. But that seems extremely unlikely.

Test coverage

Given that it touches pretty much all integration tests, I'd say we have awesome coverage on this one.

datadog-ddstaging · 2024-01-04T12:03:53Z

Datadog Report

Branch report: kevin/create_suspended
Commit report: 86b40ba
Test service: dd-trace-dotnet

✅ 0 Failed, 304231 Passed, 996 Skipped, 44m 19.38s Wall Time
❄️ 1 New Flaky

New Flaky Tests (1)

TestIastPathTraversalRequest - Datadog.Trace.Security.IntegrationTests.Iast.AspNetCore5IastTestsFullSamplingIastEnabled - Last Failure

Expand for error

 Results do not match.
 Differences:
 Received: Iast.PathTraversal.AspNetCore5.IastEnabled.received.txt
 Verified: Iast.PathTraversal.AspNetCore5.IastEnabled.verified.txt
 Received Content:
 []
 Verified Content:
 [
   {
     TraceId: Id_1,
 ...

andrewlock · 2024-01-04T12:13:58Z

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

Welch test with statistical test for significance of 5%
Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5015) - mean (71ms)  : 63, 79
     .   : milestone, 71,
    master - mean (71ms)  : 62, 80
     .   : milestone, 71,

    section CallTarget+Inlining+NGEN
    This PR (5015) - mean (1,030ms)  : 1007, 1053
     .   : milestone, 1030,
    master - mean (1,023ms)  : 995, 1052
     .   : milestone, 1023,

gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5015) - mean (106ms)  : 104, 108
     .   : milestone, 106,
    master - mean (106ms)  : 102, 110
     .   : milestone, 106,

    section CallTarget+Inlining+NGEN
    This PR (5015) - mean (730ms)  : 712, 748
     .   : milestone, 730,
    master - mean (728ms)  : 704, 753
     .   : milestone, 728,

gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5015) - mean (90ms)  : 87, 94
     .   : milestone, 90,
    master - mean (90ms)  : 88, 93
     .   : milestone, 90,

    section CallTarget+Inlining+NGEN
    This PR (5015) - mean (693ms)  : 670, 716
     .   : milestone, 693,
    master - mean (688ms)  : 667, 708
     .   : milestone, 688,

gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5015) - mean (189ms)  : 186, 191
     .   : milestone, 189,
    master - mean (189ms)  : 186, 193
     .   : milestone, 189,

    section CallTarget+Inlining+NGEN
    This PR (5015) - mean (1,132ms)  : 1116, 1148
     .   : milestone, 1132,
    master - mean (1,132ms)  : 1110, 1153
     .   : milestone, 1132,

gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5015) - mean (272ms)  : 269, 275
     .   : milestone, 272,
    master - mean (273ms)  : 269, 277
     .   : milestone, 273,

    section CallTarget+Inlining+NGEN
    This PR (5015) - mean (1,078ms)  : 1054, 1103
     .   : milestone, 1078,
    master - mean (1,081ms)  : 1057, 1104
     .   : milestone, 1081,

gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5015) - mean (262ms)  : 258, 266
     .   : milestone, 262,
    master - mean (263ms)  : 259, 267
     .   : milestone, 263,

    section CallTarget+Inlining+NGEN
    This PR (5015) - mean (1,053ms)  : 1033, 1073
     .   : milestone, 1053,
    master - mean (1,054ms)  : 1028, 1080
     .   : milestone, 1054,

andrewlock · 2024-01-04T14:12:54Z

Throughput/Crank Report:zap:

Throughput results for AspNetCoreSimpleController comparing the following branches/commits:

Cases where throughput results for the PR are worse than latest master (5% drop or greater), results are shown in red.

Note that these results are based on a single point-in-time result for each branch. For full results, see one of the many, many dashboards!

gantt
    title Throughput Linux x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (5015) (11.108M)   : 0, 11107707
    master (11.261M)   : 0, 11261015
    benchmarks/2.9.0 (11.006M)   : 0, 11006202

    section Automatic
    This PR (5015) (7.537M)   : 0, 7537225
    master (7.663M)   : 0, 7663295
    benchmarks/2.9.0 (8.009M)   : 0, 8009058

    section Trace stats
    This PR (5015) (8.018M)   : 0, 8018200
    master (8.089M)   : 0, 8088714

    section Manual
    This PR (5015) (9.577M)   : 0, 9577274
    master (9.857M)   : 0, 9857347

    section Manual + Automatic
    This PR (5015) (7.488M)   : 0, 7487969
    master (7.367M)   : 0, 7366681

    section Version Conflict
    This PR (5015) (6.765M)   : 0, 6764676
    master (6.673M)   : 0, 6673172

gantt
    title Throughput Linux arm64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (5015) (9.562M)   : 0, 9561719
    master (9.537M)   : 0, 9536836
    benchmarks/2.9.0 (9.578M)   : 0, 9577814

    section Automatic
    This PR (5015) (6.699M)   : 0, 6699127
    master (6.679M)   : 0, 6679171

    section Trace stats
    This PR (5015) (6.689M)   : 0, 6688518
    master (6.711M)   : 0, 6710667

    section Manual
    This PR (5015) (8.179M)   : 0, 8178801
    master (8.368M)   : 0, 8368358

    section Manual + Automatic
    This PR (5015) (5.932M)   : crit ,0, 5932145
    master (6.325M)   : 0, 6324639

    section Version Conflict
    This PR (5015) (5.609M)   : 0, 5608592
    master (5.667M)   : 0, 5666623

gantt
    title Throughput Windows x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (5015) (10.764M)   : 0, 10764359
    master (9.953M)   : 0, 9953401
    benchmarks/2.9.0 (10.404M)   : 0, 10404037

    section Automatic
    This PR (5015) (7.445M)   : 0, 7444584
    master (7.140M)   : 0, 7140326
    benchmarks/2.9.0 (7.571M)   : 0, 7570768

    section Trace stats
    This PR (5015) (7.751M)   : 0, 7750523
    master (7.464M)   : 0, 7463828

    section Manual
    This PR (5015) (9.447M)   : 0, 9447166
    master (8.851M)   : 0, 8850660

    section Manual + Automatic
    This PR (5015) (7.182M)   : 0, 7182413
    master (6.879M)   : 0, 6879112

    section Version Conflict
    This PR (5015) (6.442M)   : 0, 6441588
    master (6.155M)   : 0, 6155439

gantt
    title Throughput Linux x64 (ASM) (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    master (7.410M)   : 0, 7410110
    benchmarks/2.9.0 (7.838M)   : 0, 7837824

    section No attack
    master (1.762M)   : 0, 1762310
    benchmarks/2.9.0 (3.205M)   : 0, 3204596

    section Attack
    master (1.415M)   : 0, 1414630
    benchmarks/2.9.0 (2.526M)   : 0, 2526337

    section Blocking
    master (3.135M)   : 0, 3135413

    section IAST default
    master (6.469M)   : 0, 6469433

    section IAST full
    master (5.815M)   : 0, 5815072

    section Base vuln
    master (0.929M)   : 0, 928828

    section IAST vuln
    master (0.852M)   : 0, 851756

andrewlock · 2024-01-04T16:55:54Z

Benchmarks Report 🐌

Benchmarks for #5015 compared to master:

1 benchmarks are faster, with geometric mean 1.337
5 benchmarks are slower, with geometric mean 1.210
1 benchmarks have more allocations

The following thresholds were used for comparing the benchmark speeds:

Mann–Whitney U test with statistical test for significance of 5%
Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	`StartStopWithChild`	net6.0	8.85μs	48.6ns	295ns	0.0214	0.00857	0	7.48 KB
master	`StartStopWithChild`	netcoreapp3.1	10.9μs	57.3ns	309ns	0.0273	0.0109	0	7.57 KB
master	`StartStopWithChild`	net472	16.9μs	52.7ns	204ns	1.36	0.375	0.128	7.96 KB
#5015	`StartStopWithChild`	net6.0	8.79μs	49.1ns	344ns	0.0263	0.0132	0	7.48 KB
#5015	`StartStopWithChild`	netcoreapp3.1	10.8μs	59.7ns	358ns	0.0215	0.0108	0	7.58 KB
#5015	`StartStopWithChild`	net472	17.3μs	49.1ns	190ns	1.35	0.344	0.112	7.96 KB

Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`WriteAndFlushEnrichedTraces`	net6.0	456μs	226ns	847ns	0	2.7 KB
master	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	636μs	120ns	464ns	0	2.7 KB
master	`WriteAndFlushEnrichedTraces`	net472	784μs	368ns	1.38μs	0.391	3.3 KB
#5015	`WriteAndFlushEnrichedTraces`	net6.0	455μs	227ns	849ns	0	2.7 KB
#5015	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	634μs	338ns	1.26μs	0	2.7 KB
#5015	`WriteAndFlushEnrichedTraces`	net472	785μs	199ns	771ns	0.391	3.3 KB

Benchmarks.Trace.Asm.AppSecBodyBenchmark - Slower ⚠️ Same allocations ✔️

Slower ⚠️ in #5015

Benchmark	diff/base	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.Asm.AppSecBodyBenchmark.ObjectExtractorSimpleBody‑net472	1.368	163.84	224.08
Benchmarks.Trace.Asm.AppSecBodyBenchmark.ObjectExtractorMoreComplexBody‑net472	1.160	3,727.45	4,322.80

Faster 🎉 in #5015

Benchmark	base/diff	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.Asm.AppSecBodyBenchmark.ObjectExtractorSimpleBody‑netcoreapp3.1	1.337	248.56	185.92

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`AllCycleSimpleBody`	net6.0	38.9μs	25.7ns	99.4ns	0.0194	0	1.77 KB
master	`AllCycleSimpleBody`	netcoreapp3.1	41.6μs	19.9ns	71.8ns	0.0207	0	1.74 KB
master	`AllCycleSimpleBody`	net472	44.9μs	16.9ns	65.4ns	0.287	0	1.81 KB
master	`AllCycleMoreComplexBody`	net6.0	200μs	147ns	569ns	0.0994	0	9.25 KB
master	`AllCycleMoreComplexBody`	netcoreapp3.1	211μs	223ns	864ns	0.105	0	9.14 KB
master	`AllCycleMoreComplexBody`	net472	225μs	94ns	352ns	1.46	0	9.32 KB
master	`ObjectExtractorSimpleBody`	net6.0	135ns	0.0635ns	0.237ns	0.00391	0	280 B
master	`ObjectExtractorSimpleBody`	netcoreapp3.1	249ns	0.171ns	0.662ns	0.00362	0	272 B
master	`ObjectExtractorSimpleBody`	net472	164ns	0.121ns	0.454ns	0.0446	0	281 B
master	`ObjectExtractorMoreComplexBody`	net6.0	3.01μs	1.09ns	4.08ns	0.0536	0	3.78 KB
master	`ObjectExtractorMoreComplexBody`	netcoreapp3.1	4.06μs	2.26ns	8.77ns	0.0508	0	3.69 KB
master	`ObjectExtractorMoreComplexBody`	net472	3.73μs	2.18ns	8.44ns	0.603	0.0056	3.8 KB
#5015	`AllCycleSimpleBody`	net6.0	38.5μs	38.4ns	149ns	0.0193	0	1.77 KB
#5015	`AllCycleSimpleBody`	netcoreapp3.1	42μs	46.2ns	179ns	0.0208	0	1.74 KB
#5015	`AllCycleSimpleBody`	net472	45.4μs	107ns	414ns	0.271	0	1.81 KB
#5015	`AllCycleMoreComplexBody`	net6.0	198μs	62.4ns	242ns	0.0989	0	9.25 KB
#5015	`AllCycleMoreComplexBody`	netcoreapp3.1	210μs	78.8ns	284ns	0.105	0	9.14 KB
#5015	`AllCycleMoreComplexBody`	net472	224μs	49.3ns	184ns	1.46	0	9.32 KB
#5015	`ObjectExtractorSimpleBody`	net6.0	134ns	0.0431ns	0.167ns	0.00393	0	280 B
#5015	`ObjectExtractorSimpleBody`	netcoreapp3.1	186ns	0.106ns	0.409ns	0.00373	0	272 B
#5015	`ObjectExtractorSimpleBody`	net472	224ns	0.103ns	0.386ns	0.0446	0	281 B
#5015	`ObjectExtractorMoreComplexBody`	net6.0	2.96μs	1.21ns	4.68ns	0.0533	0	3.78 KB
#5015	`ObjectExtractorMoreComplexBody`	netcoreapp3.1	4.16μs	0.744ns	2.78ns	0.0501	0	3.69 KB
#5015	`ObjectExtractorMoreComplexBody`	net472	4.32μs	1.28ns	4.78ns	0.603	0.00649	3.8 KB

Benchmarks.Trace.Asm.AppSecWafBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`RunWaf(args=NestedMap (10))`	net6.0	50.9μs	116ns	435ns	0.228	0	16.06 KB
master	`RunWaf(args=NestedMap (10))`	netcoreapp3.1	70.3μs	390ns	2.56μs	0.22	0	16.06 KB
master	`RunWaf(args=NestedMap (10))`	net472	96.5μs	43.8ns	164ns	2.54	0.0959	16.14 KB
master	`RunWafTwice(args=NestedMap (10))`	net6.0	53.7μs	118ns	486ns	0.211	0	16.6 KB
master	`RunWafTwice(args=NestedMap (10))`	netcoreapp3.1	71.1μs	185ns	783ns	0.225	0	16.58 KB
master	`RunWafTwice(args=NestedMap (10))`	net472	108μs	243ns	940ns	2.65	0.104	16.69 KB
master	`RunWafWithAttack(args=Neste(...)tack) [22])`	net6.0	108μs	35ns	135ns	0.27	0	22.41 KB
master	`RunWafWithAttack(args=Neste(...)tack) [22])`	netcoreapp3.1	131μs	615ns	2.54μs	0.272	0	22.36 KB
master	`RunWafWithAttack(args=Neste(...)tack) [22])`	net472	164μs	117ns	454ns	3.53	0.164	22.7 KB
master	`RunWaf(args=NestedMap (100))`	net6.0	109μs	618ns	4.33μs	0.444	0	32.76 KB
master	`RunWaf(args=NestedMap (100))`	netcoreapp3.1	135μs	675ns	3.16μs	0.399	0	33.33 KB
master	`RunWaf(args=NestedMap (100))`	net472	194μs	633ns	2.45μs	5.29	0.371	33.67 KB
master	`RunWafTwice(args=NestedMap (100))`	net6.0	111μs	401ns	1.55μs	0.458	0	33.3 KB
master	`RunWafTwice(args=NestedMap (100))`	netcoreapp3.1	139μs	723ns	3.68μs	0.413	0	33.86 KB
master	`RunWafTwice(args=NestedMap (100))`	net472	194μs	80ns	310ns	5.43	0.388	34.23 KB
master	`RunWafWithAttack(args=Neste(...)tack) [23])`	net6.0	156μs	41.4ns	160ns	0.548	0	39.1 KB
master	`RunWafWithAttack(args=Neste(...)tack) [23])`	netcoreapp3.1	198μs	156ns	603ns	0.494	0	39.63 KB
master	`RunWafWithAttack(args=Neste(...)tack) [23])`	net472	254μs	125ns	469ns	6.34	0.507	40.23 KB
master	`RunWaf(args=NestedMap (20))`	net6.0	103μs	556ns	3.15μs	0.441	0	32.18 KB
master	`RunWaf(args=NestedMap (20))`	netcoreapp3.1	132μs	718ns	4.18μs	0.447	0	32.3 KB
master	`RunWaf(args=NestedMap (20))`	net472	189μs	963ns	4.2μs	5.18	0.37	32.63 KB
master	`RunWafTwice(args=NestedMap (20))`	net6.0	106μs	620ns	5.58μs	0.457	0	32.72 KB
master	`RunWafTwice(args=NestedMap (20))`	netcoreapp3.1	141μs	720ns	3.3μs	0.402	0	32.82 KB
master	`RunWafTwice(args=NestedMap (20))`	net472	197μs	810ns	3.14μs	5.24	0.388	33.19 KB
master	`RunWafWithAttack(args=Neste(...)tack) [22])`	net6.0	156μs	129ns	447ns	0.543	0	38.53 KB
master	`RunWafWithAttack(args=Neste(...)tack) [22])`	netcoreapp3.1	198μs	803ns	3.11μs	0.489	0	38.6 KB
master	`RunWafWithAttack(args=Neste(...)tack) [22])`	net472	260μs	1.27μs	5.08μs	6.13	0.522	39.2 KB
#5015	`RunWaf(args=NestedMap (10))`	net6.0	50.1μs	10.5ns	40.5ns	0.226	0	16.06 KB
#5015	`RunWaf(args=NestedMap (10))`	netcoreapp3.1	69.9μs	395ns	2.85μs	0.202	0	16.06 KB
#5015	`RunWaf(args=NestedMap (10))`	net472	100μs	35.5ns	138ns	2.53	0.0953	16.14 KB
#5015	`RunWafTwice(args=NestedMap (10))`	net6.0	58μs	342ns	3.38μs	0.209	0	16.6 KB
#5015	`RunWafTwice(args=NestedMap (10))`	netcoreapp3.1	74.3μs	418ns	2.8μs	0.211	0	16.58 KB
#5015	`RunWafTwice(args=NestedMap (10))`	net472	104μs	61.3ns	237ns	2.65	0.104	16.69 KB
#5015	`RunWafWithAttack(args=Neste(...)tack) [22])`	net6.0	108μs	66.2ns	239ns	0.284	0	22.41 KB
#5015	`RunWafWithAttack(args=Neste(...)tack) [22])`	netcoreapp3.1	132μs	497ns	1.86μs	0.266	0	22.36 KB
#5015	`RunWafWithAttack(args=Neste(...)tack) [22])`	net472	164μs	805ns	3.32μs	3.58	0.159	22.7 KB
#5015	`RunWaf(args=NestedMap (100))`	net6.0	109μs	23.8ns	85.6ns	0.436	0	32.76 KB
#5015	`RunWaf(args=NestedMap (100))`	netcoreapp3.1	139μs	111ns	429ns	0.416	0	33.33 KB
#5015	`RunWaf(args=NestedMap (100))`	net472	187μs	101ns	378ns	5.29	0.371	33.67 KB
#5015	`RunWafTwice(args=NestedMap (100))`	net6.0	102μs	38.8ns	140ns	0.437	0	33.3 KB
#5015	`RunWafTwice(args=NestedMap (100))`	netcoreapp3.1	139μs	90ns	337ns	0.458	0	33.86 KB
#5015	`RunWafTwice(args=NestedMap (100))`	net472	193μs	61ns	220ns	5.43	0.388	34.23 KB
#5015	`RunWafWithAttack(args=Neste(...)tack) [23])`	net6.0	156μs	73ns	273ns	0.503	0	39.1 KB
#5015	`RunWafWithAttack(args=Neste(...)tack) [23])`	netcoreapp3.1	205μs	863ns	3.34μs	0.506	0	39.63 KB
#5015	`RunWafWithAttack(args=Neste(...)tack) [23])`	net472	257μs	202ns	782ns	6.33	0.527	40.23 KB
#5015	`RunWaf(args=NestedMap (20))`	net6.0	101μs	192ns	718ns	0.45	0	32.18 KB
#5015	`RunWaf(args=NestedMap (20))`	netcoreapp3.1	134μs	668ns	3.06μs	0.404	0	32.3 KB
#5015	`RunWaf(args=NestedMap (20))`	net472	193μs	1.05μs	5.83μs	5.12	0.372	32.63 KB
#5015	`RunWafTwice(args=NestedMap (20))`	net6.0	103μs	51.5ns	178ns	0.458	0	32.72 KB
#5015	`RunWafTwice(args=NestedMap (20))`	netcoreapp3.1	138μs	756ns	4.35μs	0.418	0	32.82 KB
#5015	`RunWafTwice(args=NestedMap (20))`	net472	194μs	124ns	479ns	5.25	0.389	33.19 KB
#5015	`RunWafWithAttack(args=Neste(...)tack) [22])`	net6.0	159μs	127ns	493ns	0.556	0	38.53 KB
#5015	`RunWafWithAttack(args=Neste(...)tack) [22])`	netcoreapp3.1	202μs	871ns	3.26μs	0.506	0	38.6 KB
#5015	`RunWafWithAttack(args=Neste(...)tack) [22])`	net472	254μs	161ns	602ns	6.21	0.507	39.2 KB

Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendRequest`	net6.0	170μs	132ns	511ns	0.17	18.25 KB
master	`SendRequest`	netcoreapp3.1	193μs	247ns	925ns	0.193	20.41 KB
master	`SendRequest`	net472	0.00135ns	0.000386ns	0.0015ns	0	0 b
#5015	`SendRequest`	net6.0	171μs	152ns	587ns	0.171	18.25 KB
#5015	`SendRequest`	netcoreapp3.1	191μs	294ns	1.14μs	0.189	20.41 KB
#5015	`SendRequest`	net472	0.000778ns	0.000309ns	0.0012ns	0	0 b

Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	`WriteAndFlushEnrichedTraces`	net6.0	538μs	2.33μs	8.4μs	0.532	0	0	41.51 KB
master	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	668μs	1.48μs	5.73μs	0.334	0	0	41.69 KB
master	`WriteAndFlushEnrichedTraces`	net472	820μs	3.95μs	16.8μs	8.39	2.52	0.419	53.23 KB
#5015	`WriteAndFlushEnrichedTraces`	net6.0	561μs	1.93μs	7.48μs	0.539	0	0	41.62 KB
#5015	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	648μs	785ns	3.04μs	0.329	0	0	41.72 KB
#5015	`WriteAndFlushEnrichedTraces`	net472	866μs	4.33μs	18.9μs	8.39	2.52	0.419	53.23 KB

Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`ExecuteNonQuery`	net6.0	1.08μs	0.671ns	2.6ns	0.0108	768 B
master	`ExecuteNonQuery`	netcoreapp3.1	1.48μs	0.812ns	3.14ns	0.0105	768 B
master	`ExecuteNonQuery`	net472	1.8μs	0.784ns	3.03ns	0.116	730 B
#5015	`ExecuteNonQuery`	net6.0	1.08μs	0.711ns	2.75ns	0.0108	768 B
#5015	`ExecuteNonQuery`	netcoreapp3.1	1.42μs	0.441ns	1.59ns	0.01	768 B
#5015	`ExecuteNonQuery`	net472	1.77μs	0.949ns	3.55ns	0.116	730 B

Benchmarks.Trace.ElasticsearchBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`CallElasticsearch`	net6.0	1.19μs	0.53ns	1.98ns	0.0131	936 B
master	`CallElasticsearch`	netcoreapp3.1	1.52μs	1.6ns	6ns	0.0129	936 B
master	`CallElasticsearch`	net472	2.47μs	0.425ns	1.59ns	0.151	955 B
master	`CallElasticsearchAsync`	net6.0	1.3μs	1.99ns	7.45ns	0.013	912 B
master	`CallElasticsearchAsync`	netcoreapp3.1	1.62μs	1.24ns	4.81ns	0.013	984 B
master	`CallElasticsearchAsync`	net472	2.74μs	1.03ns	4ns	0.16	1.01 KB
#5015	`CallElasticsearch`	net6.0	1.08μs	0.335ns	1.3ns	0.0131	936 B
#5015	`CallElasticsearch`	netcoreapp3.1	1.54μs	0.748ns	2.9ns	0.0123	936 B
#5015	`CallElasticsearch`	net472	2.49μs	1.63ns	6.32ns	0.152	955 B
#5015	`CallElasticsearchAsync`	net6.0	1.25μs	0.482ns	1.87ns	0.0125	912 B
#5015	`CallElasticsearchAsync`	netcoreapp3.1	1.71μs	1.12ns	4.18ns	0.0129	984 B
#5015	`CallElasticsearchAsync`	net472	2.76μs	0.689ns	2.58ns	0.16	1.01 KB

Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`ExecuteAsync`	net6.0	1.34μs	1.17ns	4.53ns	0.013	912 B
master	`ExecuteAsync`	netcoreapp3.1	1.69μs	0.641ns	2.48ns	0.0119	912 B
master	`ExecuteAsync`	net472	1.72μs	1.61ns	6.25ns	0.139	875 B
#5015	`ExecuteAsync`	net6.0	1.37μs	2.19ns	8.47ns	0.0123	912 B
#5015	`ExecuteAsync`	netcoreapp3.1	1.57μs	0.875ns	3.39ns	0.0119	912 B
#5015	`ExecuteAsync`	net472	1.8μs	1.23ns	4.75ns	0.138	875 B

Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendAsync`	net6.0	4.17μs	2.57ns	9.95ns	0.0288	2.1 KB
master	`SendAsync`	netcoreapp3.1	4.94μs	2.04ns	7.9ns	0.0347	2.63 KB
master	`SendAsync`	net472	7.62μs	4.02ns	15.6ns	0.522	3.31 KB
#5015	`SendAsync`	net6.0	4.21μs	1.14ns	4.13ns	0.0296	2.1 KB
#5015	`SendAsync`	netcoreapp3.1	5.01μs	1.59ns	5.96ns	0.0351	2.63 KB
#5015	`SendAsync`	net472	7.62μs	1.84ns	6.63ns	0.522	3.31 KB

Benchmarks.Trace.Iast.StringAspectsBenchmark - Same speed ✔️ More allocations ⚠️

More allocations ⚠️ in #5015

Benchmark	Base Allocated	Diff Allocated	Change	Change %
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark(parameters: System.Collections.Generic.List`1[System.String])‑net472	58.52 KB	59.7 KB	1.18 KB	2.01%

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Allocated
master	`StringConcatBenchmark(parameters=Syste(...)ring] [48])`	net6.0	55μs	283ns	1.3μs	43.44 KB
master	`StringConcatBenchmark(parameters=Syste(...)ring] [48])`	netcoreapp3.1	53μs	201ns	752ns	42.64 KB
master	`StringConcatBenchmark(parameters=Syste(...)ring] [48])`	net472	37.6μs	97.7ns	366ns	58.52 KB
master	`StringConcatAspectBenchmark(parameters=Syste(...)ring] [48])`	net6.0	65μs	250ns	936ns	43.29 KB
master	`StringConcatAspectBenchmark(parameters=Syste(...)ring] [48])`	netcoreapp3.1	101μs	3.88μs	38.4μs	42.64 KB
master	`StringConcatAspectBenchmark(parameters=Syste(...)ring] [48])`	net472	63.5μs	253ns	912ns	57.34 KB
#5015	`StringConcatBenchmark(parameters=Syste(...)ring] [48])`	net6.0	51.3μs	186ns	696ns	43.44 KB
#5015	`StringConcatBenchmark(parameters=Syste(...)ring] [48])`	netcoreapp3.1	54μs	288ns	1.53μs	42.64 KB
#5015	`StringConcatBenchmark(parameters=Syste(...)ring] [48])`	net472	37.8μs	60.5ns	218ns	59.7 KB
#5015	`StringConcatAspectBenchmark(parameters=Syste(...)ring] [48])`	net6.0	66.8μs	179ns	621ns	43.29 KB
#5015	`StringConcatAspectBenchmark(parameters=Syste(...)ring] [48])`	netcoreapp3.1	97.2μs	3.07μs	30.4μs	42.64 KB
#5015	`StringConcatAspectBenchmark(parameters=Syste(...)ring] [48])`	net472	63μs	311ns	1.28μs	57.34 KB

Benchmarks.Trace.ILoggerBenchmark - Slower ⚠️ Same allocations ✔️

Slower ⚠️ in #5015

Benchmark	diff/base	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.ILoggerBenchmark.EnrichedLog‑net6.0	1.127	1,436.85	1,618.96

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	1.44μs	0.494ns	1.91ns	0.0223	1.57 KB
master	`EnrichedLog`	netcoreapp3.1	2.33μs	2.12ns	7.94ns	0.0209	1.57 KB
master	`EnrichedLog`	net472	2.52μs	3.65ns	14.1ns	0.238	1.5 KB
#5015	`EnrichedLog`	net6.0	1.62μs	1.32ns	5.13ns	0.0221	1.57 KB
#5015	`EnrichedLog`	netcoreapp3.1	2.13μs	0.752ns	2.82ns	0.0212	1.57 KB
#5015	`EnrichedLog`	net472	2.54μs	3.81ns	14.8ns	0.237	1.5 KB

Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`EnrichedLog`	net6.0	112μs	124ns	463ns	0	0	4.21 KB
master	`EnrichedLog`	netcoreapp3.1	121μs	196ns	757ns	0.0602	0	4.21 KB
master	`EnrichedLog`	net472	150μs	134ns	517ns	0.671	0.224	4.39 KB
#5015	`EnrichedLog`	net6.0	114μs	217ns	841ns	0	0	4.21 KB
#5015	`EnrichedLog`	netcoreapp3.1	118μs	238ns	922ns	0	0	4.21 KB
#5015	`EnrichedLog`	net472	147μs	133ns	515ns	0.661	0.22	4.39 KB

Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	3.14μs	1.26ns	4.88ns	0.0294	2.13 KB
master	`EnrichedLog`	netcoreapp3.1	4.13μs	1.92ns	7.43ns	0.0289	2.13 KB
master	`EnrichedLog`	net472	4.91μs	2.43ns	9.41ns	0.308	1.95 KB
#5015	`EnrichedLog`	net6.0	2.94μs	1.06ns	4.1ns	0.0295	2.13 KB
#5015	`EnrichedLog`	netcoreapp3.1	4.19μs	1.39ns	5.02ns	0.0278	2.13 KB
#5015	`EnrichedLog`	net472	4.93μs	1.13ns	4.07ns	0.309	1.95 KB

Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendReceive`	net6.0	1.4μs	0.755ns	2.92ns	0.0154	1.1 KB
master	`SendReceive`	netcoreapp3.1	1.75μs	0.822ns	3.18ns	0.0149	1.1 KB
master	`SendReceive`	net472	2.13μs	3.25ns	12.6ns	0.177	1.12 KB
#5015	`SendReceive`	net6.0	1.39μs	1.89ns	7.32ns	0.0153	1.1 KB
#5015	`SendReceive`	netcoreapp3.1	1.77μs	0.612ns	2.29ns	0.0151	1.1 KB
#5015	`SendReceive`	net472	2.05μs	1.02ns	3.96ns	0.177	1.12 KB

Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	2.67μs	1.12ns	4.35ns	0.0214	1.53 KB
master	`EnrichedLog`	netcoreapp3.1	3.81μs	1.02ns	3.97ns	0.0207	1.58 KB
master	`EnrichedLog`	net472	4.29μs	0.997ns	3.59ns	0.311	1.97 KB
#5015	`EnrichedLog`	net6.0	2.58μs	1.53ns	5.94ns	0.0205	1.53 KB
#5015	`EnrichedLog`	netcoreapp3.1	4.01μs	1.25ns	4.69ns	0.02	1.58 KB
#5015	`EnrichedLog`	net472	4.25μs	1.84ns	7.14ns	0.31	1.97 KB

Benchmarks.Trace.SpanBenchmark - Slower ⚠️ Same allocations ✔️

Slower ⚠️ in #5015

Benchmark	diff/base	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.SpanBenchmark.StartFinishSpan‑net6.0	1.222	455.79	556.94

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`StartFinishSpan`	net6.0	456ns	0.347ns	1.3ns	0.00759	536 B
master	`StartFinishSpan`	netcoreapp3.1	712ns	0.272ns	1.05ns	0.0072	536 B
master	`StartFinishSpan`	net472	766ns	0.384ns	1.49ns	0.0852	538 B
master	`StartFinishScope`	net6.0	524ns	0.268ns	1ns	0.00918	656 B
master	`StartFinishScope`	netcoreapp3.1	852ns	0.461ns	1.73ns	0.00892	656 B
master	`StartFinishScope`	net472	964ns	1.79ns	6.93ns	0.0979	618 B
#5015	`StartFinishSpan`	net6.0	556ns	0.812ns	3.14ns	0.0075	536 B
#5015	`StartFinishSpan`	netcoreapp3.1	698ns	0.461ns	1.72ns	0.00733	536 B
#5015	`StartFinishSpan`	net472	747ns	0.982ns	3.8ns	0.0853	538 B
#5015	`StartFinishScope`	net6.0	528ns	0.268ns	1.04ns	0.00931	656 B
#5015	`StartFinishScope`	netcoreapp3.1	911ns	0.754ns	2.92ns	0.00889	656 B
#5015	`StartFinishScope`	net472	972ns	0.407ns	1.52ns	0.0982	618 B

Benchmarks.Trace.TraceAnnotationsBenchmark - Slower ⚠️ Same allocations ✔️

Slower ⚠️ in #5015

Benchmark	diff/base	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.TraceAnnotationsBenchmark.RunOnMethodBegin‑net6.0	1.187	573.54	680.52

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`RunOnMethodBegin`	net6.0	574ns	0.142ns	0.548ns	0.0094	656 B
master	`RunOnMethodBegin`	netcoreapp3.1	988ns	1.92ns	7.44ns	0.0088	656 B
master	`RunOnMethodBegin`	net472	1.1μs	0.99ns	3.83ns	0.098	618 B
#5015	`RunOnMethodBegin`	net6.0	680ns	0.331ns	1.28ns	0.00923	656 B
#5015	`RunOnMethodBegin`	netcoreapp3.1	996ns	0.299ns	1.16ns	0.00898	656 B
#5015	`RunOnMethodBegin`	net472	1.07μs	0.257ns	0.997ns	0.0982	618 B

…of sync over async

andrewlock

I reviewed all the irrelevant parts and glossed over the important implementation details. That's how code reviews are supposed to work, right? 🤔

tracer/test/Datadog.Trace.TestHelpers/MemoryDumpHelper.cs

andrewlock · 2024-01-05T10:16:53Z

tracer/test/Datadog.Trace.TestHelpers/ProfilerHelper.cs

-                MemoryDumpHelper.MonitorCrashes(process.Id);
+                using var suspendedProcess = NativeProcess.CreateProcess.StartSuspendedProcess(startInfo);
+
+                MemoryDumpHelper.MonitorCrashes(suspendedProcess.Id).Wait();


andrewlock · 2024-01-05T10:18:43Z

tracer/test/Datadog.Trace.TestHelpers/NativeProcess/SuspendedProcess.cs

+            throw new ObjectDisposedException(nameof(SuspendedProcess));
+        }
+
+        _isResumed = true;


Should we check against resuming twice? do we care?

Meh. We're grown-ups.

Speak for yourself

andrewlock · 2024-01-05T10:19:13Z

tracer/test/Datadog.Trace.TestHelpers/NativeProcess/NativeMethods.cs

+    internal const int ERROR_EXE_MACHINE_TYPE_MISMATCH = 216;
+
+    [DllImport("kernel32.dll", CharSet = CharSet.Ansi, SetLastError = true, BestFitMapping = false)]
+    internal static extern bool DuplicateHandle(


Yup, this all looks correct :hide:

Actually the CharSet and BestFitMapping are useless here. Good catch.

lucaspimentel · 2024-01-09T17:34:44Z

tracer/test/Datadog.Trace.TestHelpers/ProfilerHelper.cs

            {
-                MemoryDumpHelper.MonitorCrashes(process.Id);
+                using var suspendedProcess = NativeProcess.CreateProcess.StartSuspendedProcess(startInfo);


Is it worth having a toggle switch to enable/disable this?
(I expect the answer to be "what's the point?")

Let's just add it when/if we need it 😛

zacharycmontoya · 2024-01-10T01:19:20Z

tracer/test/Datadog.Trace.TestHelpers/NativeProcess/CreateProcess.cs

+
+    internal static SuspendedProcess StartSuspendedProcess(ProcessStartInfo startInfo)
+    {
+        // See knowledge base article Q190351 for an explanation of the following code.  Noteworthy tricky points:


Is there a link to the article?

Oh. No, that's copy/pasted from Microsoft's code

zacharycmontoya

I really glossed over the files in the NativeProcess folder, but I trust that this works 👍🏼

Add an API to create a suspended process

e334edd

github-actions bot added the area:tests unit tests, integration tests label Jan 4, 2024

Fix field names on netfx

060c53d

Kevin Gosse added 2 commits January 4, 2024 13:41

Looks like the process needs the handle

795abad

Fix compilation error

c933f1b

Wait for procdump to be ready (make it sync for now)

1aede78

kevingosse marked this pull request as ready for review January 4, 2024 16:45

kevingosse requested a review from a team as a code owner January 4, 2024 16:45

Move the MemoryDumpHelper to a dedicated thread to reduce the impact …

4385a65

…of sync over async

andrewlock approved these changes Jan 5, 2024

View reviewed changes

Kevin Gosse added 4 commits January 5, 2024 11:43

Address PR feedback

133478a

Fix race condition

6b48e6d

Try stuff

3095be3

Download procdump only once

93f7707

kevingosse force-pushed the kevin/create_suspended branch from 93f7707 to beee0d1 Compare January 8, 2024 11:27

Use the Process global lock

621229b

kevingosse force-pushed the kevin/create_suspended branch from beee0d1 to 621229b Compare January 8, 2024 11:56

Kevin Gosse added 6 commits January 8, 2024 14:11

Fix warning

d8c479d

The name of the field is different on netfx

bca5695

Try fixing the procdump issue

01fdb1c

Try increasing the timeout

092147b

Use the right version of procdump

76a024e

That's the one

86b40ba

lucaspimentel reviewed Jan 9, 2024

View reviewed changes

lucaspimentel approved these changes Jan 9, 2024

View reviewed changes

zacharycmontoya reviewed Jan 10, 2024

View reviewed changes

zacharycmontoya approved these changes Jan 10, 2024

View reviewed changes

kevingosse merged commit 2232939 into master Jan 10, 2024

kevingosse deleted the kevin/create_suspended branch January 10, 2024 13:48

github-actions bot added this to the vNext milestone Jan 10, 2024

Add an API to create a suspended process #5015

Add an API to create a suspended process #5015

Uh oh!

Conversation

kevingosse commented Jan 4, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary of changes

Reason for change

Implementation details

Test coverage

Uh oh!

datadog-ddstaging bot commented Jan 4, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Datadog Report

New Flaky Tests (1)

Uh oh!

andrewlock commented Jan 4, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Execution-Time Benchmarks Report ⏱️

Uh oh!

andrewlock commented Jan 4, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Throughput/Crank Report:zap:

Uh oh!

andrewlock commented Jan 4, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks Report 🐌

Benchmark details

Raw results

Raw results

Slower ⚠️ in #5015

Faster 🎉 in #5015

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

More allocations ⚠️ in #5015

Raw results

Slower ⚠️ in #5015

Raw results

Raw results

Raw results

Raw results

Raw results

Slower ⚠️ in #5015

Raw results

Slower ⚠️ in #5015

Raw results

Uh oh!

andrewlock left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

zacharycmontoya left a comment

Choose a reason for hiding this comment

Uh oh!

kevingosse commented Jan 4, 2024 •

edited

Loading

datadog-ddstaging bot commented Jan 4, 2024 •

edited

Loading

andrewlock commented Jan 4, 2024 •

edited

Loading

andrewlock commented Jan 4, 2024 •

edited

Loading

andrewlock commented Jan 4, 2024 •

edited

Loading