Skip to content

Commit ab17339

Browse files
github-actions[bot]thomhurst
github-actions[bot]
and
thomhurst
authored
Update README.md (#2643)
Co-authored-by: thomhurst <[email protected]>
1 parent a68ecda commit ab17339

File tree

1 file changed

+73
-76
lines changed

1 file changed

+73
-76
lines changed

‎README.md‎

Lines changed: 73 additions & 76 deletions
Original file line numberDiff line numberDiff line change
@@ -431,19 +431,19 @@ dotnet add package TUnit --prerelease
431431
432432
BenchmarkDotNet v0.15.2, macOS Sonoma 14.7.6 (23H626) [Darwin 23.6.0]
433433
Apple M1 (Virtual), 1 CPU, 3 logical and 3 physical cores
434-
.NET SDK 9.0.301
435-
[Host] : .NET 9.0.6 (9.0.625.26613), Arm64 RyuJIT AdvSIMD
436-
.NET 9.0 : .NET 9.0.6 (9.0.625.26613), Arm64 RyuJIT AdvSIMD
434+
.NET SDK 9.0.302
435+
[Host] : .NET 9.0.7 (9.0.725.31616), Arm64 RyuJIT AdvSIMD
436+
.NET 9.0 : .NET 9.0.7 (9.0.725.31616), Arm64 RyuJIT AdvSIMD
437437
438438
Job=.NET 9.0 Runtime=.NET 9.0
439439
440440
```
441-
| Method | Mean | Error | StdDev | Median |
442-
|------------- |-----------:|---------:|----------:|-----------:|
443-
| Build_TUnit | 1,197.4 ms | 87.87 ms | 249.28 ms | 1,105.6 ms |
444-
| Build_NUnit | 799.6 ms | 9.50 ms | 7.93 ms | 798.9 ms |
445-
| Build_xUnit | 781.1 ms | 13.08 ms | 10.93 ms | 778.8 ms |
446-
| Build_MSTest | 850.8 ms | 16.72 ms | 26.52 ms | 841.9 ms |
441+
| Method | Mean | Error | StdDev |
442+
|------------- |---------:|---------:|---------:|
443+
| Build_TUnit | 996.3 ms | 19.88 ms | 48.75 ms |
444+
| Build_NUnit | 823.5 ms | 14.06 ms | 24.26 ms |
445+
| Build_xUnit | 772.0 ms | 8.66 ms | 7.23 ms |
446+
| Build_MSTest | 846.3 ms | 16.79 ms | 23.54 ms |
447447

448448

449449

@@ -453,19 +453,19 @@ Job=.NET 9.0 Runtime=.NET 9.0
453453
454454
BenchmarkDotNet v0.15.2, Linux Ubuntu 24.04.2 LTS (Noble Numbat)
455455
AMD EPYC 7763, 1 CPU, 4 logical and 2 physical cores
456-
.NET SDK 9.0.301
457-
[Host] : .NET 9.0.6 (9.0.625.26613), X64 RyuJIT AVX2
458-
.NET 9.0 : .NET 9.0.6 (9.0.625.26613), X64 RyuJIT AVX2
456+
.NET SDK 9.0.302
457+
[Host] : .NET 9.0.7 (9.0.725.31616), X64 RyuJIT AVX2
458+
.NET 9.0 : .NET 9.0.7 (9.0.725.31616), X64 RyuJIT AVX2
459459
460460
Job=.NET 9.0 Runtime=.NET 9.0
461461
462462
```
463463
| Method | Mean | Error | StdDev |
464464
|------------- |--------:|---------:|---------:|
465-
| Build_TUnit | 1.969 s | 0.0388 s | 0.0658 s |
466-
| Build_NUnit | 1.495 s | 0.0269 s | 0.0238 s |
467-
| Build_xUnit | 1.493 s | 0.0242 s | 0.0226 s |
468-
| Build_MSTest | 1.513 s | 0.0177 s | 0.0166 s |
465+
| Build_TUnit | 1.906 s | 0.0380 s | 0.0545 s |
466+
| Build_NUnit | 1.456 s | 0.0189 s | 0.0177 s |
467+
| Build_xUnit | 1.451 s | 0.0125 s | 0.0111 s |
468+
| Build_MSTest | 1.469 s | 0.0115 s | 0.0107 s |
469469

470470

471471

@@ -475,19 +475,19 @@ Job=.NET 9.0 Runtime=.NET 9.0
475475
476476
BenchmarkDotNet v0.15.2, Windows 10 (10.0.20348.3807) (Hyper-V)
477477
AMD EPYC 7763 2.44GHz, 1 CPU, 4 logical and 2 physical cores
478-
.NET SDK 9.0.301
479-
[Host] : .NET 9.0.6 (9.0.625.26613), X64 RyuJIT AVX2
480-
.NET 9.0 : .NET 9.0.6 (9.0.625.26613), X64 RyuJIT AVX2
478+
.NET SDK 9.0.302
479+
[Host] : .NET 9.0.7 (9.0.725.31616), X64 RyuJIT AVX2
480+
.NET 9.0 : .NET 9.0.7 (9.0.725.31616), X64 RyuJIT AVX2
481481
482482
Job=.NET 9.0 Runtime=.NET 9.0
483483
484484
```
485485
| Method | Mean | Error | StdDev |
486486
|------------- |--------:|---------:|---------:|
487-
| Build_TUnit | 1.955 s | 0.0386 s | 0.0589 s |
488-
| Build_NUnit | 1.563 s | 0.0294 s | 0.0275 s |
489-
| Build_xUnit | 1.517 s | 0.0287 s | 0.0319 s |
490-
| Build_MSTest | 1.541 s | 0.0248 s | 0.0232 s |
487+
| Build_TUnit | 1.961 s | 0.0386 s | 0.0554 s |
488+
| Build_NUnit | 1.553 s | 0.0271 s | 0.0254 s |
489+
| Build_xUnit | 1.528 s | 0.0164 s | 0.0137 s |
490+
| Build_MSTest | 1.569 s | 0.0165 s | 0.0137 s |
491491

492492

493493
### Scenario: A single test that completes instantly (including spawning a new process and initialising the test framework)
@@ -498,20 +498,20 @@ Job=.NET 9.0 Runtime=.NET 9.0
498498
499499
BenchmarkDotNet v0.15.2, macOS Sonoma 14.7.6 (23H626) [Darwin 23.6.0]
500500
Apple M1 (Virtual), 1 CPU, 3 logical and 3 physical cores
501-
.NET SDK 9.0.301
502-
[Host] : .NET 9.0.6 (9.0.625.26613), Arm64 RyuJIT AdvSIMD
503-
.NET 9.0 : .NET 9.0.6 (9.0.625.26613), Arm64 RyuJIT AdvSIMD
501+
.NET SDK 9.0.302
502+
[Host] : .NET 9.0.7 (9.0.725.31616), Arm64 RyuJIT AdvSIMD
503+
.NET 9.0 : .NET 9.0.7 (9.0.725.31616), Arm64 RyuJIT AdvSIMD
504504
505505
Job=.NET 9.0 Runtime=.NET 9.0
506506
507507
```
508-
| Method | Mean | Error | StdDev |
509-
|---------- |----------:|----------:|----------:|
510-
| TUnit_AOT | 75.22 ms | 0.404 ms | 0.358 ms |
511-
| TUnit | 469.34 ms | 8.643 ms | 9.953 ms |
512-
| NUnit | 706.44 ms | 11.712 ms | 10.383 ms |
513-
| xUnit | 734.35 ms | 14.193 ms | 18.454 ms |
514-
| MSTest | 619.72 ms | 10.187 ms | 9.529 ms |
508+
| Method | Mean | Error | StdDev |
509+
|---------- |----------:|---------:|---------:|
510+
| TUnit_AOT | 70.66 ms | 0.855 ms | 0.800 ms |
511+
| TUnit | 478.89 ms | 4.367 ms | 3.647 ms |
512+
| NUnit | 706.56 ms | 9.634 ms | 8.045 ms |
513+
| xUnit | 720.40 ms | 9.537 ms | 8.921 ms |
514+
| MSTest | 626.29 ms | 9.566 ms | 8.480 ms |
515515

516516

517517

@@ -521,20 +521,20 @@ Job=.NET 9.0 Runtime=.NET 9.0
521521
522522
BenchmarkDotNet v0.15.2, Linux Ubuntu 24.04.2 LTS (Noble Numbat)
523523
AMD EPYC 7763, 1 CPU, 4 logical and 2 physical cores
524-
.NET SDK 9.0.301
525-
[Host] : .NET 9.0.6 (9.0.625.26613), X64 RyuJIT AVX2
526-
.NET 9.0 : .NET 9.0.6 (9.0.625.26613), X64 RyuJIT AVX2
524+
.NET SDK 9.0.302
525+
[Host] : .NET 9.0.7 (9.0.725.31616), X64 RyuJIT AVX2
526+
.NET 9.0 : .NET 9.0.7 (9.0.725.31616), X64 RyuJIT AVX2
527527
528528
Job=.NET 9.0 Runtime=.NET 9.0
529529
530530
```
531531
| Method | Mean | Error | StdDev |
532532
|---------- |------------:|----------:|----------:|
533-
| TUnit_AOT | 27.28 ms | 0.739 ms | 2.178 ms |
534-
| TUnit | 826.51 ms | 16.185 ms | 16.621 ms |
535-
| NUnit | 1,288.58 ms | 16.823 ms | 15.736 ms |
536-
| xUnit | 1,335.37 ms | 9.836 ms | 9.201 ms |
537-
| MSTest | 1,130.57 ms | 14.008 ms | 13.104 ms |
533+
| TUnit_AOT | 26.93 ms | 0.623 ms | 1.807 ms |
534+
| TUnit | 809.76 ms | 15.895 ms | 15.611 ms |
535+
| NUnit | 1,279.12 ms | 8.892 ms | 8.317 ms |
536+
| xUnit | 1,331.71 ms | 11.166 ms | 10.445 ms |
537+
| MSTest | 1,124.72 ms | 9.222 ms | 7.701 ms |
538538

539539

540540

@@ -544,20 +544,20 @@ Job=.NET 9.0 Runtime=.NET 9.0
544544
545545
BenchmarkDotNet v0.15.2, Windows 10 (10.0.20348.3807) (Hyper-V)
546546
AMD EPYC 7763 2.44GHz, 1 CPU, 4 logical and 2 physical cores
547-
.NET SDK 9.0.301
548-
[Host] : .NET 9.0.6 (9.0.625.26613), X64 RyuJIT AVX2
549-
.NET 9.0 : .NET 9.0.6 (9.0.625.26613), X64 RyuJIT AVX2
547+
.NET SDK 9.0.302
548+
[Host] : .NET 9.0.7 (9.0.725.31616), X64 RyuJIT AVX2
549+
.NET 9.0 : .NET 9.0.7 (9.0.725.31616), X64 RyuJIT AVX2
550550
551551
Job=.NET 9.0 Runtime=.NET 9.0
552552
553553
```
554554
| Method | Mean | Error | StdDev |
555555
|---------- |------------:|----------:|----------:|
556-
| TUnit_AOT | 58.46 ms | 1.789 ms | 5.219 ms |
557-
| TUnit | 895.06 ms | 17.378 ms | 26.539 ms |
558-
| NUnit | 1,359.08 ms | 26.245 ms | 25.777 ms |
559-
| xUnit | 1,382.43 ms | 13.794 ms | 12.903 ms |
560-
| MSTest | 1,176.92 ms | 11.738 ms | 9.802 ms |
556+
| TUnit_AOT | 54.78 ms | 1.365 ms | 3.916 ms |
557+
| TUnit | 882.17 ms | 17.505 ms | 26.201 ms |
558+
| NUnit | 1,358.25 ms | 22.774 ms | 28.802 ms |
559+
| xUnit | 1,401.37 ms | 27.676 ms | 25.888 ms |
560+
| MSTest | 1,183.35 ms | 16.903 ms | 15.811 ms |
561561

562562

563563
### Scenario: A test that takes 50ms to execute, repeated 100 times (including spawning a new process and initialising the test framework)
@@ -568,23 +568,20 @@ Job=.NET 9.0 Runtime=.NET 9.0
568568
569569
BenchmarkDotNet v0.15.2, macOS Sonoma 14.7.6 (23H626) [Darwin 23.6.0]
570570
Apple M1 (Virtual), 1 CPU, 3 logical and 3 physical cores
571-
.NET SDK 9.0.301
572-
[Host] : .NET 9.0.6 (9.0.625.26613), Arm64 RyuJIT AdvSIMD
573-
.NET 9.0 : .NET 9.0.6 (9.0.625.26613), Arm64 RyuJIT AdvSIMD
571+
.NET SDK 9.0.302
572+
[Host] : .NET 9.0.7 (9.0.725.31616), Arm64 RyuJIT AdvSIMD
573+
.NET 9.0 : .NET 9.0.7 (9.0.725.31616), Arm64 RyuJIT AdvSIMD
574574
575575
Job=.NET 9.0 Runtime=.NET 9.0
576576
577577
```
578578
| Method | Mean | Error | StdDev |
579579
|---------- |------------:|----------:|----------:|
580-
| TUnit_AOT | 243.3 ms | 11.49 ms | 33.70 ms |
581-
| TUnit | 634.9 ms | 20.22 ms | 59.60 ms |
582-
| NUnit | 13,981.2 ms | 279.41 ms | 583.22 ms |
583-
| xUnit | NA | NA | NA |
584-
| MSTest | 14,256.3 ms | 282.60 ms | 643.62 ms |
585-
586-
Benchmarks with issues:
587-
RuntimeBenchmarks.xUnit: .NET 9.0(Runtime=.NET 9.0)
580+
| TUnit_AOT | 246.8 ms | 10.98 ms | 32.37 ms |
581+
| TUnit | 655.8 ms | 22.11 ms | 65.20 ms |
582+
| NUnit | 14,193.3 ms | 266.33 ms | 531.88 ms |
583+
| xUnit | 14,360.3 ms | 285.99 ms | 537.16 ms |
584+
| MSTest | 14,287.4 ms | 284.77 ms | 575.26 ms |
588585

589586

590587

@@ -594,20 +591,20 @@ Benchmarks with issues:
594591
595592
BenchmarkDotNet v0.15.2, Linux Ubuntu 24.04.2 LTS (Noble Numbat)
596593
AMD EPYC 7763, 1 CPU, 4 logical and 2 physical cores
597-
.NET SDK 9.0.301
598-
[Host] : .NET 9.0.6 (9.0.625.26613), X64 RyuJIT AVX2
599-
.NET 9.0 : .NET 9.0.6 (9.0.625.26613), X64 RyuJIT AVX2
594+
.NET SDK 9.0.302
595+
[Host] : .NET 9.0.7 (9.0.725.31616), X64 RyuJIT AVX2
596+
.NET 9.0 : .NET 9.0.7 (9.0.725.31616), X64 RyuJIT AVX2
600597
601598
Job=.NET 9.0 Runtime=.NET 9.0
602599
603600
```
604601
| Method | Mean | Error | StdDev |
605602
|---------- |------------:|----------:|----------:|
606-
| TUnit_AOT | 74.51 ms | 0.570 ms | 0.476 ms |
607-
| TUnit | 907.93 ms | 17.675 ms | 25.349 ms |
608-
| NUnit | 6,290.38 ms | 15.471 ms | 14.471 ms |
609-
| xUnit | 6,429.98 ms | 20.035 ms | 18.740 ms |
610-
| MSTest | 6,246.55 ms | 17.250 ms | 16.136 ms |
603+
| TUnit_AOT | 73.74 ms | 0.434 ms | 0.362 ms |
604+
| TUnit | 891.83 ms | 17.107 ms | 16.802 ms |
605+
| NUnit | 6,269.02 ms | 8.872 ms | 8.299 ms |
606+
| xUnit | 6,409.66 ms | 10.871 ms | 10.169 ms |
607+
| MSTest | 6,242.22 ms | 11.491 ms | 10.748 ms |
611608

612609

613610

@@ -617,20 +614,20 @@ Job=.NET 9.0 Runtime=.NET 9.0
617614
618615
BenchmarkDotNet v0.15.2, Windows 10 (10.0.20348.3807) (Hyper-V)
619616
AMD EPYC 7763 2.44GHz, 1 CPU, 4 logical and 2 physical cores
620-
.NET SDK 9.0.301
621-
[Host] : .NET 9.0.6 (9.0.625.26613), X64 RyuJIT AVX2
622-
.NET 9.0 : .NET 9.0.6 (9.0.625.26613), X64 RyuJIT AVX2
617+
.NET SDK 9.0.302
618+
[Host] : .NET 9.0.7 (9.0.725.31616), X64 RyuJIT AVX2
619+
.NET 9.0 : .NET 9.0.7 (9.0.725.31616), X64 RyuJIT AVX2
623620
624621
Job=.NET 9.0 Runtime=.NET 9.0
625622
626623
```
627624
| Method | Mean | Error | StdDev |
628625
|---------- |-----------:|---------:|---------:|
629-
| TUnit_AOT | 110.5 ms | 1.84 ms | 1.97 ms |
630-
| TUnit | 985.9 ms | 19.53 ms | 26.73 ms |
631-
| NUnit | 7,531.4 ms | 41.54 ms | 34.69 ms |
632-
| xUnit | 7,585.3 ms | 18.56 ms | 17.36 ms |
633-
| MSTest | 7,467.0 ms | 22.65 ms | 21.19 ms |
626+
| TUnit_AOT | 112.4 ms | 2.24 ms | 3.42 ms |
627+
| TUnit | 956.8 ms | 18.62 ms | 26.70 ms |
628+
| NUnit | 7,552.8 ms | 49.11 ms | 45.94 ms |
629+
| xUnit | 7,683.9 ms | 61.00 ms | 57.06 ms |
630+
| MSTest | 7,577.2 ms | 43.62 ms | 40.80 ms |
634631

635632

636633

0 commit comments

Comments
 (0)