Skip to content

[Benchmarks] Add Micro-benchmarks #6802

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 19 commits into from
Apr 14, 2025
Merged

[Benchmarks] Add Micro-benchmarks #6802

merged 19 commits into from
Apr 14, 2025

Conversation

faydef
Copy link
Contributor

@faydef faydef commented Mar 28, 2025 •
edited
Loading

Summary of changes

This PR introduces running benchmarks through the Benchmarking-platform all while keeping the current benchmarks running.
The tests are setup to run:

  • On commit to the master branch
  • On schedule if BENCHMARK_RUN is set to "true" for the scheduled pipeline.
  • On manual trigger (keep in mind that it requires build artifacts from the Azure Pipeline)

Reason for change

Related to efforts to use Gitlab as our main CI Provider.

Implementation details

Test coverage

Other details

Results from the benchmarks report do differ as we do not run on master currently so the comparaison is done against an old seeded run

@datadog-datadog-prod-us1
Copy link

datadog-datadog-prod-us1 bot commented Mar 28, 2025 •
edited
Loading

Datadog Report

Branch report: fayssal/test-micro-delivery
Commit report: 6051808
Test service: dd-trace-dotnet

❌ 1 Failed (0 Known Flaky), 249714 Passed, 2371 Skipped, 20h 57m 30.54s Total Time

❌ Failed Tests (1)

  • Profiler_allocation_cpu_walltime - scenarios - Details

    Expand for error 
     
 ___Iteration 47___ions of GenericsAllocation on 1 thread(s)cation 500 times with 1 thread(s).n\monitoring-home\win-x86\Datadog.Profiler.Native.dlltation 5=compute fibonacci 6=n sleeping threads 7=async calls 8=iterator calls 9=allocate array of Generic<int>> 10=threads competing for a lock 11=lunix signal handler 12=trigger garbage collections 13=memory leak 14=short lived threads] [--param <any number to pass to the scenario - used for contention duration for example>] [--timeout <duration in seconds> | --run-infinitely]uteWithThreadLocal(System.Threading.Tasks.Task ByRef, System.Threading.Thread)
    at System.Threading.Tasks.Task.ExecuteEntryUnsafe(System.Threading.Thread)
    at System.Threading.Tasks.ThreadPoolTaskScheduler+<>c.<.cctor>b__10_0(System.Object)
    at System.Threading.Thread.StartCallback()

@andrewlock
Copy link
Member

andrewlock commented Mar 28, 2025 •
edited
Loading

Benchmarks Report for tracer 🐌

Benchmarks for #6802 compared to master:

  • 1 benchmarks are faster, with geometric mean 1.225
  • 4 benchmarks are slower, with geometric mean 1.171
  • 1 benchmarks have fewer allocations
  • 2 benchmarks have more allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed ✔️ More allocations ⚠️

More allocations ⚠️ in #6802

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.ActivityBenchmark.StartStopWithChild‑net472 6.2 KB 6.23 KB 35 B 0.56%

Fewer allocations 🎉 in #6802

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.ActivityBenchmark.StartStopWithChild‑netcoreapp3.1 5.8 KB 5.75 KB -46 B -0.79%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartStopWithChild net6.0 7.73μs 44.5ns 361ns 0.0269 0.0115 0 5.61 KB
master StartStopWithChild netcoreapp3.1 10.3μs 56.9ns 336ns 0.0257 0.0103 0 5.8 KB
master StartStopWithChild net472 16.1μs 61.2ns 237ns 1.04 0.316 0.111 6.2 KB
#6802 StartStopWithChild net6.0 7.85μs 136ns 1.36μs 0 0 0 5.59 KB
#6802 StartStopWithChild netcoreapp3.1 10.1μs 97.2ns 967ns 0 0 0 5.75 KB
#6802 StartStopWithChild net472 15.5μs 86.9ns 563ns 1.06 0.302 0.0754 6.23 KB
Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ More allocations ⚠️

More allocations ⚠️ in #6802

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.AgentWriterBenchmark.WriteAndFlushEnrichedTraces‑net472 3.3 KB 3.35 KB 47 B 1.43%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 473μs 564ns 2.18μs 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 657μs 444ns 1.72μs 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces net472 849μs 777ns 3.01μs 0.425 0 0 3.3 KB
#6802 WriteAndFlushEnrichedTraces net6.0 497μs 390ns 1.41μs 0 0 0 2.7 KB
#6802 WriteAndFlushEnrichedTraces netcoreapp3.1 646μs 657ns 2.28μs 0 0 0 2.7 KB
#6802 WriteAndFlushEnrichedTraces net472 850μs 840ns 3.14μs 0 0 0 3.35 KB
Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net6.0 129μs 317ns 1.23μs 0.196 0 0 14.47 KB
master SendRequest netcoreapp3.1 145μs 293ns 1.1μs 0.218 0 0 17.27 KB
master SendRequest net472 3.06E‑05ns 2.48E‑05ns 9.62E‑05ns 0 0 0 0 b
#6802 SendRequest net6.0 128μs 733ns 5.24μs 0 0 0 14.47 KB
#6802 SendRequest netcoreapp3.1 145μs 696ns 2.69μs 0 0 0 17.27 KB
#6802 SendRequest net472 0.00213ns 0.000875ns 0.00316ns 0 0 0 0 b
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 607μs 3.53μs 32.8μs 0.316 0 0 41.65 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 724μs 4.13μs 29.2μs 0.357 0 0 41.74 KB
master WriteAndFlushEnrichedTraces net472 871μs 4.16μs 16.6μs 8.56 2.57 0.428 53.35 KB
#6802 WriteAndFlushEnrichedTraces net6.0 592μs 4.43μs 42.8μs 0 0 0 41.77 KB
#6802 WriteAndFlushEnrichedTraces netcoreapp3.1 696μs 4.41μs 43.4μs 0 0 0 41.72 KB
#6802 WriteAndFlushEnrichedTraces net472 868μs 5μs 44.1μs 7.81 3.91 0 53.47 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net6.0 1.4μs 1.79ns 6.92ns 0.0146 0 0 1.02 KB
master ExecuteNonQuery netcoreapp3.1 1.75μs 0.755ns 2.82ns 0.014 0 0 1.02 KB
master ExecuteNonQuery net472 2.14μs 3.89ns 15.1ns 0.157 0.00107 0 987 B
#6802 ExecuteNonQuery net6.0 1.35μs 5.71ns 31.3ns 0.0132 0 0 1.02 KB
#6802 ExecuteNonQuery netcoreapp3.1 1.9μs 9.62ns 46.1ns 0.00928 0 0 1.02 KB
#6802 ExecuteNonQuery net472 2.08μs 4.58ns 17.1ns 0.155 0.0103 0 987 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net6.0 1.2μs 1.1ns 4.26ns 0.0133 0 0 976 B
master CallElasticsearch netcoreapp3.1 1.61μs 0.981ns 3.54ns 0.0128 0 0 976 B
master CallElasticsearch net472 2.67μs 1.16ns 4.33ns 0.157 0 0 995 B
master CallElasticsearchAsync net6.0 1.15μs 0.566ns 2.12ns 0.0133 0 0 952 B
master CallElasticsearchAsync netcoreapp3.1 1.64μs 0.825ns 3.09ns 0.0134 0 0 1.02 KB
master CallElasticsearchAsync net472 2.6μs 0.83ns 3.11ns 0.166 0 0 1.05 KB
#6802 CallElasticsearch net6.0 1.23μs 1.19ns 4.46ns 0.0124 0 0 976 B
#6802 CallElasticsearch netcoreapp3.1 1.5μs 1.38ns 4.99ns 0.00749 0 0 976 B
#6802 CallElasticsearch net472 2.58μs 3.1ns 11.2ns 0.155 0 0 995 B
#6802 CallElasticsearchAsync net6.0 1.27μs 0.845ns 3.27ns 0.0126 0 0 952 B
#6802 CallElasticsearchAsync netcoreapp3.1 1.69μs 0.925ns 3.46ns 0.00846 0 0 1.02 KB
#6802 CallElasticsearchAsync net472 2.66μs 1.71ns 6.18ns 0.159 0 0 1.05 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net6.0 1.37μs 1.79ns 6.69ns 0.0133 0 0 952 B
master ExecuteAsync netcoreapp3.1 1.54μs 0.637ns 2.38ns 0.0124 0 0 952 B
master ExecuteAsync net472 1.83μs 0.443ns 1.72ns 0.145 0 0 915 B
#6802 ExecuteAsync net6.0 1.29μs 0.973ns 3.77ns 0.013 0 0 952 B
#6802 ExecuteAsync netcoreapp3.1 1.62μs 0.726ns 2.52ns 0.00805 0 0 952 B
#6802 ExecuteAsync net472 1.82μs 1.32ns 4.94ns 0.145 0 0 915 B
Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net6.0 4.3μs 1.39ns 5.18ns 0.0323 0 0 2.31 KB
master SendAsync netcoreapp3.1 5.39μs 3.62ns 14ns 0.0377 0 0 2.85 KB
master SendAsync net472 7.44μs 2.12ns 7.63ns 0.496 0 0 3.12 KB
#6802 SendAsync net6.0 4.24μs 2.49ns 9.65ns 0.0212 0 0 2.31 KB
#6802 SendAsync netcoreapp3.1 5.22μs 4.11ns 14.8ns 0.026 0 0 2.85 KB
#6802 SendAsync net472 7.47μs 3.65ns 13.7ns 0.485 0 0 3.12 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 1.48μs 0.655ns 2.36ns 0.0239 0 0 1.69 KB
master EnrichedLog netcoreapp3.1 2.32μs 2.84ns 10.6ns 0.0232 0 0 1.69 KB
master EnrichedLog net472 2.65μs 1.65ns 6.41ns 0.258 0 0 1.62 KB
#6802 EnrichedLog net6.0 1.61μs 1.76ns 6.84ns 0.024 0 0 1.69 KB
#6802 EnrichedLog netcoreapp3.1 2.2μs 4.49ns 16.8ns 0.022 0 0 1.69 KB
#6802 EnrichedLog net472 2.6μs 2.28ns 8.83ns 0.247 0 0 1.62 KB
Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 112μs 172ns 664ns 0 0 0 4.3 KB
master EnrichedLog netcoreapp3.1 117μs 184ns 713ns 0.0585 0 0 4.3 KB
master EnrichedLog net472 151μs 140ns 542ns 0.684 0.228 0 4.49 KB
#6802 EnrichedLog net6.0 111μs 180ns 696ns 0 0 0 4.31 KB
#6802 EnrichedLog netcoreapp3.1 116μs 244ns 947ns 0 0 0 4.3 KB
#6802 EnrichedLog net472 149μs 144ns 560ns 0 0 0 4.49 KB
Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 2.94μs 2.14ns 8.3ns 0.0308 0 0 2.25 KB
master EnrichedLog netcoreapp3.1 4.22μs 0.687ns 2.57ns 0.0294 0 0 2.25 KB
master EnrichedLog net472 4.98μs 1.51ns 5.84ns 0.327 0 0 2.07 KB
#6802 EnrichedLog net6.0 3.13μs 2.28ns 8.82ns 0.0312 0 0 2.25 KB
#6802 EnrichedLog netcoreapp3.1 4.24μs 2.18ns 7.56ns 0.0211 0 0 2.25 KB
#6802 EnrichedLog net472 4.82μs 3.35ns 13ns 0.315 0 0 2.07 KB
Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net6.0 1.45μs 0.604ns 2.34ns 0.0159 0 0 1.14 KB
master SendReceive netcoreapp3.1 1.77μs 0.948ns 3.67ns 0.015 0 0 1.14 KB
master SendReceive net472 2.17μs 1ns 3.75ns 0.183 0 0 1.16 KB
#6802 SendReceive net6.0 1.32μs 0.87ns 3.37ns 0.0133 0 0 1.14 KB
#6802 SendReceive netcoreapp3.1 1.71μs 1.59ns 6.15ns 0.00859 0 0 1.14 KB
#6802 SendReceive net472 2.1μs 1.43ns 5.16ns 0.178 0 0 1.16 KB
Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 2.84μs 1.65ns 6.18ns 0.0228 0 0 1.62 KB
master EnrichedLog netcoreapp3.1 3.92μs 1.68ns 6.28ns 0.0214 0 0 1.67 KB
master EnrichedLog net472 4.58μs 2.16ns 8.35ns 0.326 0 0 2.07 KB
#6802 EnrichedLog net6.0 2.71μs 2.48ns 9.61ns 0.0136 0 0 1.62 KB
#6802 EnrichedLog netcoreapp3.1 3.87μs 2.35ns 8.49ns 0.0194 0 0 1.67 KB
#6802 EnrichedLog net472 4.39μs 6.99ns 25.2ns 0.307 0 0 2.07 KB
Benchmarks.Trace.SpanBenchmark - Slower ⚠️ Same allocations ✔️

Slower ⚠️ in #6802

Benchmark diff/base Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.SpanBenchmark.StartFinishScope‑net472 1.180 790.42 932.86
Benchmarks.Trace.SpanBenchmark.StartFinishScope‑net6.0 1.139 485.72 553.30
Benchmarks.Trace.SpanBenchmark.StartFinishSpan‑netcoreapp3.1 1.130 572.07 646.51

Faster 🎉 in #6802

Benchmark base/diff Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.SpanBenchmark.StartFinishSpan‑net6.0 1.225 475.88 388.60

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net6.0 476ns 0.21ns 0.758ns 0.00805 0 0 576 B
master StartFinishSpan netcoreapp3.1 572ns 0.187ns 0.699ns 0.00783 0 0 576 B
master StartFinishSpan net472 621ns 1.04ns 3.89ns 0.0916 0 0 578 B
master StartFinishScope net6.0 486ns 0.227ns 0.85ns 0.00969 0 0 696 B
master StartFinishScope netcoreapp3.1 704ns 0.264ns 1.02ns 0.00954 0 0 696 B
master StartFinishScope net472 790ns 0.225ns 0.843ns 0.104 0 0 658 B
#6802 StartFinishSpan net6.0 389ns 0.738ns 2.86ns 0.00778 0 0 576 B
#6802 StartFinishSpan netcoreapp3.1 646ns 0.807ns 3.02ns 0.00628 0 0 576 B
#6802 StartFinishSpan net472 594ns 0.298ns 1.07ns 0.0916 0 0 578 B
#6802 StartFinishScope net6.0 553ns 0.406ns 1.52ns 0.00823 0 0 696 B
#6802 StartFinishScope netcoreapp3.1 727ns 0.676ns 2.44ns 0.00722 0 0 696 B
#6802 StartFinishScope net472 933ns 0.388ns 1.5ns 0.103 0 0 658 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Slower ⚠️ Same allocations ✔️

Slower ⚠️ in #6802

Benchmark diff/base Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.TraceAnnotationsBenchmark.RunOnMethodBegin‑net6.0 1.239 595.41 737.76

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net6.0 595ns 0.22ns 0.851ns 0.00974 0 0 696 B
master RunOnMethodBegin netcoreapp3.1 900ns 1.32ns 5.13ns 0.00955 0 0 696 B
master RunOnMethodBegin net472 1.21μs 0.833ns 3.23ns 0.104 0 0 658 B
#6802 RunOnMethodBegin net6.0 738ns 0.727ns 2.72ns 0.00737 0 0 696 B
#6802 RunOnMethodBegin netcoreapp3.1 968ns 1.19ns 4.46ns 0.00478 0 0 696 B
#6802 RunOnMethodBegin net472 1.14μs 0.846ns 3.28ns 0.102 0 0 658 B

@andrewlock
Copy link
Member

andrewlock commented Mar 28, 2025 •
edited
Loading

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

  • Welch test with statistical test for significance of 5%
  • Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6802) - mean (70ms)  : 66, 75
     .   : milestone, 70,
    master - mean (70ms)  : 66, 74
     .   : milestone, 70,

    section CallTarget+Inlining+NGEN
    This PR (6802) - mean (1,002ms)  : 980, 1023
     .   : milestone, 1002,
    master - mean (1,001ms)  : 980, 1022
     .   : milestone, 1001,
Loading 
gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6802) - mean (103ms)  : 101, 105
     .   : milestone, 103,
    master - mean (103ms)  : 102, 105
     .   : milestone, 103,

    section CallTarget+Inlining+NGEN
    This PR (6802) - mean (692ms)  : 673, 711
     .   : milestone, 692,
    master - mean (689ms)  : 671, 708
     .   : milestone, 689,
Loading 
gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6802) - mean (90ms)  : 87, 93
     .   : milestone, 90,
    master - mean (90ms)  : 88, 92
     .   : milestone, 90,

    section CallTarget+Inlining+NGEN
    This PR (6802) - mean (641ms)  : 625, 657
     .   : milestone, 641,
    master - mean (649ms)  : 627, 671
     .   : milestone, 649,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6802) - mean (190ms)  : 181, 199
     .   : milestone, 190,
    master - mean (189ms)  : 185, 193
     .   : milestone, 189,

    section CallTarget+Inlining+NGEN
    This PR (6802) - mean (1,140ms)  : 1017, 1263
     .   : milestone, 1140,
    master - mean (1,097ms)  : 1072, 1123
     .   : milestone, 1097,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6802) - mean (269ms)  : 263, 274
     .   : milestone, 269,
    master - mean (269ms)  : 265, 273
     .   : milestone, 269,

    section CallTarget+Inlining+NGEN
    This PR (6802) - mean (877ms)  : 839, 914
     .   : milestone, 877,
    master - mean (875ms)  : 847, 903
     .   : milestone, 875,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6802) - mean (260ms)  : 257, 263
     .   : milestone, 260,
    master - mean (261ms)  : 256, 265
     .   : milestone, 261,

    section CallTarget+Inlining+NGEN
    This PR (6802) - mean (848ms)  : 815, 881
     .   : milestone, 848,
    master - mean (858ms)  : 828, 888
     .   : milestone, 858,
Loading

@faydef faydef changed the title Fayssal/test micro delivery [Benchmarks] Add Micro-benchmarks Mar 28, 2025
@DataDog DataDog deleted a comment from pr-commenter bot Mar 31, 2025
@faydef faydef force-pushed the fayssal/test-micro-delivery branch from 3e51348 to ddbe3d2 Compare March 31, 2025 14:18
@DataDog DataDog deleted a comment from pr-commenter bot Mar 31, 2025
@pr-commenter
Copy link

pr-commenter bot commented Mar 31, 2025 •
edited
Loading

Benchmarks

Benchmarks Report for tracer 🐌

Benchmarks for #6802 compared to master:

  • 8 benchmarks are faster, with geometric mean 1.543
  • 1 benchmarks are slower, with geometric mean 1.135
  • 5 benchmarks have fewer allocations
  • 12 benchmarks have more allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Slower ⚠️ More allocations ⚠️

Slower ⚠️ in #6802

Benchmark diff/base Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.ActivityBenchmark.StartStopWithChild‑net472 1.135 19,666.26 22,312.24

More allocations ⚠️ in #6802

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.ActivityBenchmark.StartStopWithChild‑net6.0 5.47 KB 5.62 KB 155 B 2.84%
Benchmarks.Trace.ActivityBenchmark.StartStopWithChild‑netcoreapp3.1 5.66 KB 5.81 KB 153 B 2.71%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartStopWithChild net6.0 13.3μs 369ns 3.69μs 0 0 0 5.47 KB
master StartStopWithChild netcoreapp3.1 16μs 339ns 3.37μs 0 0 0 5.66 KB
master StartStopWithChild net472 19.7μs 130ns 1.23μs 0.845 0 0 6.18 KB
#6802 StartStopWithChild net6.0 10.5μs 57.2ns 323ns 0 0 0 5.62 KB
#6802 StartStopWithChild netcoreapp3.1 13.5μs 70.6ns 339ns 0 0 0 5.81 KB
#6802 StartStopWithChild net472 22.3μs 125ns 811ns 1.03 0.345 0.115 6.21 KB
Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Fewer allocations 🎉

Fewer allocations 🎉 in #6802

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.AgentWriterBenchmark.WriteAndFlushEnrichedTraces‑net6.0 2.77 KB 2.7 KB -69 B -2.49%
Benchmarks.Trace.AgentWriterBenchmark.WriteAndFlushEnrichedTraces‑netcoreapp3.1 2.91 KB 2.7 KB -214 B -7.34%
Benchmarks.Trace.AgentWriterBenchmark.WriteAndFlushEnrichedTraces‑net472 4.1 KB 3.31 KB -791 B -19.31%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 938μs 3.21μs 21μs 0 0 0 2.77 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 1.03ms 1.73μs 11μs 0 0 0 2.91 KB
master WriteAndFlushEnrichedTraces net472 1.21ms 145ns 525ns 0 0 0 4.1 KB
#6802 WriteAndFlushEnrichedTraces net6.0 940μs 44.7ns 173ns 0 0 0 2.7 KB
#6802 WriteAndFlushEnrichedTraces netcoreapp3.1 1ms 318ns 1.15μs 0 0 0 2.7 KB
#6802 WriteAndFlushEnrichedTraces net472 1.21ms 215ns 832ns 0 0 0 3.31 KB
Benchmarks.Trace.AspNetCoreBenchmark - Faster 🎉 Fewer allocations 🎉

Faster 🎉 in #6802

Benchmark base/diff Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.AspNetCoreBenchmark.SendRequest‑net6.0 2.021 122,440.44 60,598.92
Benchmarks.Trace.AspNetCoreBenchmark.SendRequest‑netcoreapp3.1 2.008 143,830.15 71,638.43

Fewer allocations 🎉 in #6802

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.AspNetCoreBenchmark.SendRequest‑netcoreapp3.1 18.72 KB 17.42 KB -1.31 KB -6.97%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net6.0 123μs 467ns 2.64μs 0 0 0 14.53 KB
master SendRequest netcoreapp3.1 143μs 637ns 2.3μs 0 0 0 18.72 KB
master SendRequest net472 0.00926ns 0.00356ns 0.0138ns 0 0 0 0 b
#6802 SendRequest net6.0 60.6μs 39.1ns 141ns 0 0 0 14.52 KB
#6802 SendRequest netcoreapp3.1 71.7μs 61.6ns 239ns 0 0 0 17.42 KB
#6802 SendRequest net472 0.00382ns 0.00205ns 0.00792ns 0 0 0 0 b
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed ✔️ Fewer allocations 🎉

Fewer allocations 🎉 in #6802

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑net472 58.37 KB 56.36 KB -2.01 KB -3.45%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 729μs 4.12μs 29.4μs 0 0 0 42.09 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 704μs 4.93μs 45.8μs 0 0 0 42.2 KB
master WriteAndFlushEnrichedTraces net472 1.01ms 6.73μs 66.7μs 0 0 0 58.37 KB
#6802 WriteAndFlushEnrichedTraces net6.0 665μs 524ns 2.03μs 0 0 0 42.17 KB
#6802 WriteAndFlushEnrichedTraces netcoreapp3.1 708μs 1.92μs 6.94μs 0 0 0 42.14 KB
#6802 WriteAndFlushEnrichedTraces net472 885μs 3.91μs 15.2μs 8.33 0 0 56.36 KB
Benchmarks.Trace.DbCommandBenchmark - Faster 🎉 Same allocations ✔️

Faster 🎉 in #6802

Benchmark base/diff Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.DbCommandBenchmark.ExecuteNonQuery‑net472 1.125 2,971.85 2,642.37

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net6.0 1.77μs 5.02ns 33.7ns 0 0 0 1.02 KB
master ExecuteNonQuery netcoreapp3.1 2.39μs 5.36ns 20.1ns 0 0 0 1.02 KB
master ExecuteNonQuery net472 2.96μs 7.54ns 29.2ns 0.144 0 0 987 B
#6802 ExecuteNonQuery net6.0 1.86μs 9.63ns 48.1ns 0 0 0 1.02 KB
#6802 ExecuteNonQuery netcoreapp3.1 2.57μs 5.79ns 22.4ns 0 0 0 1.02 KB
#6802 ExecuteNonQuery net472 2.64μs 2.48ns 9.61ns 0.146 0.0133 0 987 B
Benchmarks.Trace.ElasticsearchBenchmark - Faster 🎉 Same allocations ✔️

Faster 🎉 in #6802

Benchmark base/diff Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.ElasticsearchBenchmark.CallElasticsearch‑netcoreapp3.1 1.618 3,686.57 2,278.69

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net6.0 1.69μs 7.62ns 49.4ns 0 0 0 1.03 KB
master CallElasticsearch netcoreapp3.1 3.68μs 3.08ns 11.1ns 0 0 0 1.03 KB
master CallElasticsearch net472 3.51μs 12.3ns 47.8ns 0 0 0 1.04 KB
master CallElasticsearchAsync net6.0 1.77μs 11.4ns 106ns 0 0 0 1.01 KB
master CallElasticsearchAsync netcoreapp3.1 2.35μs 3.64ns 27ns 0 0 0 1.08 KB
master CallElasticsearchAsync net472 3.56μs 5.26ns 20.4ns 0 0 0 1.1 KB
#6802 CallElasticsearch net6.0 1.67μs 8.66ns 40.6ns 0 0 0 1.03 KB
#6802 CallElasticsearch netcoreapp3.1 2.28μs 3.51ns 13.6ns 0 0 0 1.03 KB
#6802 CallElasticsearch net472 3.43μs 2.11ns 7.9ns 0.155 0 0 1.04 KB
#6802 CallElasticsearchAsync net6.0 1.79μs 7.71ns 29.9ns 0 0 0 1.01 KB
#6802 CallElasticsearchAsync netcoreapp3.1 2.42μs 6.82ns 26.4ns 0 0 0 1.08 KB
#6802 CallElasticsearchAsync net472 3.61μs 3.86ns 15ns 0.161 0 0 1.1 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net6.0 1.79μs 2.52ns 18.2ns 0 0 0 952 B
master ExecuteAsync netcoreapp3.1 2.25μs 2.6ns 18.2ns 0 0 0 952 B
master ExecuteAsync net472 2.5μs 6.3ns 24.4ns 0.127 0 0 915 B
#6802 ExecuteAsync net6.0 1.77μs 9.24ns 41.3ns 0 0 0 952 B
#6802 ExecuteAsync netcoreapp3.1 2.34μs 10.7ns 41.5ns 0 0 0 952 B
#6802 ExecuteAsync net472 2.62μs 1.44ns 8.76ns 0.144 0 0 915 B
Benchmarks.Trace.HttpClientBenchmark - Faster 🎉 Same allocations ✔️

Faster 🎉 in #6802

Benchmark base/diff Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.HttpClientBenchmark.SendAsync‑netcoreapp3.1 1.782 14,765.54 8,286.63

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net6.0 8.73μs 256ns 2.56μs 0 0 0 2.36 KB
master SendAsync netcoreapp3.1 14.7μs 43ns 166ns 0 0 0 2.9 KB
master SendAsync net472 12μs 12.3ns 47.6ns 0 0 0 3.18 KB
#6802 SendAsync net6.0 6.83μs 6.17ns 22.3ns 0 0 0 2.36 KB
#6802 SendAsync netcoreapp3.1 8.25μs 24.8ns 95.9ns 0 0 0 2.9 KB
#6802 SendAsync net472 12.2μs 13.3ns 51.5ns 0.489 0 0 3.18 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ More allocations ⚠️

More allocations ⚠️ in #6802

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.ILoggerBenchmark.EnrichedLog‑net472 1.62 KB 1.68 KB 53 B 3.27%
Benchmarks.Trace.ILoggerBenchmark.EnrichedLog‑netcoreapp3.1 1.7 KB 1.74 KB 49 B 2.89%
Benchmarks.Trace.ILoggerBenchmark.EnrichedLog‑net6.0 1.7 KB 1.74 KB 48 B 2.83%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 2.52μs 8.32ns 59.4ns 0 0 0 1.7 KB
master EnrichedLog netcoreapp3.1 4.06μs 99.6ns 996ns 0 0 0 1.7 KB
master EnrichedLog net472 3.8μs 4.8ns 17.9ns 0.189 0 0 1.62 KB
#6802 EnrichedLog net6.0 2.61μs 14.3ns 81ns 0 0 0 1.74 KB
#6802 EnrichedLog netcoreapp3.1 3.47μs 10.6ns 41.1ns 0 0 0 1.74 KB
#6802 EnrichedLog net472 3.96μs 5.87ns 22.7ns 0.258 0 0 1.68 KB
Benchmarks.Trace.Log4netBenchmark - Faster 🎉 More allocations ⚠️

Faster 🎉 in #6802

Benchmark base/diff Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.Log4netBenchmark.EnrichedLog‑net6.0 1.203 149,158.59 123,969.10
Benchmarks.Trace.Log4netBenchmark.EnrichedLog‑netcoreapp3.1 1.134 147,888.19 130,423.23

More allocations ⚠️ in #6802

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.Log4netBenchmark.EnrichedLog‑netcoreapp3.1 4.33 KB 4.35 KB 23 B 0.53%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 148μs 541ns 1.95μs 0 0 0 4.33 KB
master EnrichedLog netcoreapp3.1 148μs 123ns 443ns 0 0 0 4.33 KB
master EnrichedLog net472 168μs 77.8ns 291ns 0 0 0 4.54 KB
#6802 EnrichedLog net6.0 124μs 233ns 871ns 0 0 0 4.35 KB
#6802 EnrichedLog netcoreapp3.1 131μs 619ns 2.48μs 0 0 0 4.35 KB
#6802 EnrichedLog net472 169μs 190ns 735ns 0 0 0 4.56 KB
Benchmarks.Trace.NLogBenchmark - Faster 🎉 More allocations ⚠️

Faster 🎉 in #6802

Benchmark base/diff Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.NLogBenchmark.EnrichedLog‑net6.0 1.787 9,122.28 5,103.63

More allocations ⚠️ in #6802

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.NLogBenchmark.EnrichedLog‑net472 2.07 KB 2.12 KB 50 B 2.41%
Benchmarks.Trace.NLogBenchmark.EnrichedLog‑netcoreapp3.1 2.25 KB 2.3 KB 51 B 2.26%
Benchmarks.Trace.NLogBenchmark.EnrichedLog‑net6.0 2.25 KB 2.3 KB 50 B 2.22%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 9.12μs 18ns 69.6ns 0 0 0 2.25 KB
master EnrichedLog netcoreapp3.1 8.11μs 225ns 2.25μs 0 0 0 2.25 KB
master EnrichedLog net472 7.31μs 4.92ns 18.4ns 0 0 0 2.07 KB
#6802 EnrichedLog net6.0 5.09μs 17.8ns 68.8ns 0 0 0 2.3 KB
#6802 EnrichedLog netcoreapp3.1 6.69μs 32ns 132ns 0 0 0 2.3 KB
#6802 EnrichedLog net472 7.66μs 4.85ns 18.8ns 0.306 0 0 2.12 KB
Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net6.0 1.91μs 5.87ns 37.1ns 0 0 0 1.2 KB
master SendReceive netcoreapp3.1 2.53μs 4.98ns 35.9ns 0 0 0 1.2 KB
master SendReceive net472 3.08μs 2.98ns 11.2ns 0.155 0 0 1.2 KB
#6802 SendReceive net6.0 2μs 2.55ns 9.89ns 0 0 0 1.2 KB
#6802 SendReceive netcoreapp3.1 2.53μs 3.65ns 14.1ns 0 0 0 1.2 KB
#6802 SendReceive net472 3.01μs 2.24ns 8.66ns 0.18 0 0 1.2 KB
Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ More allocations ⚠️

More allocations ⚠️ in #6802

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.SerilogBenchmark.EnrichedLog‑net6.0 1.6 KB 1.62 KB 24 B 1.50%
Benchmarks.Trace.SerilogBenchmark.EnrichedLog‑netcoreapp3.1 1.65 KB 1.67 KB 24 B 1.46%
Benchmarks.Trace.SerilogBenchmark.EnrichedLog‑net472 2.04 KB 2.07 KB 29 B 1.42%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 4.23μs 23.5ns 156ns 0 0 0 1.6 KB
master EnrichedLog netcoreapp3.1 6.28μs 96.4ns 964ns 0 0 0 1.65 KB
master EnrichedLog net472 6.46μs 7.1ns 26.6ns 0.322 0 0 2.04 KB
#6802 EnrichedLog net6.0 4.19μs 3.07ns 11.9ns 0 0 0 1.62 KB
#6802 EnrichedLog netcoreapp3.1 5.73μs 18.5ns 71.6ns 0 0 0 1.67 KB
#6802 EnrichedLog net472 6.33μs 7.16ns 26.8ns 0.313 0 0 2.07 KB
Benchmarks.Trace.SpanBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net6.0 716ns 1.49ns 5.78ns 0 0 0 576 B
master StartFinishSpan netcoreapp3.1 917ns 3.27ns 12.2ns 0 0 0 576 B
master StartFinishSpan net472 925ns 0.205ns 0.739ns 0.046 0 0 578 B
master StartFinishScope net6.0 891ns 1.18ns 4.4ns 0 0 0 696 B
master StartFinishScope netcoreapp3.1 1.15μs 0.774ns 2.9ns 0 0 0 696 B
master StartFinishScope net472 1.07μs 2.31ns 8.64ns 0.0536 0 0 658 B
#6802 StartFinishSpan net6.0 732ns 3.97ns 22.1ns 0 0 0 576 B
#6802 StartFinishSpan netcoreapp3.1 958ns 4.64ns 18.6ns 0 0 0 576 B
#6802 StartFinishSpan net472 938ns 0.153ns 0.572ns 0.0904 0 0 578 B
#6802 StartFinishScope net6.0 870ns 4.56ns 20.9ns 0 0 0 696 B
#6802 StartFinishScope netcoreapp3.1 1.14μs 5.83ns 27.4ns 0 0 0 696 B
#6802 StartFinishScope net472 1.1μs 0.107ns 0.401ns 0.0993 0 0 658 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net6.0 973ns 1.57ns 10.3ns 0 0 0 696 B
master RunOnMethodBegin netcoreapp3.1 1.31μs 2.93ns 11.4ns 0 0 0 696 B
master RunOnMethodBegin net472 1.34μs 1.7ns 6.6ns 0.0671 0 0 658 B
#6802 RunOnMethodBegin net6.0 1.01μs 5.01ns 20.6ns 0 0 0 696 B
#6802 RunOnMethodBegin netcoreapp3.1 1.43μs 5.69ns 22ns 0 0 0 696 B
#6802 RunOnMethodBegin net472 1.4μs 0.41ns 1.54ns 0.0979 0 0 658 B

@faydef faydef marked this pull request as ready for review April 7, 2025 07:01
@faydef faydef requested a review from a team as a code owner April 7, 2025 07:01
kevingosse
kevingosse requested changes Apr 8, 2025
View reviewed changes
@faydef @kevingosse
Fix: sync method
f7d6420 
Co-authored-by: Kevin Gosse <[email protected]>
@datadog-ddstaging
Copy link

datadog-ddstaging bot commented Apr 8, 2025 •
edited
Loading

Datadog Report

Branch report: fayssal/test-micro-delivery
Commit report: 6051808
Test service: dd-trace-dotnet

✅ 0 Failed, 4 Passed, 0 Skipped, 0s Total Time

@faydef @bricefriha
Simplify temp file path
4e83a4c 
Co-authored-by: Brice Friha <[email protected]>
bricefriha
bricefriha suggested changes Apr 9, 2025
View reviewed changes
@faydef @bricefriha
Simplify temp file path
c9a93bd 
Co-authored-by: Brice Friha <[email protected]>
@faydef faydef requested review from kevingosse and bricefriha April 9, 2025 11:12
bricefriha
bricefriha approved these changes Apr 10, 2025
View reviewed changes
Copy link
Contributor

@bricefriha bricefriha left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Personally, LGTM, but I'm not a member. So my approval means nothing really 😄
(and the Ci is falling)

@faydef faydef merged commit 8684739 into master Apr 14, 2025
123 of 126 checks passed
@faydef faydef deleted the fayssal/test-micro-delivery branch April 14, 2025 10:52
@github-actions github-actions bot added this to the vNext-v3 milestone Apr 14, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@kevingosse kevingosse kevingosse approved these changes

@bricefriha bricefriha bricefriha approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
vNext-v3
Development

Successfully merging this pull request may close these issues.
4 participants
@faydef @andrewlock @kevingosse @bricefriha