Increase the margin for the number of threads in RuntimeMetricsWriterTests #5329

kevingosse · 2024-03-21T10:07:23Z

Summary of changes

Increase the range of valid values for the number of threads in RuntimeMetricsWriterTests.ShouldCaptureProcessMetrics.

Reason for change

I initially thought that 100 was plenty enough, but we have tests that create a large number of threads to stresstest things (UnmanagedMemoryPoolTests for instance). Let's hope that 500 is enough.

datadog-ddstaging · 2024-03-21T10:22:25Z

Datadog Report

Branch report: kevin/extra_threads
Commit report: 3e4f41e
Test service: dd-trace-dotnet

✅ 0 Failed, 328862 Passed, 1603 Skipped, 35m 31.42s Wall Time

andrewlock · 2024-03-21T10:30:47Z

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

Welch test with statistical test for significance of 5%
Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5329) - mean (72ms)  : 65, 78
     .   : milestone, 72,
    master - mean (73ms)  : 64, 82
     .   : milestone, 73,

    section CallTarget+Inlining+NGEN
    This PR (5329) - mean (1,000ms)  : 975, 1026
     .   : milestone, 1000,
    master - mean (1,002ms)  : 980, 1024
     .   : milestone, 1002,

gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5329) - mean (109ms)  : 106, 111
     .   : milestone, 109,
    master - mean (110ms)  : 106, 114
     .   : milestone, 110,

    section CallTarget+Inlining+NGEN
    This PR (5329) - mean (717ms)  : 696, 738
     .   : milestone, 717,
    master - mean (718ms)  : 697, 738
     .   : milestone, 718,

gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5329) - mean (93ms)  : 91, 96
     .   : milestone, 93,
    master - mean (94ms)  : 90, 97
     .   : milestone, 94,

    section CallTarget+Inlining+NGEN
    This PR (5329) - mean (678ms)  : 646, 710
     .   : milestone, 678,
    master - mean (680ms)  : 661, 699
     .   : milestone, 680,

gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5329) - mean (187ms)  : 185, 190
     .   : milestone, 187,
    master - mean (188ms)  : 185, 191
     .   : milestone, 188,

    section CallTarget+Inlining+NGEN
    This PR (5329) - mean (1,079ms)  : 1051, 1107
     .   : milestone, 1079,
    master - mean (1,078ms)  : 1060, 1097
     .   : milestone, 1078,

gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5329) - mean (271ms)  : 266, 276
     .   : milestone, 271,
    master - mean (271ms)  : 267, 275
     .   : milestone, 271,

    section CallTarget+Inlining+NGEN
    This PR (5329) - mean (874ms)  : 851, 896
     .   : milestone, 874,
    master - mean (872ms)  : 849, 894
     .   : milestone, 872,

gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5329) - mean (260ms)  : 256, 264
     .   : milestone, 260,
    master - mean (261ms)  : 257, 265
     .   : milestone, 261,

    section CallTarget+Inlining+NGEN
    This PR (5329) - mean (852ms)  : 822, 883
     .   : milestone, 852,
    master - mean (856ms)  : 830, 882
     .   : milestone, 856,

andrewlock · 2024-03-21T10:48:56Z

Benchmarks Report for tracer 🐌

Benchmarks for #5329 compared to master:

1 benchmarks are faster, with geometric mean 1.177
All benchmarks have the same allocations

The following thresholds were used for comparing the benchmark speeds:

Mann–Whitney U test with statistical test for significance of 5%
Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	`StartStopWithChild`	net6.0	9.67μs	59.4ns	591ns	0.0185	0.00461	0	7.51 KB
master	`StartStopWithChild`	netcoreapp3.1	11μs	57.8ns	289ns	0.022	0.011	0	7.61 KB
master	`StartStopWithChild`	net472	18.7μs	128ns	1.27μs	1.33	0.333	0.0965	8.05 KB
#5329	`StartStopWithChild`	net6.0	8.94μs	49.8ns	327ns	0.0224	0.00895	0	7.5 KB
#5329	`StartStopWithChild`	netcoreapp3.1	10.9μs	59.3ns	366ns	0.0263	0.0105	0	7.61 KB
#5329	`StartStopWithChild`	net472	17.1μs	67ns	259ns	1.35	0.35	0.102	8.06 KB

Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`WriteAndFlushEnrichedTraces`	net6.0	471μs	162ns	626ns	0	2.7 KB
master	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	622μs	300ns	1.16μs	0	2.7 KB
master	`WriteAndFlushEnrichedTraces`	net472	825μs	328ns	1.23μs	0.408	3.3 KB
#5329	`WriteAndFlushEnrichedTraces`	net6.0	454μs	193ns	696ns	0	2.7 KB
#5329	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	621μs	119ns	430ns	0	2.7 KB
#5329	`WriteAndFlushEnrichedTraces`	net472	805μs	261ns	941ns	0.403	3.3 KB

Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendRequest`	net6.0	171μs	226ns	844ns	0.17	18.33 KB
master	`SendRequest`	netcoreapp3.1	196μs	237ns	918ns	0.195	20.49 KB
master	`SendRequest`	net472	0.000187ns	0.00014ns	0.000522ns	0	0 b
#5329	`SendRequest`	net6.0	170μs	191ns	716ns	0.255	18.33 KB
#5329	`SendRequest`	netcoreapp3.1	194μs	293ns	1.13μs	0.193	20.49 KB
#5329	`SendRequest`	net472	0.00135ns	0.000572ns	0.00222ns	0	0 b

Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	`WriteAndFlushEnrichedTraces`	net6.0	553μs	259ns	967ns	0.546	0	0	41.7 KB
master	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	651μs	1.2μs	4.66μs	0.324	0	0	41.81 KB
master	`WriteAndFlushEnrichedTraces`	net472	858μs	2.83μs	10.6μs	8.19	2.59	0.431	53.24 KB
#5329	`WriteAndFlushEnrichedTraces`	net6.0	555μs	994ns	3.85μs	0.556	0	0	41.78 KB
#5329	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	657μs	541ns	2.09μs	0.319	0	0	41.73 KB
#5329	`WriteAndFlushEnrichedTraces`	net472	893μs	4.06μs	16.7μs	8.04	2.23	0.446	53.24 KB

Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`ExecuteNonQuery`	net6.0	1.18μs	1.03ns	4.01ns	0.0112	784 B
master	`ExecuteNonQuery`	netcoreapp3.1	1.64μs	0.402ns	1.45ns	0.0107	784 B
master	`ExecuteNonQuery`	net472	1.78μs	0.905ns	3.5ns	0.119	746 B
#5329	`ExecuteNonQuery`	net6.0	1.27μs	0.475ns	1.84ns	0.0108	784 B
#5329	`ExecuteNonQuery`	netcoreapp3.1	1.49μs	1.63ns	6.31ns	0.0104	784 B
#5329	`ExecuteNonQuery`	net472	1.87μs	0.484ns	1.81ns	0.119	746 B

Benchmarks.Trace.ElasticsearchBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`CallElasticsearch`	net6.0	1.3μs	1.95ns	7.54ns	0.014	0	1.01 KB
master	`CallElasticsearch`	netcoreapp3.1	1.66μs	1.13ns	4.39ns	0.0136	0	1.01 KB
master	`CallElasticsearch`	net472	2.57μs	0.802ns	3ns	0.161	0.00129	1.02 KB
master	`CallElasticsearchAsync`	net6.0	1.42μs	1.11ns	4.3ns	0.0142	0	984 B
master	`CallElasticsearchAsync`	netcoreapp3.1	1.66μs	2.61ns	9.78ns	0.0141	0	1.06 KB
master	`CallElasticsearchAsync`	net472	2.68μs	1.54ns	5.55ns	0.171	0.00134	1.08 KB
#5329	`CallElasticsearch`	net6.0	1.36μs	1.87ns	7.01ns	0.014	0	1.01 KB
#5329	`CallElasticsearch`	netcoreapp3.1	1.64μs	8.87ns	48.6ns	0.0138	0	1.01 KB
#5329	`CallElasticsearch`	net472	2.6μs	2.01ns	7.78ns	0.162	0	1.02 KB
#5329	`CallElasticsearchAsync`	net6.0	1.45μs	0.743ns	2.78ns	0.0138	0	984 B
#5329	`CallElasticsearchAsync`	netcoreapp3.1	1.73μs	0.958ns	3.71ns	0.0139	0	1.06 KB
#5329	`CallElasticsearchAsync`	net472	2.65μs	1.66ns	6.42ns	0.17	0	1.08 KB

Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`ExecuteAsync`	net6.0	1.28μs	0.967ns	3.74ns	0.0128	928 B
master	`ExecuteAsync`	netcoreapp3.1	1.67μs	1.58ns	5.9ns	0.0124	928 B
master	`ExecuteAsync`	net472	1.9μs	0.762ns	2.95ns	0.141	891 B
#5329	`ExecuteAsync`	net6.0	1.33μs	0.611ns	2.29ns	0.0133	928 B
#5329	`ExecuteAsync`	netcoreapp3.1	1.68μs	1.17ns	4.53ns	0.0125	928 B
#5329	`ExecuteAsync`	net472	1.98μs	1.02ns	3.82ns	0.141	891 B

Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendAsync`	net6.0	4.2μs	2.79ns	10.8ns	0.0294	2.16 KB
master	`SendAsync`	netcoreapp3.1	5.18μs	2.42ns	9.39ns	0.0361	2.7 KB
master	`SendAsync`	net472	7.73μs	4.03ns	15.1ns	0.484	3.05 KB
#5329	`SendAsync`	net6.0	4.15μs	1.76ns	6.81ns	0.031	2.16 KB
#5329	`SendAsync`	netcoreapp3.1	5.01μs	1.55ns	5.79ns	0.0351	2.7 KB
#5329	`SendAsync`	net472	7.73μs	3.55ns	13.7ns	0.482	3.05 KB

Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	1.52μs	0.695ns	2.6ns	0.0229	1.65 KB
master	`EnrichedLog`	netcoreapp3.1	2.3μs	8.34ns	32.3ns	0.0227	1.65 KB
master	`EnrichedLog`	net472	2.83μs	3.61ns	14ns	0.249	1.57 KB
#5329	`EnrichedLog`	net6.0	1.49μs	0.521ns	1.8ns	0.0232	1.65 KB
#5329	`EnrichedLog`	netcoreapp3.1	2.31μs	1.01ns	3.77ns	0.0231	1.65 KB
#5329	`EnrichedLog`	net472	2.61μs	2.27ns	8.79ns	0.249	1.57 KB

Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`EnrichedLog`	net6.0	112μs	64.4ns	241ns	0.0562	0	4.23 KB
master	`EnrichedLog`	netcoreapp3.1	118μs	155ns	599ns	0	0	4.23 KB
master	`EnrichedLog`	net472	147μs	127ns	494ns	0.657	0.219	4.41 KB
#5329	`EnrichedLog`	net6.0	114μs	301ns	1.16μs	0.0564	0	4.23 KB
#5329	`EnrichedLog`	netcoreapp3.1	119μs	90ns	349ns	0.0595	0	4.23 KB
#5329	`EnrichedLog`	net472	146μs	109ns	420ns	0.657	0.219	4.41 KB

Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	3.07μs	1.03ns	3.85ns	0.0307	2.21 KB
master	`EnrichedLog`	netcoreapp3.1	4.29μs	4.46ns	17.3ns	0.0299	2.21 KB
master	`EnrichedLog`	net472	5.09μs	1.85ns	7.18ns	0.32	2.02 KB
#5329	`EnrichedLog`	net6.0	3.04μs	0.981ns	3.67ns	0.0303	2.21 KB
#5329	`EnrichedLog`	netcoreapp3.1	4.26μs	2.67ns	10.4ns	0.0296	2.21 KB
#5329	`EnrichedLog`	net472	4.88μs	2.05ns	7.93ns	0.321	2.02 KB

Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendReceive`	net6.0	1.37μs	0.867ns	3.36ns	0.0164	1.18 KB
master	`SendReceive`	netcoreapp3.1	1.89μs	0.891ns	3.34ns	0.016	1.18 KB
master	`SendReceive`	net472	2.36μs	1.76ns	6.81ns	0.187	1.18 KB
#5329	`SendReceive`	net6.0	1.39μs	0.419ns	1.57ns	0.016	1.18 KB
#5329	`SendReceive`	netcoreapp3.1	1.76μs	2.44ns	9.44ns	0.0157	1.18 KB
#5329	`SendReceive`	net472	2.24μs	1.57ns	6.09ns	0.187	1.18 KB

Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	2.64μs	0.717ns	2.68ns	0.0211	1.55 KB
master	`EnrichedLog`	netcoreapp3.1	4.06μs	1.01ns	3.66ns	0.0205	1.6 KB
master	`EnrichedLog`	net472	4.51μs	1.97ns	7.37ns	0.316	1.99 KB
#5329	`EnrichedLog`	net6.0	2.61μs	0.586ns	2.19ns	0.0211	1.55 KB
#5329	`EnrichedLog`	netcoreapp3.1	3.96μs	1.45ns	5.62ns	0.0216	1.6 KB
#5329	`EnrichedLog`	net472	4.49μs	2.75ns	10.7ns	0.314	1.99 KB

Benchmarks.Trace.SpanBenchmark - Faster 🎉 Same allocations ✔️

Faster 🎉 in #5329

Benchmark	base/diff	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.SpanBenchmark.StartFinishScope‑net472	1.177	1,053.19	894.68

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`StartFinishSpan`	net6.0	483ns	0.807ns	3.02ns	0.00761	552 B
master	`StartFinishSpan`	netcoreapp3.1	710ns	0.615ns	2.38ns	0.00752	552 B
master	`StartFinishSpan`	net472	782ns	0.396ns	1.53ns	0.0878	554 B
master	`StartFinishScope`	net6.0	620ns	0.915ns	3.55ns	0.00928	672 B
master	`StartFinishScope`	netcoreapp3.1	840ns	0.78ns	3.02ns	0.00918	672 B
master	`StartFinishScope`	net472	1.05μs	1.14ns	4.28ns	0.101	634 B
#5329	`StartFinishSpan`	net6.0	538ns	0.695ns	2.69ns	0.00768	552 B
#5329	`StartFinishSpan`	netcoreapp3.1	712ns	0.764ns	2.86ns	0.00735	552 B
#5329	`StartFinishSpan`	net472	778ns	0.6ns	2.32ns	0.0879	554 B
#5329	`StartFinishScope`	net6.0	571ns	0.305ns	1.18ns	0.0095	672 B
#5329	`StartFinishScope`	netcoreapp3.1	876ns	0.952ns	3.69ns	0.00919	672 B
#5329	`StartFinishScope`	net472	895ns	0.573ns	2.22ns	0.1	634 B

Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`RunOnMethodBegin`	net6.0	683ns	0.781ns	3.03ns	0.00958	672 B
master	`RunOnMethodBegin`	netcoreapp3.1	955ns	0.782ns	3.03ns	0.00911	672 B
master	`RunOnMethodBegin`	net472	1.17μs	1.6ns	5.76ns	0.1	634 B
#5329	`RunOnMethodBegin`	net6.0	631ns	0.176ns	0.681ns	0.00954	672 B
#5329	`RunOnMethodBegin`	netcoreapp3.1	858ns	0.876ns	3.39ns	0.00887	672 B
#5329	`RunOnMethodBegin`	net472	1.07μs	1.04ns	4.02ns	0.1	634 B

andrewlock · 2024-03-21T11:44:58Z

Throughput/Crank Report:zap:

Throughput results for AspNetCoreSimpleController comparing the following branches/commits:

Cases where throughput results for the PR are worse than latest master (5% drop or greater), results are shown in red.

Note that these results are based on a single point-in-time result for each branch. For full results, see one of the many, many dashboards!

gantt
    title Throughput Linux x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (5329) (10.928M)   : 0, 10927662
    master (11.096M)   : 0, 11096138
    benchmarks/2.9.0 (10.743M)   : 0, 10743092

    section Automatic
    This PR (5329) (7.430M)   : 0, 7429964
    master (7.482M)   : 0, 7481686
    benchmarks/2.9.0 (7.823M)   : 0, 7823252

    section Trace stats
    This PR (5329) (7.904M)   : 0, 7904001
    master (7.944M)   : 0, 7944133

    section Manual
    This PR (5329) (9.577M)   : 0, 9577327
    master (9.429M)   : 0, 9429044

    section Manual + Automatic
    This PR (5329) (6.935M)   : 0, 6935119
    master (6.975M)   : 0, 6975116

    section Version Conflict
    This PR (5329) (6.354M)   : 0, 6353679
    master (6.384M)   : 0, 6383585

gantt
    title Throughput Linux arm64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (5329) (9.589M)   : 0, 9589189
    master (9.571M)   : 0, 9570993
    benchmarks/2.9.0 (9.656M)   : 0, 9656465

    section Automatic
    This PR (5329) (6.649M)   : 0, 6648870
    master (6.470M)   : 0, 6469945

    section Trace stats
    This PR (5329) (6.927M)   : 0, 6927415
    master (7.095M)   : 0, 7095354

    section Manual
    This PR (5329) (8.332M)   : 0, 8331933
    master (8.040M)   : 0, 8039576

    section Manual + Automatic
    This PR (5329) (6.125M)   : 0, 6124662
    master (6.260M)   : 0, 6259761

    section Version Conflict
    This PR (5329) (5.696M)   : 0, 5695694
    master (5.905M)   : 0, 5905251

gantt
    title Throughput Windows x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (5329) (9.824M)   : 0, 9824229
    master (10.013M)   : 0, 10013130
    benchmarks/2.9.0 (9.967M)   : 0, 9967407

    section Automatic
    This PR (5329) (6.854M)   : 0, 6854040
    master (7.046M)   : 0, 7046320
    benchmarks/2.9.0 (7.375M)   : 0, 7374511

    section Trace stats
    This PR (5329) (7.243M)   : 0, 7242879
    master (7.437M)   : 0, 7437175

    section Manual
    This PR (5329) (8.808M)   : 0, 8807653
    master (8.864M)   : 0, 8864242

    section Manual + Automatic
    This PR (5329) (6.716M)   : 0, 6715852
    master (6.834M)   : 0, 6834408

    section Version Conflict
    This PR (5329) (6.064M)   : 0, 6064203
    master (6.123M)   : 0, 6122721

andrewlock

Classic

andrewlock · 2024-03-21T14:09:15Z

tracer/test/Datadog.Trace.Tests/RuntimeMetrics/RuntimeMetricsWriterTests.cs

-                // To future generations: if 100 is not enough, feel free to bump it up. We're really just checking that the value is "realistic".
-                actualNumberOfThreads.Should().NotBeNull().And.BeGreaterThan(0).And.BeInRange(expectedNumberOfThreads - 100, expectedNumberOfThreads + 100);
+                // A margin of 500 threads seem like a lot, but we have tests that spawn a large number of threads to try to find race conditions
+                actualNumberOfThreads.Should().NotBeNull().And.BeGreaterThan(0).And.BeInRange(expectedNumberOfThreads - 500, expectedNumberOfThreads + 500);


Is this going to be safer? It feels like we're mostly just checking "is a number" here 😅

Suggested change

actualNumberOfThreads.Should().NotBeNull().And.BeGreaterThan(0).And.BeInRange(expectedNumberOfThreads - 500, expectedNumberOfThreads + 500);

actualNumberOfThreads.Should().NotBeNull().And.BeGreaterThanOrEqualTo(expectedNumberOfThreads );

"Is a number bigger than 0 and not too crazy" 😛

not too crazy

expectedNumberOfThreads + 500 seems pretty crazy to me 😛

Force a crash when too many threads

c43ef73

github-actions bot added the area:tests unit tests, integration tests label Mar 21, 2024

Try another way

4edd3cd

kevingosse added 2 commits March 21, 2024 14:34

Increate number of threads

b58a103

Cleanup

3e4f41e

kevingosse changed the title ~~Force a crash when too many threads~~ Increase the margin for the number of threads in RuntimeMetricsWriterTests Mar 21, 2024

kevingosse marked this pull request as ready for review March 21, 2024 13:37

kevingosse requested a review from a team as a code owner March 21, 2024 13:37

andrewlock approved these changes Mar 21, 2024

View reviewed changes

kevingosse merged commit 6249896 into master Mar 21, 2024
56 checks passed

kevingosse deleted the kevin/extra_threads branch March 21, 2024 18:38

github-actions bot added this to the vNext milestone Mar 21, 2024

andrewlock modified the milestones: vNext, vNext-v2 Apr 17, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Increase the margin for the number of threads in RuntimeMetricsWriterTests #5329

Increase the margin for the number of threads in RuntimeMetricsWriterTests #5329

kevingosse commented Mar 21, 2024 •

edited

Loading

datadog-ddstaging bot commented Mar 21, 2024 •

edited

Loading

andrewlock commented Mar 21, 2024 •

edited

Loading

andrewlock commented Mar 21, 2024 •

edited

Loading

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Faster 🎉 in #5329

Raw results

Raw results

andrewlock commented Mar 21, 2024 •

edited

Loading

andrewlock left a comment

andrewlock Mar 21, 2024

kevingosse Mar 21, 2024

andrewlock Mar 22, 2024

	actualNumberOfThreads.Should().NotBeNull().And.BeGreaterThan(0).And.BeInRange(expectedNumberOfThreads - 500, expectedNumberOfThreads + 500);
	actualNumberOfThreads.Should().NotBeNull().And.BeGreaterThanOrEqualTo(expectedNumberOfThreads );

Increase the margin for the number of threads in RuntimeMetricsWriterTests #5329

Increase the margin for the number of threads in RuntimeMetricsWriterTests #5329

Conversation

kevingosse commented Mar 21, 2024 • edited Loading

Summary of changes

Reason for change

datadog-ddstaging bot commented Mar 21, 2024 • edited Loading

Datadog Report

andrewlock commented Mar 21, 2024 • edited Loading

Execution-Time Benchmarks Report ⏱️

andrewlock commented Mar 21, 2024 • edited Loading

Benchmarks Report for tracer 🐌

Benchmark details

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Faster 🎉 in #5329

Raw results

Raw results

andrewlock commented Mar 21, 2024 • edited Loading

Throughput/Crank Report:zap:

andrewlock left a comment

Choose a reason for hiding this comment

andrewlock Mar 21, 2024

Choose a reason for hiding this comment

kevingosse Mar 21, 2024

Choose a reason for hiding this comment

andrewlock Mar 22, 2024

Choose a reason for hiding this comment

kevingosse commented Mar 21, 2024 •

edited

Loading

datadog-ddstaging bot commented Mar 21, 2024 •

edited

Loading

andrewlock commented Mar 21, 2024 •

edited

Loading

andrewlock commented Mar 21, 2024 •

edited

Loading

andrewlock commented Mar 21, 2024 •

edited

Loading