If a lot of snapshots have changed, we should check them all #5563

andrewlock · 2024-05-15T08:21:57Z

Summary of changes

Fix missing integration-change detection for callsite integrations
If a lot of snapshots have changed, runn all TFMs

Reason for change

We've had several cases recently (e.g. #5545) where a PR merge has broken master because some tests are not run on the PR. This improves the "risky" detection to try to prevent master breaking so much

Implementation details

We already detect when integration hook points are added or changed, but that was only looking at calltarget previously. This adds support for checking for aspects too (callsite).

Additionally, if there are a lot of snapshot changes (> 100) we also trigger checking other TFMs, as there's a high probability the non-tested TFMs will be impacted

Test coverage

I haven't tested it, easier to observe in the wild and make sure it's working

Other details

kevingosse

100 is totally the right number nods nods

datadog-ddstaging · 2024-05-15T08:39:06Z

Datadog Report

Branch report: andrew/ci/run-all-tfms-in-more-scenarios
Commit report: ff67778
Test service: dd-trace-dotnet

✅ 0 Failed, 336959 Passed, 1565 Skipped, 13h 36m 8.86s Total Time
❄️ 1 New Flaky

New Flaky Tests (1)

Finish_StartTimeInThePastWithNoEndTime_DurationProperlyComputed - Datadog.Trace.Tests.SpanTests - Last Failure
Expand for error
```
 Assert.True() Failure
 Expected: True
 Actual:   False
```

andrewlock · 2024-05-15T08:46:11Z

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

Welch test with statistical test for significance of 5%
Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5563) - mean (73ms)  : 66, 81
     .   : milestone, 73,
    master - mean (71ms)  : 66, 76
     .   : milestone, 71,

    section CallTarget+Inlining+NGEN
    This PR (5563) - mean (997ms)  : 967, 1028
     .   : milestone, 997,
    master - mean (993ms)  : 968, 1017
     .   : milestone, 993,

gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5563) - mean (111ms)  : 108, 114
     .   : milestone, 111,
    master - mean (110ms)  : 107, 112
     .   : milestone, 110,

    section CallTarget+Inlining+NGEN
    This PR (5563) - mean (710ms)  : 692, 729
     .   : milestone, 710,
    master - mean (700ms)  : 676, 724
     .   : milestone, 700,

gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5563) - mean (96ms)  : 93, 99
     .   : milestone, 96,
    master - mean (94ms)  : 90, 99
     .   : milestone, 94,

    section CallTarget+Inlining+NGEN
    This PR (5563) - mean (655ms)  : 635, 674
     .   : milestone, 655,
    master - mean (655ms)  : 634, 677
     .   : milestone, 655,

gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5563) - mean (192ms)  : 188, 196
     .   : milestone, 192,
    master - mean (191ms)  : 187, 196
     .   : milestone, 191,

    section CallTarget+Inlining+NGEN
    This PR (5563) - mean (1,116ms)  : 1090, 1143
     .   : milestone, 1116,
    master - mean (1,112ms)  : 1090, 1134
     .   : milestone, 1112,

gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5563) - mean (274ms)  : 270, 277
     .   : milestone, 274,
    master - mean (276ms)  : 272, 280
     .   : milestone, 276,

    section CallTarget+Inlining+NGEN
    This PR (5563) - mean (913ms)  : 896, 931
     .   : milestone, 913,
    master - mean (912ms)  : 898, 927
     .   : milestone, 912,

gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5563) - mean (264ms)  : 260, 269
     .   : milestone, 264,
    master - mean (264ms)  : 260, 267
     .   : milestone, 264,

    section CallTarget+Inlining+NGEN
    This PR (5563) - mean (905ms)  : 886, 923
     .   : milestone, 905,
    master - mean (896ms)  : 877, 915
     .   : milestone, 896,

andrewlock · 2024-05-15T08:58:37Z

Benchmarks Report for tracer 🐌

Benchmarks for #5563 compared to master:

2 benchmarks are faster, with geometric mean 1.138
All benchmarks have the same allocations

The following thresholds were used for comparing the benchmark speeds:

Mann–Whitney U test with statistical test for significance of 5%
Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	`StartStopWithChild`	net6.0	8.84μs	43ns	188ns	0.0305	0.0087	0	7.56 KB
master	`StartStopWithChild`	netcoreapp3.1	10.9μs	58.9ns	312ns	0.0218	0.0109	0	7.66 KB
master	`StartStopWithChild`	net472	17.4μs	38.6ns	144ns	1.34	0.325	0.111	8.06 KB
#5563	`StartStopWithChild`	net6.0	8.67μs	46.2ns	253ns	0.0252	0.0126	0.0042	7.55 KB
#5563	`StartStopWithChild`	netcoreapp3.1	10.9μs	58.9ns	338ns	0.0304	0.0152	0	7.65 KB
#5563	`StartStopWithChild`	net472	17.4μs	74.5ns	289ns	1.35	0.348	0.0934	8.07 KB

Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`WriteAndFlushEnrichedTraces`	net6.0	482μs	403ns	1.51μs	0	2.7 KB
master	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	639μs	570ns	2.21μs	0	2.7 KB
master	`WriteAndFlushEnrichedTraces`	net472	833μs	654ns	2.53μs	0.417	3.3 KB
#5563	`WriteAndFlushEnrichedTraces`	net6.0	476μs	208ns	804ns	0	2.7 KB
#5563	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	621μs	255ns	986ns	0	2.7 KB
#5563	`WriteAndFlushEnrichedTraces`	net472	831μs	348ns	1.35μs	0.414	3.3 KB

Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendRequest`	net6.0	167μs	129ns	499ns	0.249	18.49 KB
master	`SendRequest`	netcoreapp3.1	192μs	283ns	1.09μs	0.192	20.65 KB
master	`SendRequest`	net472	0.000334ns	0.000156ns	0.000539ns	0	0 b
#5563	`SendRequest`	net6.0	170μs	150ns	580ns	0.169	18.49 KB
#5563	`SendRequest`	netcoreapp3.1	191μs	290ns	1.12μs	0.192	20.65 KB
#5563	`SendRequest`	net472	0.00076ns	0.000293ns	0.0011ns	0	0 b

Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	`WriteAndFlushEnrichedTraces`	net6.0	547μs	1μs	3.47μs	0.548	0	0	41.6 KB
master	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	667μs	1.74μs	6.27μs	0.332	0	0	41.58 KB
master	`WriteAndFlushEnrichedTraces`	net472	843μs	4.01μs	16.5μs	8.25	2.6	0.434	53.24 KB
#5563	`WriteAndFlushEnrichedTraces`	net6.0	539μs	853ns	3.3μs	0.532	0	0	41.6 KB
#5563	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	659μs	1.85μs	7.18μs	0.331	0	0	41.55 KB
#5563	`WriteAndFlushEnrichedTraces`	net472	876μs	3.02μs	11.7μs	8.25	2.6	0.434	53.26 KB

Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`ExecuteNonQuery`	net6.0	1.13μs	0.583ns	2.26ns	0.0114	808 B
master	`ExecuteNonQuery`	netcoreapp3.1	1.51μs	0.678ns	2.62ns	0.0107	808 B
master	`ExecuteNonQuery`	net472	1.82μs	0.446ns	1.73ns	0.122	770 B
#5563	`ExecuteNonQuery`	net6.0	1.11μs	0.706ns	2.64ns	0.0116	808 B
#5563	`ExecuteNonQuery`	netcoreapp3.1	1.56μs	1.04ns	3.89ns	0.0109	808 B
#5563	`ExecuteNonQuery`	net472	1.76μs	0.864ns	3.35ns	0.122	770 B

Benchmarks.Trace.ElasticsearchBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`CallElasticsearch`	net6.0	1.29μs	1.47ns	5.7ns	0.0144	0	1.03 KB
master	`CallElasticsearch`	netcoreapp3.1	1.55μs	1.59ns	6.15ns	0.0145	0	1.03 KB
master	`CallElasticsearch`	net472	2.52μs	1.11ns	4.16ns	0.165	0.00126	1.04 KB
master	`CallElasticsearchAsync`	net6.0	1.32μs	0.794ns	3.08ns	0.0143	0	1.01 KB
master	`CallElasticsearchAsync`	netcoreapp3.1	1.67μs	3.77ns	14.6ns	0.0142	0	1.08 KB
master	`CallElasticsearchAsync`	net472	2.69μs	1.91ns	7.41ns	0.174	0	1.1 KB
#5563	`CallElasticsearch`	net6.0	1.24μs	1.02ns	3.95ns	0.0142	0	1.03 KB
#5563	`CallElasticsearch`	netcoreapp3.1	1.6μs	1.29ns	4.99ns	0.0138	0	1.03 KB
#5563	`CallElasticsearch`	net472	2.47μs	2.32ns	9ns	0.165	0	1.04 KB
#5563	`CallElasticsearchAsync`	net6.0	1.26μs	0.931ns	3.61ns	0.0138	0	1.01 KB
#5563	`CallElasticsearchAsync`	netcoreapp3.1	1.7μs	1.31ns	4.89ns	0.0146	0	1.08 KB
#5563	`CallElasticsearchAsync`	net472	2.68μs	1.94ns	7.26ns	0.175	0.00134	1.1 KB

Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`ExecuteAsync`	net6.0	1.34μs	0.799ns	3.1ns	0.013	952 B
master	`ExecuteAsync`	netcoreapp3.1	1.68μs	0.765ns	2.65ns	0.0126	952 B
master	`ExecuteAsync`	net472	1.74μs	0.567ns	2.2ns	0.145	915 B
#5563	`ExecuteAsync`	net6.0	1.24μs	0.803ns	3.01ns	0.0137	952 B
#5563	`ExecuteAsync`	netcoreapp3.1	1.6μs	0.712ns	2.66ns	0.0128	952 B
#5563	`ExecuteAsync`	net472	1.78μs	1.14ns	4.4ns	0.145	915 B

Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendAsync`	net6.0	4.18μs	1.19ns	4.62ns	0.0314	2.27 KB
master	`SendAsync`	netcoreapp3.1	5.36μs	2.77ns	10.4ns	0.0374	2.81 KB
master	`SendAsync`	net472	7.83μs	3.62ns	14ns	0.504	3.18 KB
#5563	`SendAsync`	net6.0	4.22μs	4.59ns	16.6ns	0.0319	2.27 KB
#5563	`SendAsync`	netcoreapp3.1	4.98μs	2.21ns	8.26ns	0.0375	2.81 KB
#5563	`SendAsync`	net472	7.93μs	2.92ns	11.3ns	0.502	3.18 KB

Benchmarks.Trace.ILoggerBenchmark - Faster 🎉 Same allocations ✔️

Faster 🎉 in #5563

Benchmark	base/diff	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.ILoggerBenchmark.EnrichedLog‑net6.0	1.122	1,612.54	1,436.65

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	1.61μs	0.332ns	1.29ns	0.0234	1.7 KB
master	`EnrichedLog`	netcoreapp3.1	2.16μs	1.69ns	6.32ns	0.0227	1.7 KB
master	`EnrichedLog`	net472	2.73μs	2.74ns	10.3ns	0.257	1.62 KB
#5563	`EnrichedLog`	net6.0	1.44μs	0.754ns	2.92ns	0.0237	1.7 KB
#5563	`EnrichedLog`	netcoreapp3.1	2.23μs	1.02ns	3.8ns	0.0226	1.7 KB
#5563	`EnrichedLog`	net472	2.67μs	2.39ns	9.25ns	0.257	1.62 KB

Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`EnrichedLog`	net6.0	113μs	193ns	746ns	0.0565	0	4.28 KB
master	`EnrichedLog`	netcoreapp3.1	121μs	202ns	784ns	0	0	4.28 KB
master	`EnrichedLog`	net472	151μs	174ns	672ns	0.676	0.225	4.46 KB
#5563	`EnrichedLog`	net6.0	113μs	231ns	893ns	0.0561	0	4.28 KB
#5563	`EnrichedLog`	netcoreapp3.1	117μs	110ns	425ns	0.0581	0	4.28 KB
#5563	`EnrichedLog`	net472	148μs	389ns	1.51μs	0.676	0.225	4.46 KB

Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	3.09μs	0.976ns	3.65ns	0.0307	2.25 KB
master	`EnrichedLog`	netcoreapp3.1	4.15μs	1.95ns	7.56ns	0.029	2.25 KB
master	`EnrichedLog`	net472	4.69μs	1.52ns	5.7ns	0.327	2.07 KB
#5563	`EnrichedLog`	net6.0	3.06μs	0.75ns	2.81ns	0.0307	2.25 KB
#5563	`EnrichedLog`	netcoreapp3.1	4.18μs	2.49ns	9.31ns	0.0313	2.25 KB
#5563	`EnrichedLog`	net472	4.89μs	1.49ns	5.75ns	0.328	2.07 KB

Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`SendReceive`	net6.0	1.33μs	1.12ns	4.33ns	0.0167	0	1.2 KB
master	`SendReceive`	netcoreapp3.1	1.85μs	1.27ns	4.93ns	0.0158	0	1.2 KB
master	`SendReceive`	net472	2.24μs	0.574ns	2.07ns	0.191	0.00113	1.2 KB
#5563	`SendReceive`	net6.0	1.37μs	0.814ns	3.15ns	0.0171	0	1.2 KB
#5563	`SendReceive`	netcoreapp3.1	1.79μs	1.91ns	7.4ns	0.0162	0	1.2 KB
#5563	`SendReceive`	net472	2.31μs	1.05ns	3.92ns	0.191	0	1.2 KB

Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	2.76μs	1.13ns	4.08ns	0.0221	1.6 KB
master	`EnrichedLog`	netcoreapp3.1	3.93μs	1.23ns	4.62ns	0.0217	1.65 KB
master	`EnrichedLog`	net472	4.55μs	3.5ns	13.6ns	0.322	2.04 KB
#5563	`EnrichedLog`	net6.0	2.73μs	1.05ns	4.05ns	0.0218	1.6 KB
#5563	`EnrichedLog`	netcoreapp3.1	3.94μs	3.19ns	11.9ns	0.0217	1.65 KB
#5563	`EnrichedLog`	net472	4.3μs	4.72ns	18.3ns	0.323	2.04 KB

Benchmarks.Trace.SpanBenchmark - Faster 🎉 Same allocations ✔️

Faster 🎉 in #5563

Benchmark	base/diff	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.SpanBenchmark.StartFinishScope‑net6.0	1.154	631.13	546.95

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`StartFinishSpan`	net6.0	464ns	0.268ns	1ns	0.00815	576 B
master	`StartFinishSpan`	netcoreapp3.1	673ns	0.521ns	2.02ns	0.00782	576 B
master	`StartFinishSpan`	net472	779ns	0.48ns	1.86ns	0.0917	578 B
master	`StartFinishScope`	net6.0	631ns	0.953ns	3.69ns	0.00985	696 B
master	`StartFinishScope`	netcoreapp3.1	904ns	1.81ns	6.76ns	0.00963	696 B
master	`StartFinishScope`	net472	1.02μs	0.347ns	1.25ns	0.104	658 B
#5563	`StartFinishSpan`	net6.0	496ns	0.219ns	0.791ns	0.00793	576 B
#5563	`StartFinishSpan`	netcoreapp3.1	680ns	0.371ns	1.39ns	0.00769	576 B
#5563	`StartFinishSpan`	net472	794ns	0.384ns	1.49ns	0.0915	578 B
#5563	`StartFinishScope`	net6.0	547ns	0.181ns	0.678ns	0.00966	696 B
#5563	`StartFinishScope`	netcoreapp3.1	859ns	0.414ns	1.6ns	0.00945	696 B
#5563	`StartFinishScope`	net472	1.01μs	1.11ns	4.3ns	0.104	658 B

Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`RunOnMethodBegin`	net6.0	601ns	0.483ns	1.87ns	0.00983	696 B
master	`RunOnMethodBegin`	netcoreapp3.1	990ns	1.37ns	5.29ns	0.00939	696 B
master	`RunOnMethodBegin`	net472	1.11μs	0.671ns	2.6ns	0.104	658 B
#5563	`RunOnMethodBegin`	net6.0	645ns	0.242ns	0.937ns	0.0097	696 B
#5563	`RunOnMethodBegin`	netcoreapp3.1	908ns	0.616ns	2.3ns	0.00923	696 B
#5563	`RunOnMethodBegin`	net472	1.21μs	0.846ns	3.28ns	0.104	658 B

andrewlock · 2024-05-15T09:09:04Z

Throughput/Crank Report:zap:

Throughput results for AspNetCoreSimpleController comparing the following branches/commits:

Cases where throughput results for the PR are worse than latest master (5% drop or greater), results are shown in red.

Note that these results are based on a single point-in-time result for each branch. For full results, see one of the many, many dashboards!

gantt
    title Throughput Linux x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (5563) (11.881M)   : 0, 11881446
    master (11.862M)   : 0, 11862130
    benchmarks/2.9.0 (11.817M)   : 0, 11816893

    section Automatic
    This PR (5563) (8.121M)   : 0, 8121168
    master (8.015M)   : 0, 8014959
    benchmarks/2.9.0 (8.480M)   : 0, 8480079

    section Trace stats
    master (8.328M)   : 0, 8328315

    section Manual
    This PR (5563) (10.393M)   : 0, 10393265
    master (10.258M)   : 0, 10258408

    section Manual + Automatic
    This PR (5563) (7.594M)   : 0, 7593927
    master (7.540M)   : 0, 7540293

    section Version Conflict
    master (6.769M)   : 0, 6769041

gantt
    title Throughput Linux arm64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (5563) (9.445M)   : 0, 9444694
    master (9.688M)   : 0, 9688323
    benchmarks/2.9.0 (9.598M)   : 0, 9597540

    section Automatic
    This PR (5563) (6.638M)   : 0, 6638260
    master (6.554M)   : 0, 6554363

    section Trace stats
    master (6.850M)   : 0, 6849767

    section Manual
    This PR (5563) (8.145M)   : 0, 8144678
    master (8.245M)   : 0, 8245403

    section Manual + Automatic
    This PR (5563) (6.066M)   : 0, 6065729
    master (6.190M)   : 0, 6190348

    section Version Conflict
    master (5.680M)   : 0, 5679678

gantt
    title Throughput Windows x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (5563) (9.856M)   : 0, 9855602
    master (9.781M)   : 0, 9781004
    benchmarks/2.9.0 (10.041M)   : 0, 10040980

    section Automatic
    This PR (5563) (7.114M)   : 0, 7113852
    master (6.964M)   : 0, 6964106
    benchmarks/2.9.0 (7.137M)   : 0, 7137430

    section Trace stats
    master (7.347M)   : 0, 7347028

    section Manual
    This PR (5563) (8.799M)   : 0, 8798889
    master (8.809M)   : 0, 8809283

    section Manual + Automatic
    This PR (5563) (6.826M)   : 0, 6826187
    master (6.754M)   : 0, 6754006

    section Version Conflict
    master (6.038M)   : 0, 6038242

If a lot of snapshots have changed, we should check them all

ff67778

andrewlock added the area:builds project files, build scripts, pipelines, versioning, releases, packages label May 15, 2024

andrewlock requested a review from a team as a code owner May 15, 2024 08:21

kevingosse approved these changes May 15, 2024

View reviewed changes

andrewlock merged commit 76a7fc9 into master May 16, 2024
55 of 57 checks passed

andrewlock deleted the andrew/ci/run-all-tfms-in-more-scenarios branch May 16, 2024 07:29

github-actions bot added this to the vNext-v2 milestone May 16, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

If a lot of snapshots have changed, we should check them all #5563

If a lot of snapshots have changed, we should check them all #5563

andrewlock commented May 15, 2024

kevingosse left a comment

datadog-ddstaging bot commented May 15, 2024 •

edited

Loading

andrewlock commented May 15, 2024

andrewlock commented May 15, 2024

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Faster 🎉 in #5563

Raw results

Raw results

Raw results

Raw results

Raw results

Faster 🎉 in #5563

Raw results

Raw results

andrewlock commented May 15, 2024

If a lot of snapshots have changed, we should check them all #5563

If a lot of snapshots have changed, we should check them all #5563

Conversation

andrewlock commented May 15, 2024

Summary of changes

Reason for change

Implementation details

Test coverage

Other details

kevingosse left a comment

Choose a reason for hiding this comment

datadog-ddstaging bot commented May 15, 2024 • edited Loading

Datadog Report

New Flaky Tests (1)

andrewlock commented May 15, 2024

Execution-Time Benchmarks Report ⏱️

andrewlock commented May 15, 2024

Benchmarks Report for tracer 🐌

Benchmark details

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Faster 🎉 in #5563

Raw results

Raw results

Raw results

Raw results

Raw results

Faster 🎉 in #5563

Raw results

Raw results

andrewlock commented May 15, 2024

Throughput/Crank Report:zap:

datadog-ddstaging bot commented May 15, 2024 •

edited

Loading