Ensure we also clean hidden folders #5399

andrewlock · 2024-04-04T11:07:51Z

Summary of changes

Update the "clean" step to delete hidden e.g. .nuke files and folders

Reason for change

rm -rf * doesn't delete "hidden" folders/files like .nuke or .github so the clean steps weren't working, and meant that subsequent checkout steps were failing

Implementation details

Delete the whole directory and recreate it. I looked at other options, but this was just the easiest and the easiest to understand

Test coverage

Tested against the branch manually

Other details

Third time lucky...

datadog-ddstaging · 2024-04-04T11:30:49Z

Datadog Report

Branch report: andrew/ci/try-to-understand-failing-smoke-tests
Commit report: 55ce50b
Test service: dd-trace-dotnet

✅ 0 Failed, 332933 Passed, 1614 Skipped, 43m 41.14s Wall Time

andrewlock · 2024-04-04T11:32:37Z

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

Welch test with statistical test for significance of 5%
Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5399) - mean (74ms)  : 66, 83
     .   : milestone, 74,
    master - mean (73ms)  : 66, 81
     .   : milestone, 73,

    section CallTarget+Inlining+NGEN
    This PR (5399) - mean (997ms)  : 971, 1024
     .   : milestone, 997,
    master - mean (994ms)  : 972, 1016
     .   : milestone, 994,

gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5399) - mean (109ms)  : 106, 112
     .   : milestone, 109,
    master - mean (110ms)  : 107, 112
     .   : milestone, 110,

    section CallTarget+Inlining+NGEN
    This PR (5399) - mean (717ms)  : 694, 740
     .   : milestone, 717,
    master - mean (713ms)  : 689, 737
     .   : milestone, 713,

gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5399) - mean (94ms)  : 91, 97
     .   : milestone, 94,
    master - mean (95ms)  : 91, 98
     .   : milestone, 95,

    section CallTarget+Inlining+NGEN
    This PR (5399) - mean (669ms)  : 649, 689
     .   : milestone, 669,
    master - mean (681ms)  : 656, 705
     .   : milestone, 681,

gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5399) - mean (188ms)  : 186, 191
     .   : milestone, 188,
    master - mean (188ms)  : 185, 192
     .   : milestone, 188,

    section CallTarget+Inlining+NGEN
    This PR (5399) - mean (1,082ms)  : 1062, 1102
     .   : milestone, 1082,
    master - mean (1,076ms)  : 1044, 1107
     .   : milestone, 1076,

gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5399) - mean (272ms)  : 268, 277
     .   : milestone, 272,
    master - mean (271ms)  : 265, 277
     .   : milestone, 271,

    section CallTarget+Inlining+NGEN
    This PR (5399) - mean (873ms)  : 846, 901
     .   : milestone, 873,
    master - mean (873ms)  : 849, 898
     .   : milestone, 873,

gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5399) - mean (261ms)  : 258, 264
     .   : milestone, 261,
    master - mean (260ms)  : 252, 268
     .   : milestone, 260,

    section CallTarget+Inlining+NGEN
    This PR (5399) - mean (860ms)  : 837, 884
     .   : milestone, 860,
    master - mean (854ms)  : 828, 880
     .   : milestone, 854,

andrewlock · 2024-04-04T13:25:06Z

Benchmarks Report for tracer 🐌

Benchmarks for #5399 compared to master:

2 benchmarks are faster, with geometric mean 1.121
All benchmarks have the same allocations

The following thresholds were used for comparing the benchmark speeds:

Mann–Whitney U test with statistical test for significance of 5%
Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	`StartStopWithChild`	net6.0	8.59μs	48.1ns	330ns	0.0173	0.00863	0	7.51 KB
master	`StartStopWithChild`	netcoreapp3.1	10.7μs	55.6ns	278ns	0.0324	0.0162	0	7.6 KB
master	`StartStopWithChild`	net472	17μs	80.2ns	310ns	1.36	0.34	0.119	8.04 KB
#5399	`StartStopWithChild`	net6.0	8.9μs	45.4ns	257ns	0.0236	0.00942	0	7.51 KB
#5399	`StartStopWithChild`	netcoreapp3.1	10.7μs	59.5ns	362ns	0.0259	0.0103	0	7.6 KB
#5399	`StartStopWithChild`	net472	16.8μs	41ns	148ns	1.33	0.312	0.101	8.05 KB

Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`WriteAndFlushEnrichedTraces`	net6.0	453μs	218ns	845ns	0	2.7 KB
master	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	589μs	119ns	444ns	0	2.7 KB
master	`WriteAndFlushEnrichedTraces`	net472	815μs	142ns	530ns	0.406	3.3 KB
#5399	`WriteAndFlushEnrichedTraces`	net6.0	474μs	298ns	1.12μs	0	2.7 KB
#5399	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	608μs	209ns	752ns	0	2.7 KB
#5399	`WriteAndFlushEnrichedTraces`	net472	820μs	303ns	1.18μs	0.408	3.3 KB

Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendRequest`	net6.0	169μs	150ns	560ns	0.17	18.33 KB
master	`SendRequest`	netcoreapp3.1	189μs	342ns	1.33μs	0.188	20.49 KB
master	`SendRequest`	net472	0.000145ns	7.84E‑05ns	0.000293ns	0	0 b
#5399	`SendRequest`	net6.0	168μs	153ns	572ns	0.253	18.33 KB
#5399	`SendRequest`	netcoreapp3.1	193μs	343ns	1.33μs	0.193	20.49 KB
#5399	`SendRequest`	net472	0.00226ns	0.000581ns	0.00225ns	0	0 b

Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	`WriteAndFlushEnrichedTraces`	net6.0	554μs	855ns	3.31μs	0.539	0	0	41.72 KB
master	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	632μs	563ns	1.95μs	0.327	0	0	41.69 KB
master	`WriteAndFlushEnrichedTraces`	net472	904μs	3.54μs	13.7μs	8.42	2.66	0.443	53.35 KB
#5399	`WriteAndFlushEnrichedTraces`	net6.0	550μs	274ns	988ns	0.558	0	0	41.79 KB
#5399	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	667μs	986ns	3.82μs	0.332	0	0	41.79 KB
#5399	`WriteAndFlushEnrichedTraces`	net472	847μs	3.74μs	14.5μs	8.08	2.55	0.425	53.23 KB

Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`ExecuteNonQuery`	net6.0	1.17μs	0.218ns	0.816ns	0.0112	784 B
master	`ExecuteNonQuery`	netcoreapp3.1	1.47μs	0.604ns	2.34ns	0.0105	784 B
master	`ExecuteNonQuery`	net472	1.92μs	1.28ns	4.95ns	0.118	746 B
#5399	`ExecuteNonQuery`	net6.0	1.25μs	0.685ns	2.65ns	0.0106	784 B
#5399	`ExecuteNonQuery`	netcoreapp3.1	1.51μs	1.56ns	6.03ns	0.0106	784 B
#5399	`ExecuteNonQuery`	net472	1.89μs	0.776ns	3.01ns	0.118	746 B

Benchmarks.Trace.ElasticsearchBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`CallElasticsearch`	net6.0	1.35μs	0.358ns	1.34ns	0.0142	0	1.01 KB
master	`CallElasticsearch`	netcoreapp3.1	1.61μs	1.16ns	4.19ns	0.0137	0	1.01 KB
master	`CallElasticsearch`	net472	2.57μs	1.48ns	5.74ns	0.162	0	1.02 KB
master	`CallElasticsearchAsync`	net6.0	1.38μs	0.851ns	3.3ns	0.0138	0	984 B
master	`CallElasticsearchAsync`	netcoreapp3.1	1.74μs	0.655ns	2.54ns	0.014	0	1.06 KB
master	`CallElasticsearchAsync`	net472	2.69μs	2.79ns	10.8ns	0.171	0.00134	1.08 KB
#5399	`CallElasticsearch`	net6.0	1.28μs	0.59ns	2.21ns	0.0141	0	1.01 KB
#5399	`CallElasticsearch`	netcoreapp3.1	1.61μs	0.549ns	2.05ns	0.0137	0	1.01 KB
#5399	`CallElasticsearch`	net472	2.57μs	1.33ns	5.17ns	0.161	0	1.02 KB
#5399	`CallElasticsearchAsync`	net6.0	1.4μs	0.551ns	2.13ns	0.0139	0	984 B
#5399	`CallElasticsearchAsync`	netcoreapp3.1	1.68μs	0.958ns	3.71ns	0.0143	0	1.06 KB
#5399	`CallElasticsearchAsync`	net472	2.68μs	1.15ns	4.13ns	0.171	0	1.08 KB

Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`ExecuteAsync`	net6.0	1.41μs	0.881ns	3.41ns	0.0127	0	928 B
master	`ExecuteAsync`	netcoreapp3.1	1.69μs	1.47ns	5.68ns	0.012	0	928 B
master	`ExecuteAsync`	net472	1.9μs	1.5ns	5.41ns	0.141	0.000949	891 B
#5399	`ExecuteAsync`	net6.0	1.31μs	0.646ns	2.42ns	0.0131	0	928 B
#5399	`ExecuteAsync`	netcoreapp3.1	1.67μs	2.19ns	8.5ns	0.0124	0	928 B
#5399	`ExecuteAsync`	net472	1.93μs	1.73ns	6.23ns	0.141	0	891 B

Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendAsync`	net6.0	4.13μs	3.15ns	11.8ns	0.0288	2.16 KB
master	`SendAsync`	netcoreapp3.1	5.06μs	2.89ns	10.4ns	0.0353	2.7 KB
master	`SendAsync`	net472	7.63μs	2.32ns	8.36ns	0.484	3.05 KB
#5399	`SendAsync`	net6.0	4.12μs	1.06ns	3.98ns	0.0307	2.16 KB
#5399	`SendAsync`	netcoreapp3.1	5.03μs	13.2ns	51.3ns	0.0369	2.7 KB
#5399	`SendAsync`	net472	7.69μs	3.15ns	11.8ns	0.481	3.05 KB

Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	1.47μs	2.71ns	9.76ns	0.0228	1.65 KB
master	`EnrichedLog`	netcoreapp3.1	2.17μs	0.625ns	2.34ns	0.0228	1.65 KB
master	`EnrichedLog`	net472	2.67μs	1.62ns	6.27ns	0.25	1.57 KB
#5399	`EnrichedLog`	net6.0	1.54μs	0.308ns	1.15ns	0.0231	1.65 KB
#5399	`EnrichedLog`	netcoreapp3.1	2.27μs	1.33ns	4.97ns	0.0228	1.65 KB
#5399	`EnrichedLog`	net472	2.66μs	2.59ns	10ns	0.25	1.57 KB

Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`EnrichedLog`	net6.0	110μs	74.3ns	288ns	0.0553	0	4.23 KB
master	`EnrichedLog`	netcoreapp3.1	116μs	109ns	420ns	0	0	4.23 KB
master	`EnrichedLog`	net472	146μs	62ns	240ns	0.657	0.219	4.41 KB
#5399	`EnrichedLog`	net6.0	111μs	87.2ns	326ns	0.0557	0	4.23 KB
#5399	`EnrichedLog`	netcoreapp3.1	117μs	188ns	728ns	0	0	4.23 KB
#5399	`EnrichedLog`	net472	147μs	146ns	564ns	0.661	0.22	4.41 KB

Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	3μs	1.44ns	5.18ns	0.0315	2.21 KB
master	`EnrichedLog`	netcoreapp3.1	4.3μs	2.26ns	8.73ns	0.0301	2.21 KB
master	`EnrichedLog`	net472	4.9μs	1.46ns	5.67ns	0.32	2.02 KB
#5399	`EnrichedLog`	net6.0	3.06μs	0.636ns	2.38ns	0.0306	2.21 KB
#5399	`EnrichedLog`	netcoreapp3.1	4.15μs	1.98ns	7.41ns	0.0291	2.21 KB
#5399	`EnrichedLog`	net472	5.11μs	1.98ns	7.66ns	0.32	2.02 KB

Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendReceive`	net6.0	1.49μs	1.03ns	3.71ns	0.0163	1.18 KB
master	`SendReceive`	netcoreapp3.1	1.83μs	1.19ns	4.61ns	0.0155	1.18 KB
master	`SendReceive`	net472	2.12μs	0.93ns	3.48ns	0.187	1.18 KB
#5399	`SendReceive`	net6.0	1.44μs	1.1ns	4.28ns	0.0165	1.18 KB
#5399	`SendReceive`	netcoreapp3.1	1.82μs	1.7ns	6.12ns	0.0161	1.18 KB
#5399	`SendReceive`	net472	2.26μs	1.33ns	4.8ns	0.187	1.18 KB

Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	2.77μs	0.695ns	2.6ns	0.0207	1.55 KB
master	`EnrichedLog`	netcoreapp3.1	4.01μs	2.92ns	10.9ns	0.0201	1.6 KB
master	`EnrichedLog`	net472	4.45μs	2.84ns	11ns	0.316	1.99 KB
#5399	`EnrichedLog`	net6.0	2.66μs	0.901ns	3.37ns	0.0213	1.55 KB
#5399	`EnrichedLog`	netcoreapp3.1	3.98μs	1.72ns	6.67ns	0.022	1.6 KB
#5399	`EnrichedLog`	net472	4.54μs	3.25ns	12.6ns	0.315	1.99 KB

Benchmarks.Trace.SpanBenchmark - Faster 🎉 Same allocations ✔️

Faster 🎉 in #5399

Benchmark	base/diff	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.SpanBenchmark.StartFinishSpan‑net6.0	1.122	523.84	467.04

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`StartFinishSpan`	net6.0	524ns	0.278ns	1.04ns	0.00775	552 B
master	`StartFinishSpan`	netcoreapp3.1	692ns	1.59ns	6.15ns	0.00736	552 B
master	`StartFinishSpan`	net472	740ns	1.2ns	4.65ns	0.0877	554 B
master	`StartFinishScope`	net6.0	575ns	1.29ns	4.84ns	0.00954	672 B
master	`StartFinishScope`	netcoreapp3.1	824ns	1.4ns	5.43ns	0.00893	672 B
master	`StartFinishScope`	net472	929ns	1.94ns	7.51ns	0.101	634 B
#5399	`StartFinishSpan`	net6.0	467ns	0.245ns	0.948ns	0.00771	552 B
#5399	`StartFinishSpan`	netcoreapp3.1	747ns	0.381ns	1.43ns	0.0075	552 B
#5399	`StartFinishSpan`	net472	806ns	0.701ns	2.72ns	0.0879	554 B
#5399	`StartFinishScope`	net6.0	545ns	0.238ns	0.889ns	0.00957	672 B
#5399	`StartFinishScope`	netcoreapp3.1	828ns	0.456ns	1.71ns	0.00895	672 B
#5399	`StartFinishScope`	net472	956ns	1.53ns	5.94ns	0.1	634 B

Benchmarks.Trace.TraceAnnotationsBenchmark - Faster 🎉 Same allocations ✔️

Faster 🎉 in #5399

Benchmark	base/diff	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.TraceAnnotationsBenchmark.RunOnMethodBegin‑netcoreapp3.1	1.119	1,072.73	958.28

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`RunOnMethodBegin`	net6.0	691ns	1.01ns	3.92ns	0.00933	672 B
master	`RunOnMethodBegin`	netcoreapp3.1	1.07μs	2.59ns	10ns	0.00904	672 B
master	`RunOnMethodBegin`	net472	1.13μs	2.24ns	8.68ns	0.1	634 B
#5399	`RunOnMethodBegin`	net6.0	721ns	0.954ns	3.7ns	0.00933	672 B
#5399	`RunOnMethodBegin`	netcoreapp3.1	956ns	1.45ns	5.62ns	0.00914	672 B
#5399	`RunOnMethodBegin`	net472	1.12μs	2.11ns	8.15ns	0.1	634 B

andrewlock added the area:builds project files, build scripts, pipelines, versioning, releases, packages label Apr 4, 2024

andrewlock requested a review from a team as a code owner April 4, 2024 11:07

bouwkast approved these changes Apr 4, 2024

View reviewed changes

kevingosse approved these changes Apr 4, 2024

View reviewed changes

andrewlock added 2 commits April 4, 2024 14:19

Ensure we also clean hidden folders

817690f

Use a different approach that doesn't clean the .git folder

ab19dfc

OK, a _bit_ simpler maybe?

55ce50b

andrewlock force-pushed the andrew/ci/try-to-understand-failing-smoke-tests branch from f03c086 to 55ce50b Compare April 4, 2024 13:27

andrewlock merged commit 1131072 into master Apr 4, 2024
54 of 56 checks passed

andrewlock deleted the andrew/ci/try-to-understand-failing-smoke-tests branch April 4, 2024 15:47

github-actions bot added this to the vNext-v2 milestone Apr 4, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Ensure we also clean hidden folders #5399

Ensure we also clean hidden folders #5399

andrewlock commented Apr 4, 2024

datadog-ddstaging bot commented Apr 4, 2024 •

edited

Loading

andrewlock commented Apr 4, 2024 •

edited

Loading

andrewlock commented Apr 4, 2024 •

edited

Loading

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Faster 🎉 in #5399

Raw results

Faster 🎉 in #5399

Raw results

Ensure we also clean hidden folders #5399

Ensure we also clean hidden folders #5399

Conversation

andrewlock commented Apr 4, 2024

Summary of changes

Reason for change

Implementation details

Test coverage

Other details

datadog-ddstaging bot commented Apr 4, 2024 • edited Loading

Datadog Report

andrewlock commented Apr 4, 2024 • edited Loading

Execution-Time Benchmarks Report ⏱️

andrewlock commented Apr 4, 2024 • edited Loading

Benchmarks Report for tracer 🐌

Benchmark details

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Faster 🎉 in #5399

Raw results

Faster 🎉 in #5399

Raw results

datadog-ddstaging bot commented Apr 4, 2024 •

edited

Loading

andrewlock commented Apr 4, 2024 •

edited

Loading

andrewlock commented Apr 4, 2024 •

edited

Loading