Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor the exclude list for assembly instrumentation #3137

Merged
merged 2 commits into from
Aug 31, 2022

Conversation

andrewlock
Copy link
Member

Summary of changes

Refactor the "exclude by pattern" list to allow specific inclusions.

Reason for change

In #1663 we switched from excluding the Microsoft.Extensions prefix to excluding each of the non-logging assemblies individually.

This is not future proof (as new Microsoft.Extensions libraries could (will) be added, that we don't want to instrument), and increases the number of assemblies we need to match against.

Implementation details

As a workaround, adds an explicit "include" list for assemblies which match the "exclude" prefix, but which should be included anyway.

I used a goto. I understand if I'm not allowed to touch C++ again.

Test coverage

Should be covered by existing

Other details

Should make it easier for @NachoEchevarria's work on System.Diagnostics.Process.

@andrewlock andrewlock added type:refactor area:native-library Automatic instrumentation native C++ code (Datadog.Trace.ClrProfiler.Native) labels Aug 30, 2022
@andrewlock andrewlock requested a review from a team as a code owner August 30, 2022 10:11
{
Logger::Debug("ModuleLoadFinished matched module by pattern: ", module_id, " ", module_info.assembly.name,
"but assembly is explicitly included");
goto inject;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

😲 😲 😲 😲 😲

Copy link
Member

@tonyredondo tonyredondo Aug 30, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can leave it like this or or or use a break; to prevent my eyes from bleeding. 😂

Copy link
Member Author

@andrewlock andrewlock Aug 30, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't use a break because we're two for-loops deep at this point 😛 We want to break out of both loops (because we've already matched the pattern, so don't want to check the others)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's ways of removing the break if it's too offensive 😉

We could check if the assembly is in the include_assembly list first. Only check the skip_assembly_prefixes if it's not in that list.

Given that most assemblies won't be on the list, the approach approach I used should have less overhead I think, by avoiding the extra loop for every assembly? It's a tiny difference though 😉 Happy to change it if you think that's preferable! 🙂

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤦🏻 true.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

or add the typical bool "found" and if (found) {break} once again... 🙄🙈

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

or add the typical bool "found" and if (found) {break} once again...

Yeah, I think this is preferable. Too much of a coincidence that I add a goto and all our builds suddenly fail 🙈 . They say it's DNS, but I dunno, seems sus 🤫

@andrewlock

This comment has been minimized.

@andrewlock andrewlock force-pushed the andrew/simplify-assembly-skip-check branch from 692d69b to 3c3c6b4 Compare August 30, 2022 18:43
@andrewlock

This comment has been minimized.

@andrewlock

This comment has been minimized.

In #1663 we switched from excluding the Microsoft.Extensions prefix to excluding each of the non-logging assemblies individually.

This is not future proof (as new libraries could be added), and increases the number of assemblies we need to match against.

As a workaround, list adds an explicit "include" list for assemblies which match the "exclude" prefix, but which should be included anyway.
@andrewlock andrewlock force-pushed the andrew/simplify-assembly-skip-check branch from 3c3c6b4 to b5c00bd Compare August 31, 2022 08:52
@andrewlock
Copy link
Member Author

Benchmarks Report 🐌

Benchmarks for #3137 compared to master:

  • All benchmarks have the same speed
  • All benchmarks have the same allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net472 712μs 241ns 932ns 0.374 0 0 3.18 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 451μs 177ns 662ns 0 0 0 2.58 KB
#3137 WriteAndFlushEnrichedTraces net472 721μs 659ns 2.55μs 0.355 0 0 3.18 KB
#3137 WriteAndFlushEnrichedTraces netcoreapp3.1 473μs 109ns 394ns 0 0 0 2.58 KB
Benchmarks.Trace.AppSecBodyBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net472 198ns 0.881ns 3.41ns 0.0675 0 0 425 B
master AllCycleSimpleBody netcoreapp3.1 248ns 0.762ns 2.95ns 0.0058 0 0 424 B
master AllCycleMoreComplexBody net472 193ns 0.788ns 3.05ns 0.0637 0 0 401 B
master AllCycleMoreComplexBody netcoreapp3.1 241ns 0.848ns 3.29ns 0.00539 0 0 400 B
master BodyExtractorSimpleBody net472 274ns 1.26ns 4.89ns 0.0574 0 0 361 B
master BodyExtractorSimpleBody netcoreapp3.1 242ns 0.776ns 3.01ns 0.00366 0 0 272 B
master BodyExtractorMoreComplexBody net472 14.7μs 53.5ns 207ns 1.2 0.0219 0 7.62 KB
master BodyExtractorMoreComplexBody netcoreapp3.1 12.6μs 55.6ns 215ns 0.0923 0 0 6.75 KB
#3137 AllCycleSimpleBody net472 188ns 0.615ns 2.38ns 0.0676 0 0 425 B
#3137 AllCycleSimpleBody netcoreapp3.1 240ns 0.75ns 2.9ns 0.00588 0 0 424 B
#3137 AllCycleMoreComplexBody net472 187ns 0.829ns 3.52ns 0.0638 0 0 401 B
#3137 AllCycleMoreComplexBody netcoreapp3.1 244ns 0.611ns 2.37ns 0.00542 0 0 400 B
#3137 BodyExtractorSimpleBody net472 270ns 0.903ns 3.38ns 0.0574 0 0 361 B
#3137 BodyExtractorSimpleBody netcoreapp3.1 235ns 0.633ns 2.45ns 0.00369 0 0 272 B
#3137 BodyExtractorMoreComplexBody net472 15μs 57.8ns 224ns 1.2 0.0154 0 7.62 KB
#3137 BodyExtractorMoreComplexBody netcoreapp3.1 12.6μs 42.4ns 159ns 0.0879 0 0 6.75 KB
Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
master SendRequest netcoreapp3.1 181μs 182ns 705ns 0.271 0 0 20.57 KB
#3137 SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
#3137 SendRequest netcoreapp3.1 182μs 60.3ns 233ns 0.272 0 0 20.57 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net472 1.83μs 0.401ns 1.5ns 0.15 0.000916 0 947 B
master ExecuteNonQuery netcoreapp3.1 1.45μs 0.768ns 2.98ns 0.0123 0 0 936 B
#3137 ExecuteNonQuery net472 1.91μs 0.74ns 2.77ns 0.15 0.000953 0 947 B
#3137 ExecuteNonQuery netcoreapp3.1 1.46μs 0.767ns 2.76ns 0.0123 0 0 936 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net472 2.49μs 1.05ns 3.94ns 0.184 0 0 1.16 KB
master CallElasticsearch netcoreapp3.1 1.49μs 0.428ns 1.6ns 0.0149 0 0 1.1 KB
master CallElasticsearchAsync net472 2.67μs 0.832ns 3.22ns 0.205 0 0 1.29 KB
master CallElasticsearchAsync netcoreapp3.1 1.62μs 0.442ns 1.65ns 0.0162 0 0 1.22 KB
#3137 CallElasticsearch net472 2.55μs 0.379ns 1.36ns 0.183 0 0 1.16 KB
#3137 CallElasticsearch netcoreapp3.1 1.6μs 0.443ns 1.66ns 0.0144 0 0 1.1 KB
#3137 CallElasticsearchAsync net472 2.64μs 1.06ns 4.12ns 0.204 0 0 1.29 KB
#3137 CallElasticsearchAsync netcoreapp3.1 1.65μs 0.508ns 1.83ns 0.0165 0 0 1.22 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net472 2.65μs 4.98ns 19.3ns 0.224 0 0 1.41 KB
master ExecuteAsync netcoreapp3.1 1.62μs 2.5ns 9.67ns 0.0181 0 0 1.34 KB
#3137 ExecuteAsync net472 2.62μs 5.82ns 21.8ns 0.224 0 0 1.41 KB
#3137 ExecuteAsync netcoreapp3.1 1.75μs 2.43ns 8.75ns 0.0175 0 0 1.34 KB
Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net472 5.66μs 8.67ns 32.4ns 0.439 0 0 2.77 KB
master SendAsync netcoreapp3.1 3.57μs 7.69ns 29.8ns 0.0348 0 0 2.6 KB
#3137 SendAsync net472 5.86μs 11.3ns 43.7ns 0.439 0 0 2.77 KB
#3137 SendAsync netcoreapp3.1 3.59μs 6.22ns 24.1ns 0.0361 0 0 2.6 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 3.21μs 2.06ns 7.72ns 0.287 0 0 1.81 KB
master EnrichedLog netcoreapp3.1 2.46μs 1.34ns 4.83ns 0.0245 0 0 1.85 KB
#3137 EnrichedLog net472 3.17μs 1.91ns 7.16ns 0.287 0 0 1.81 KB
#3137 EnrichedLog netcoreapp3.1 2.6μs 1.16ns 4.34ns 0.0247 0 0 1.85 KB
Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 149μs 91.4ns 354ns 0.671 0.224 0 4.65 KB
master EnrichedLog netcoreapp3.1 115μs 124ns 481ns 0 0 0 4.49 KB
#3137 EnrichedLog net472 151μs 85.5ns 331ns 0.67 0.223 0 4.65 KB
#3137 EnrichedLog netcoreapp3.1 117μs 98.3ns 368ns 0.0581 0 0 4.49 KB
Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 5.6μs 12.4ns 47.9ns 0.569 0.00279 0 3.59 KB
master EnrichedLog netcoreapp3.1 4.37μs 12.8ns 49.6ns 0.0541 0 0 3.91 KB
#3137 EnrichedLog net472 5.77μs 5.4ns 20.9ns 0.57 0.00289 0 3.59 KB
#3137 EnrichedLog netcoreapp3.1 4.27μs 12.5ns 48.6ns 0.053 0 0 3.91 KB
Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net472 2.31μs 1.13ns 4.21ns 0.218 0 0 1.37 KB
master SendReceive netcoreapp3.1 1.82μs 0.519ns 1.94ns 0.0182 0 0 1.32 KB
#3137 SendReceive net472 2.25μs 0.936ns 3.5ns 0.217 0 0 1.37 KB
#3137 SendReceive netcoreapp3.1 1.84μs 0.423ns 1.64ns 0.0175 0 0 1.32 KB
Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 5.04μs 1.76ns 6.83ns 0.353 0 0 2.23 KB
master EnrichedLog netcoreapp3.1 4.28μs 1.83ns 6.83ns 0.0234 0 0 1.8 KB
#3137 EnrichedLog net472 5.04μs 1.6ns 6.19ns 0.353 0 0 2.23 KB
#3137 EnrichedLog netcoreapp3.1 4.4μs 2.69ns 10.1ns 0.0242 0 0 1.8 KB
Benchmarks.Trace.SpanBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net472 1.11μs 0.332ns 1.28ns 0.128 0 0 810 B
master StartFinishSpan netcoreapp3.1 893ns 0.371ns 1.39ns 0.0103 0 0 760 B
master StartFinishScope net472 1.35μs 0.91ns 3.4ns 0.141 0 0 891 B
master StartFinishScope netcoreapp3.1 1.06μs 1.13ns 4.39ns 0.0121 0 0 880 B
#3137 StartFinishSpan net472 1.22μs 0.398ns 1.49ns 0.129 0 0 810 B
#3137 StartFinishSpan netcoreapp3.1 990ns 0.525ns 1.96ns 0.0103 0 0 760 B
#3137 StartFinishScope net472 1.36μs 0.294ns 1.1ns 0.141 0 0 891 B
#3137 StartFinishScope netcoreapp3.1 1.11μs 0.438ns 1.7ns 0.0122 0 0 880 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net472 1.48μs 0.658ns 2.55ns 0.141 0 0 891 B
master RunOnMethodBegin netcoreapp3.1 1.17μs 0.726ns 2.72ns 0.0118 0 0 880 B
#3137 RunOnMethodBegin net472 1.59μs 0.434ns 1.62ns 0.141 0 0 891 B
#3137 RunOnMethodBegin netcoreapp3.1 1.12μs 0.668ns 2.5ns 0.0118 0 0 880 B

@andrewlock
Copy link
Member Author

Code Coverage Report 📊

✔️ Merging #3137 into master will not change line coverage
✔️ Merging #3137 into master will not change branch coverage
⛔ Merging #3137 into master will will increase complexity by 4

master #3137 Change
Lines 17297 / 23587 17283 / 23585
Lines % 73% 73% 0% ✔️
Branches 10265 / 14548 10258 / 14548
Branches % 71% 71% 0% ✔️
Complexity 15666 15670 4

View the full report for further details:

Datadog.Trace Breakdown ✔️

master #3137 Change
Lines % 73% 73% 0% ✔️
Branches % 71% 71% 0% ✔️
Complexity 15666 15670 4

The following classes have significant coverage changes.

File Line coverage change Branch coverage change Complexity change
Datadog.Trace.Ci.CIVisibility 5% ✔️ 6% ✔️ 0 ✔️
Datadog.Trace.Agent.DiscoveryService.DiscoveryService 8% ✔️ 0% ✔️ 0 ✔️

The following classes were added in #3137:

File Line coverage Branch coverage Complexity
Datadog.Trace.Util.ThreadSafeRandom 85% 100% 9

View the full reports for further details:

@andrewlock andrewlock merged commit dc883cb into master Aug 31, 2022
@andrewlock andrewlock deleted the andrew/simplify-assembly-skip-check branch August 31, 2022 10:57
@github-actions github-actions bot added this to the vNext milestone Aug 31, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:native-library Automatic instrumentation native C++ code (Datadog.Trace.ClrProfiler.Native) type:refactor
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants