Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Profiler] Upgrade cppcheck to 2.12 #5398

Merged
merged 2 commits into from
Apr 5, 2024
Merged

Conversation

gleocadie
Copy link
Collaborator

@gleocadie gleocadie commented Apr 4, 2024

Summary of changes

Bump CppCheck + add suppressions

Reason for change

The job has 100% error rate.

Implementation details

  • Update CppCheck to 2.12
  • Add suppressions to i) ignore errors due to CppCheck inability to parse, ii) ignore not found standard include headers (less errors)

Test coverage

  • The profiler job must not fail now

Other details

@gleocadie gleocadie requested a review from a team as a code owner April 4, 2024 10:48
@github-actions github-actions bot added area:builds project files, build scripts, pipelines, versioning, releases, packages area:profiler Issues related to the continous-profiler labels Apr 4, 2024
@datadog-ddstaging
Copy link

datadog-ddstaging bot commented Apr 4, 2024

Datadog Report

Branch report: gleocadie/fix-cppcheck-windows
Commit report: f62186f
Test service: dd-trace-dotnet

❌ 1 Failed (1 Known Flaky), 332025 Passed, 1880 Skipped, 49m 16.9s Wall Time

❌ Failed Tests (1)

  • CheckErrorWhenNoAgentIsAvailable - Datadog.Profiler.IntegrationTests.WindowsOnly.EtwEventsTests - ❄️ Known flaky - Details

    Expand for error
     Exit code of "Samples.Computer01.exe" should be 0 instead of -1073741819 (= 0xC0000005)
     Expected: True
     Actual:   False
    

@gleocadie gleocadie force-pushed the gleocadie/fix-cppcheck-windows branch from 075fc23 to 5367383 Compare April 4, 2024 11:47
@gleocadie gleocadie requested a review from a team as a code owner April 4, 2024 11:47
@gleocadie gleocadie force-pushed the gleocadie/fix-cppcheck-windows branch from 5367383 to 3b864c4 Compare April 4, 2024 11:48
@@ -15,6 +15,7 @@ namespace pmr {
using namespace std::experimental::pmr;
}
#else // Not found at all
// cppcheck-suppress preprocessorErrorDirective
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since CppCheck cannot find <memory_resource> header file, it defaults to #error.
Here we suppress the error because it's not one :)

preprocessorErrorDirective:*/logging.h
cppcheckError:*/StackSamplerLoopManager.cpp
missingIncludeSystem:*
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove missing standard include headers (too noisy). CppCheck folks does not recommend otherwise we might have events from those files (which is not what we want)

Copy link
Contributor

@chrisnas chrisnas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@andrewlock
Copy link
Member

andrewlock commented Apr 4, 2024

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

  • Welch test with statistical test for significance of 5%
  • Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5398) - mean (74ms)  : 67, 81
     .   : milestone, 74,
    master - mean (74ms)  : 65, 84
     .   : milestone, 74,

    section CallTarget+Inlining+NGEN
    This PR (5398) - mean (997ms)  : 982, 1012
     .   : milestone, 997,
    master - mean (998ms)  : 976, 1020
     .   : milestone, 998,

Loading
gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5398) - mean (111ms)  : 107, 114
     .   : milestone, 111,
    master - mean (111ms)  : 106, 115
     .   : milestone, 111,

    section CallTarget+Inlining+NGEN
    This PR (5398) - mean (716ms)  : 698, 735
     .   : milestone, 716,
    master - mean (715ms)  : 691, 738
     .   : milestone, 715,

Loading
gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5398) - mean (95ms)  : 91, 98
     .   : milestone, 95,
    master - mean (94ms)  : 91, 97
     .   : milestone, 94,

    section CallTarget+Inlining+NGEN
    This PR (5398) - mean (675ms)  : 649, 700
     .   : milestone, 675,
    master - mean (674ms)  : 647, 700
     .   : milestone, 674,

Loading
gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5398) - mean (188ms)  : 185, 191
     .   : milestone, 188,
    master - mean (188ms)  : 185, 192
     .   : milestone, 188,

    section CallTarget+Inlining+NGEN
    This PR (5398) - mean (1,075ms)  : 1046, 1104
     .   : milestone, 1075,
    master - mean (1,076ms)  : 1045, 1108
     .   : milestone, 1076,

Loading
gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5398) - mean (270ms)  : 263, 277
     .   : milestone, 270,
    master - mean (270ms)  : 262, 277
     .   : milestone, 270,

    section CallTarget+Inlining+NGEN
    This PR (5398) - mean (877ms)  : 857, 897
     .   : milestone, 877,
    master - mean (863ms)  : 833, 893
     .   : milestone, 863,

Loading
gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (5398) - mean (261ms)  : 257, 265
     .   : milestone, 261,
    master - mean (260ms)  : 256, 264
     .   : milestone, 260,

    section CallTarget+Inlining+NGEN
    This PR (5398) - mean (858ms)  : 835, 881
     .   : milestone, 858,
    master - mean (845ms)  : 817, 873
     .   : milestone, 845,

Loading

@andrewlock
Copy link
Member

andrewlock commented Apr 4, 2024

Benchmarks Report for tracer 🐌

Benchmarks for #5398 compared to master:

  • 1 benchmarks are faster, with geometric mean 1.154
  • 1 benchmarks are slower, with geometric mean 1.184
  • All benchmarks have the same allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartStopWithChild net6.0 8.79μs 48.7ns 292ns 0.0242 0.0121 0 7.51 KB
master StartStopWithChild netcoreapp3.1 10.9μs 59.1ns 350ns 0.0374 0.016 0.00534 7.6 KB
master StartStopWithChild net472 16.9μs 52.7ns 204ns 1.35 0.36 0.109 8.05 KB
#5398 StartStopWithChild net6.0 9.17μs 51.2ns 328ns 0.0185 0.00925 0 7.51 KB
#5398 StartStopWithChild netcoreapp3.1 10.9μs 59.3ns 330ns 0.0212 0.00531 0 7.6 KB
#5398 StartStopWithChild net472 17μs 65.9ns 255ns 1.35 0.362 0.112 8.04 KB
Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 461μs 171ns 592ns 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 609μs 307ns 1.19μs 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces net472 819μs 184ns 712ns 0.408 0 0 3.3 KB
#5398 WriteAndFlushEnrichedTraces net6.0 445μs 149ns 576ns 0 0 0 2.7 KB
#5398 WriteAndFlushEnrichedTraces netcoreapp3.1 616μs 204ns 789ns 0 0 0 2.7 KB
#5398 WriteAndFlushEnrichedTraces net472 825μs 1.83μs 7.07μs 0.406 0 0 3.3 KB
Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net6.0 170μs 131ns 507ns 0.254 0 0 18.33 KB
master SendRequest netcoreapp3.1 192μs 335ns 1.3μs 0.192 0 0 20.49 KB
master SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
#5398 SendRequest net6.0 169μs 293ns 1.13μs 0.171 0 0 18.33 KB
#5398 SendRequest netcoreapp3.1 194μs 253ns 979ns 0.193 0 0 20.49 KB
#5398 SendRequest net472 0.00116ns 0.000315ns 0.00122ns 0 0 0 0 b
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 540μs 2.29μs 8.86μs 0.543 0 0 41.54 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 658μs 486ns 1.88μs 0.327 0 0 41.84 KB
master WriteAndFlushEnrichedTraces net472 901μs 3.82μs 14.8μs 8.36 2.64 0.44 53.25 KB
#5398 WriteAndFlushEnrichedTraces net6.0 539μs 334ns 1.16μs 0.558 0 0 41.45 KB
#5398 WriteAndFlushEnrichedTraces netcoreapp3.1 655μs 651ns 2.52μs 0.327 0 0 41.71 KB
#5398 WriteAndFlushEnrichedTraces net472 846μs 3.13μs 12.1μs 8.39 2.52 0.419 53.24 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net6.0 1.12μs 0.513ns 1.92ns 0.0107 0 0 784 B
master ExecuteNonQuery netcoreapp3.1 1.44μs 1.08ns 4.2ns 0.0108 0 0 784 B
master ExecuteNonQuery net472 1.83μs 0.773ns 2.99ns 0.118 0 0 746 B
#5398 ExecuteNonQuery net6.0 1.1μs 0.722ns 2.8ns 0.011 0 0 784 B
#5398 ExecuteNonQuery netcoreapp3.1 1.5μs 0.795ns 2.98ns 0.01 0 0 784 B
#5398 ExecuteNonQuery net472 1.83μs 0.983ns 3.68ns 0.118 0 0 746 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net6.0 1.27μs 0.592ns 2.21ns 0.014 0 0 1.01 KB
master CallElasticsearch netcoreapp3.1 1.58μs 0.966ns 3.61ns 0.0135 0 0 1.01 KB
master CallElasticsearch net472 2.49μs 1.15ns 4.47ns 0.162 0 0 1.02 KB
master CallElasticsearchAsync net6.0 1.34μs 0.78ns 3.02ns 0.0135 0 0 984 B
master CallElasticsearchAsync netcoreapp3.1 1.69μs 1.07ns 4.14ns 0.0143 0 0 1.06 KB
master CallElasticsearchAsync net472 2.7μs 2.32ns 8.97ns 0.17 0 0 1.08 KB
#5398 CallElasticsearch net6.0 1.31μs 0.779ns 3.02ns 0.0138 0 0 1.01 KB
#5398 CallElasticsearch netcoreapp3.1 1.62μs 0.881ns 3.3ns 0.014 0 0 1.01 KB
#5398 CallElasticsearch net472 2.65μs 2.45ns 9.51ns 0.162 0 0 1.02 KB
#5398 CallElasticsearchAsync net6.0 1.36μs 0.409ns 1.53ns 0.0137 0 0 984 B
#5398 CallElasticsearchAsync netcoreapp3.1 1.72μs 2.21ns 8.54ns 0.0144 0 0 1.06 KB
#5398 CallElasticsearchAsync net472 2.57μs 1.42ns 5.51ns 0.17 0 0 1.08 KB
Benchmarks.Trace.GraphQLBenchmark - Faster 🎉 Same allocations ✔️

Faster 🎉 in #5398

Benchmark base/diff Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.GraphQLBenchmark.ExecuteAsync‑net6.0 1.154 1,478.86 1,281.31

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net6.0 1.48μs 0.698ns 2.61ns 0.0132 0 0 928 B
master ExecuteAsync netcoreapp3.1 1.73μs 1.76ns 6.6ns 0.0128 0 0 928 B
master ExecuteAsync net472 2.01μs 0.79ns 2.95ns 0.141 0 0 891 B
#5398 ExecuteAsync net6.0 1.28μs 0.299ns 1.03ns 0.0129 0 0 928 B
#5398 ExecuteAsync netcoreapp3.1 1.62μs 0.649ns 2.34ns 0.0125 0 0 928 B
#5398 ExecuteAsync net472 2.03μs 1.13ns 4.36ns 0.141 0 0 891 B
Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net6.0 4.33μs 12.8ns 49.5ns 0.03 0 0 2.16 KB
master SendAsync netcoreapp3.1 5.14μs 5.08ns 19ns 0.0362 0 0 2.7 KB
master SendAsync net472 7.75μs 2.94ns 11ns 0.483 0 0 3.05 KB
#5398 SendAsync net6.0 3.99μs 2.34ns 8.77ns 0.0299 0 0 2.16 KB
#5398 SendAsync netcoreapp3.1 5.05μs 4.58ns 17.7ns 0.0379 0 0 2.7 KB
#5398 SendAsync net472 7.69μs 3.34ns 12.9ns 0.484 0 0 3.05 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 1.6μs 1.07ns 4.13ns 0.0225 0 0 1.65 KB
master EnrichedLog netcoreapp3.1 2.19μs 0.638ns 2.47ns 0.0231 0 0 1.65 KB
master EnrichedLog net472 2.68μs 0.954ns 3.44ns 0.25 0 0 1.57 KB
#5398 EnrichedLog net6.0 1.6μs 0.805ns 3.01ns 0.0231 0 0 1.65 KB
#5398 EnrichedLog netcoreapp3.1 2.27μs 0.825ns 3.09ns 0.0224 0 0 1.65 KB
#5398 EnrichedLog net472 2.63μs 2.69ns 10.1ns 0.249 0 0 1.57 KB
Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 112μs 178ns 689ns 0 0 0 4.23 KB
master EnrichedLog netcoreapp3.1 117μs 126ns 489ns 0 0 0 4.23 KB
master EnrichedLog net472 146μs 55.1ns 199ns 0.657 0.219 0 4.41 KB
#5398 EnrichedLog net6.0 113μs 220ns 854ns 0.0561 0 0 4.23 KB
#5398 EnrichedLog netcoreapp3.1 118μs 166ns 645ns 0.0594 0 0 4.23 KB
#5398 EnrichedLog net472 147μs 126ns 489ns 0.66 0.22 0 4.41 KB
Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 3.19μs 2.18ns 8.45ns 0.03 0 0 2.21 KB
master EnrichedLog netcoreapp3.1 4.28μs 2.3ns 8.9ns 0.0301 0 0 2.21 KB
master EnrichedLog net472 5.05μs 2.26ns 8.76ns 0.321 0 0 2.02 KB
#5398 EnrichedLog net6.0 3.12μs 1.88ns 7.28ns 0.0311 0 0 2.21 KB
#5398 EnrichedLog netcoreapp3.1 4.19μs 2.03ns 7.87ns 0.0291 0 0 2.21 KB
#5398 EnrichedLog net472 4.85μs 2.08ns 8.05ns 0.32 0 0 2.02 KB
Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net6.0 1.47μs 0.573ns 2.15ns 0.0161 0 0 1.18 KB
master SendReceive netcoreapp3.1 1.86μs 1.52ns 5.89ns 0.0157 0 0 1.18 KB
master SendReceive net472 2.3μs 2.27ns 8.79ns 0.187 0 0 1.18 KB
#5398 SendReceive net6.0 1.43μs 1.06ns 3.96ns 0.0164 0 0 1.18 KB
#5398 SendReceive netcoreapp3.1 1.85μs 1.35ns 5.24ns 0.0158 0 0 1.18 KB
#5398 SendReceive net472 2.31μs 1.11ns 4.28ns 0.187 0 0 1.18 KB
Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 2.76μs 1.01ns 3.77ns 0.0208 0 0 1.55 KB
master EnrichedLog netcoreapp3.1 3.86μs 1.41ns 5.26ns 0.0212 0 0 1.6 KB
master EnrichedLog net472 4.62μs 2.56ns 9.91ns 0.314 0 0 1.99 KB
#5398 EnrichedLog net6.0 2.8μs 3.35ns 12.1ns 0.0209 0 0 1.55 KB
#5398 EnrichedLog netcoreapp3.1 3.91μs 1.58ns 6.11ns 0.0213 0 0 1.6 KB
#5398 EnrichedLog net472 4.45μs 1.57ns 6.1ns 0.315 0 0 1.99 KB
Benchmarks.Trace.SpanBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net6.0 474ns 1.01ns 3.93ns 0.00778 0 0 552 B
master StartFinishSpan netcoreapp3.1 744ns 1.91ns 7.4ns 0.00752 0 0 552 B
master StartFinishSpan net472 789ns 1.07ns 3.87ns 0.0877 0 0 554 B
master StartFinishScope net6.0 601ns 1.62ns 6.27ns 0.0093 0 0 672 B
master StartFinishScope netcoreapp3.1 897ns 1.26ns 4.72ns 0.00904 0 0 672 B
master StartFinishScope net472 892ns 1.97ns 7.35ns 0.101 0 0 634 B
#5398 StartFinishSpan net6.0 518ns 2.7ns 12.7ns 0.00772 0 0 552 B
#5398 StartFinishSpan netcoreapp3.1 667ns 0.94ns 3.26ns 0.00755 0 0 552 B
#5398 StartFinishSpan net472 789ns 0.775ns 3ns 0.0877 0 0 554 B
#5398 StartFinishScope net6.0 624ns 0.465ns 1.68ns 0.00937 0 0 672 B
#5398 StartFinishScope netcoreapp3.1 849ns 3.67ns 14.2ns 0.00905 0 0 672 B
#5398 StartFinishScope net472 897ns 1.04ns 4.01ns 0.1 0 0 634 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Slower ⚠️ Same allocations ✔️

Slower ⚠️ in #5398

Benchmark diff/base Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.TraceAnnotationsBenchmark.RunOnMethodBegin‑net6.0 1.184 619.13 733.19

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net6.0 620ns 1.4ns 5.41ns 0.00939 0 0 672 B
master RunOnMethodBegin netcoreapp3.1 904ns 1.51ns 5.86ns 0.00907 0 0 672 B
master RunOnMethodBegin net472 1.07μs 2.84ns 11ns 0.1 0 0 634 B
#5398 RunOnMethodBegin net6.0 733ns 0.871ns 3.37ns 0.00954 0 0 672 B
#5398 RunOnMethodBegin netcoreapp3.1 960ns 1.95ns 7.56ns 0.00888 0 0 672 B
#5398 RunOnMethodBegin net472 1.07μs 1.42ns 5.51ns 0.1 0 0 634 B

@gleocadie gleocadie force-pushed the gleocadie/fix-cppcheck-windows branch from 3b864c4 to f62186f Compare April 5, 2024 11:34
@gleocadie gleocadie merged commit 2120452 into master Apr 5, 2024
60 of 65 checks passed
@gleocadie gleocadie deleted the gleocadie/fix-cppcheck-windows branch April 5, 2024 13:00
@github-actions github-actions bot added this to the vNext-v2 milestone Apr 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:builds project files, build scripts, pipelines, versioning, releases, packages area:profiler Issues related to the continous-profiler
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants