Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix tracing for tests executed with JUnit Platform Runner #6818

Merged
merged 2 commits into from
Mar 18, 2024
Merged

Fix tracing for tests executed with JUnit Platform Runner #6818

merged 2 commits into from
Mar 18, 2024

Conversation

nikita-tkachenko-datadog
Copy link
Contributor

@nikita-tkachenko-datadog nikita-tkachenko-datadog commented Mar 15, 2024 •
edited by jira bot
Loading

What Does This Do

Fixes tracing for tests that are run with JUnit Platform Runner.

Motivation

There are CI Visibility customers who use JUnit Platform Runner.

Additional Notes

JUnit Platform Runner provides a way to run tests that are written for JUnit 5 with a JUnit 4 runner.
The smoke test case that is added in this PR can be checked for an example of the platform runner set up and usage.

Jira ticket: CIVIS-9429

@nikita-tkachenko-datadog nikita-tkachenko-datadog added type: bug comp: ci visibility Continuous Integration Visibility labels Mar 15, 2024
@pr-commenter
Copy link

pr-commenter bot commented Mar 15, 2024 •
edited
Loading

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master nikita-tkachenko/junit-platform-runner-fix
git_commit_date 1710435433 1710515459
git_commit_sha 6d2f2ad 4c2020b
release_version 1.32.0-SNAPSHOT~6d2f2adf41 1.32.0-SNAPSHOT~4c2020bb55
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1710518611 1710518611
ci_job_id 461047670 461047670
ci_pipeline_id 30192119 30192119
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 53 metrics, 10 unstable metrics.

Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.32.0-SNAPSHOT~4c2020bb55, baseline=1.32.0-SNAPSHOT~6d2f2adf41

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.09 s) : 0, 1090181
Total [baseline] (8.586 s) : 0, 8586422
Agent [candidate] (1.091 s) : 0, 1090832
Total [candidate] (8.597 s) : 0, 8597202
section iast
Agent [baseline] (1.201 s) : 0, 1200935
Total [baseline] (9.023 s) : 0, 9023139
Agent [candidate] (1.205 s) : 0, 1205342
Total [candidate] (9.043 s) : 0, 9042731
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.212 s) : 0, 1211526
Total [baseline] (9.029 s) : 0, 9029151
Agent [candidate] (1.205 s) : 0, 1204933
Total [candidate] (9.023 s) : 0, 9022968
section iast_TELEMETRY_OFF
Agent [baseline] (1.206 s) : 0, 1205728
Total [baseline] (9.051 s) : 0, 9050604
Agent [candidate] (1.198 s) : 0, 1197561
Total [candidate] (9.019 s) : 0, 9018955
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.09 s -
Agent iast 1.201 s 110.755 ms (10.2%)
Agent iast_HARDCODED_SECRET_DISABLED 1.212 s 121.346 ms (11.1%)
Agent iast_TELEMETRY_OFF 1.206 s 115.548 ms (10.6%)
Total tracing 8.586 s -
Total iast 9.023 s 436.717 ms (5.1%)
Total iast_HARDCODED_SECRET_DISABLED 9.029 s 442.729 ms (5.2%)
Total iast_TELEMETRY_OFF 9.051 s 464.182 ms (5.4%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.091 s -
Agent iast 1.205 s 114.511 ms (10.5%)
Agent iast_HARDCODED_SECRET_DISABLED 1.205 s 114.101 ms (10.5%)
Agent iast_TELEMETRY_OFF 1.198 s 106.729 ms (9.8%)
Total tracing 8.597 s -
Total iast 9.043 s 445.529 ms (5.2%)
Total iast_HARDCODED_SECRET_DISABLED 9.023 s 425.766 ms (5.0%)
Total iast_TELEMETRY_OFF 9.019 s 421.753 ms (4.9%) 
gantt
    title insecure-bank - break down per module: candidate=1.32.0-SNAPSHOT~4c2020bb55, baseline=1.32.0-SNAPSHOT~6d2f2adf41

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (703.81 ms) : 0, 703810
BytebuddyAgent [candidate] (704.582 ms) : 0, 704582
GlobalTracer [baseline] (294.076 ms) : 0, 294076
GlobalTracer [candidate] (293.842 ms) : 0, 293842
AppSec [baseline] (49.056 ms) : 0, 49056
AppSec [candidate] (49.072 ms) : 0, 49072
Remote Config [baseline] (735.526 µs) : 0, 736
Remote Config [candidate] (735.946 µs) : 0, 736
Telemetry [baseline] (7.803 ms) : 0, 7803
Telemetry [candidate] (7.816 ms) : 0, 7816
section iast
BytebuddyAgent [baseline] (797.981 ms) : 0, 797981
BytebuddyAgent [candidate] (800.996 ms) : 0, 800996
GlobalTracer [baseline] (287.467 ms) : 0, 287467
GlobalTracer [candidate] (289.187 ms) : 0, 289187
AppSec [baseline] (48.678 ms) : 0, 48678
AppSec [candidate] (51.507 ms) : 0, 51507
Remote Config [baseline] (592.802 µs) : 0, 593
Remote Config [candidate] (611.293 µs) : 0, 611
Telemetry [baseline] (6.551 ms) : 0, 6551
Telemetry [candidate] (7.319 ms) : 0, 7319
IAST [baseline] (25.353 ms) : 0, 25353
IAST [candidate] (21.333 ms) : 0, 21333
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (804.602 ms) : 0, 804602
BytebuddyAgent [candidate] (800.286 ms) : 0, 800286
GlobalTracer [baseline] (290.526 ms) : 0, 290526
GlobalTracer [candidate] (289.292 ms) : 0, 289292
AppSec [baseline] (49.642 ms) : 0, 49642
AppSec [candidate] (50.897 ms) : 0, 50897
Remote Config [baseline] (621.489 µs) : 0, 621
Remote Config [candidate] (592.997 µs) : 0, 593
Telemetry [baseline] (6.663 ms) : 0, 6663
Telemetry [candidate] (6.575 ms) : 0, 6575
IAST [baseline] (24.867 ms) : 0, 24867
IAST [candidate] (22.867 ms) : 0, 22867
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (799.223 ms) : 0, 799223
BytebuddyAgent [candidate] (792.256 ms) : 0, 792256
GlobalTracer [baseline] (290.626 ms) : 0, 290626
GlobalTracer [candidate] (289.861 ms) : 0, 289861
AppSec [baseline] (49.517 ms) : 0, 49517
AppSec [candidate] (48.585 ms) : 0, 48585
Remote Config [baseline] (567.181 µs) : 0, 567
Remote Config [candidate] (570.849 µs) : 0, 571
Telemetry [baseline] (6.475 ms) : 0, 6475
Telemetry [candidate] (6.491 ms) : 0, 6491
IAST [baseline] (24.821 ms) : 0, 24821
IAST [candidate] (25.536 ms) : 0, 25536
Loading
Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.32.0-SNAPSHOT~4c2020bb55, baseline=1.32.0-SNAPSHOT~6d2f2adf41

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.085 s) : 0, 1084640
Total [baseline] (9.139 s) : 0, 9139075
Agent [candidate] (1.083 s) : 0, 1083169
Total [candidate] (9.223 s) : 0, 9222828
section appsec
Agent [baseline] (1.197 s) : 0, 1196848
Total [baseline] (9.294 s) : 0, 9293551
Agent [candidate] (1.202 s) : 0, 1202149
Total [candidate] (9.28 s) : 0, 9280458
section iast
Agent [baseline] (1.21 s) : 0, 1209824
Total [baseline] (9.315 s) : 0, 9315233
Agent [candidate] (1.208 s) : 0, 1208311
Total [candidate] (9.356 s) : 0, 9355548
section profiling
Agent [baseline] (1.268 s) : 0, 1268469
Total [baseline] (9.394 s) : 0, 9394482
Agent [candidate] (1.274 s) : 0, 1273518
Total [candidate] (9.434 s) : 0, 9433894
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.085 s -
Agent appsec 1.197 s 112.208 ms (10.3%)
Agent iast 1.21 s 125.184 ms (11.5%)
Agent profiling 1.268 s 183.829 ms (16.9%)
Total tracing 9.139 s -
Total appsec 9.294 s 154.476 ms (1.7%)
Total iast 9.315 s 176.158 ms (1.9%)
Total profiling 9.394 s 255.407 ms (2.8%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.083 s -
Agent appsec 1.202 s 118.98 ms (11.0%)
Agent iast 1.208 s 125.141 ms (11.6%)
Agent profiling 1.274 s 190.349 ms (17.6%)
Total tracing 9.223 s -
Total appsec 9.28 s 57.63 ms (0.6%)
Total iast 9.356 s 132.72 ms (1.4%)
Total profiling 9.434 s 211.066 ms (2.3%) 
gantt
    title petclinic - break down per module: candidate=1.32.0-SNAPSHOT~4c2020bb55, baseline=1.32.0-SNAPSHOT~6d2f2adf41

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (699.859 ms) : 0, 699859
BytebuddyAgent [candidate] (698.437 ms) : 0, 698437
GlobalTracer [baseline] (292.675 ms) : 0, 292675
GlobalTracer [candidate] (292.652 ms) : 0, 292652
AppSec [baseline] (49.034 ms) : 0, 49034
AppSec [candidate] (48.988 ms) : 0, 48988
Remote Config [baseline] (737.09 µs) : 0, 737
Remote Config [candidate] (738.766 µs) : 0, 739
Telemetry [baseline] (7.853 ms) : 0, 7853
Telemetry [candidate] (7.762 ms) : 0, 7762
section appsec
BytebuddyAgent [baseline] (694.377 ms) : 0, 694377
BytebuddyAgent [candidate] (696.881 ms) : 0, 696881
GlobalTracer [baseline] (290.385 ms) : 0, 290385
GlobalTracer [candidate] (292.092 ms) : 0, 292092
AppSec [baseline] (152.669 ms) : 0, 152669
AppSec [candidate] (153.506 ms) : 0, 153506
Remote Config [baseline] (605.553 µs) : 0, 606
Remote Config [candidate] (605.5 µs) : 0, 606
Telemetry [baseline] (6.827 ms) : 0, 6827
Telemetry [candidate] (6.866 ms) : 0, 6866
IAST [baseline] (17.792 ms) : 0, 17792
IAST [candidate] (17.935 ms) : 0, 17935
section iast
BytebuddyAgent [baseline] (805.265 ms) : 0, 805265
BytebuddyAgent [candidate] (802.337 ms) : 0, 802337
GlobalTracer [baseline] (288.672 ms) : 0, 288672
GlobalTracer [candidate] (289.741 ms) : 0, 289741
AppSec [baseline] (48.723 ms) : 0, 48723
AppSec [candidate] (48.837 ms) : 0, 48837
Remote Config [baseline] (599.869 µs) : 0, 600
Remote Config [candidate] (607.492 µs) : 0, 607
Telemetry [baseline] (6.536 ms) : 0, 6536
Telemetry [candidate] (6.536 ms) : 0, 6536
IAST [baseline] (25.453 ms) : 0, 25453
IAST [candidate] (25.829 ms) : 0, 25829
section profiling
ProfilingAgent [baseline] (93.863 ms) : 0, 93863
ProfilingAgent [candidate] (94.453 ms) : 0, 94453
BytebuddyAgent [baseline] (686.517 ms) : 0, 686517
BytebuddyAgent [candidate] (688.712 ms) : 0, 688712
GlobalTracer [baseline] (374.547 ms) : 0, 374547
GlobalTracer [candidate] (376.502 ms) : 0, 376502
AppSec [baseline] (49.349 ms) : 0, 49349
AppSec [candidate] (49.606 ms) : 0, 49606
Remote Config [baseline] (802.431 µs) : 0, 802
Remote Config [candidate] (811.974 µs) : 0, 812
Telemetry [baseline] (7.381 ms) : 0, 7381
Telemetry [candidate] (7.386 ms) : 0, 7386
Profiling [baseline] (93.887 ms) : 0, 93887
Profiling [candidate] (94.477 ms) : 0, 94477
Loading

Load

Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.32.0-SNAPSHOT~4c2020bb55, baseline=1.32.0-SNAPSHOT~6d2f2adf41
    dateFormat X
    axisFormat %s
section baseline
no_agent (365.678 µs) : 345, 386
.   : milestone, 366,
iast (465.11 µs) : 444, 486
.   : milestone, 465,
iast_FULL (532.75 µs) : 512, 553
.   : milestone, 533,
iast_GLOBAL (487.512 µs) : 467, 508
.   : milestone, 488,
iast_HARDCODED_SECRET_DISABLED (473.913 µs) : 453, 495
.   : milestone, 474,
iast_INACTIVE (443.412 µs) : 423, 464
.   : milestone, 443,
iast_TELEMETRY_OFF (459.65 µs) : 439, 480
.   : milestone, 460,
tracing (438.253 µs) : 418, 459
.   : milestone, 438,
section candidate
no_agent (358.04 µs) : 339, 377
.   : milestone, 358,
iast (470.339 µs) : 450, 491
.   : milestone, 470,
iast_FULL (531.901 µs) : 511, 553
.   : milestone, 532,
iast_GLOBAL (494.119 µs) : 472, 516
.   : milestone, 494,
iast_HARDCODED_SECRET_DISABLED (472.712 µs) : 451, 494
.   : milestone, 473,
iast_INACTIVE (441.706 µs) : 421, 463
.   : milestone, 442,
iast_TELEMETRY_OFF (474.218 µs) : 453, 496
.   : milestone, 474,
tracing (439.672 µs) : 419, 460
.   : milestone, 440,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 365.678 µs [344.902 µs, 386.455 µs] -
iast 465.11 µs [444.326 µs, 485.894 µs] 99.432 µs (27.2%)
iast_FULL 532.75 µs [512.407 µs, 553.093 µs] 167.071 µs (45.7%)
iast_GLOBAL 487.512 µs [467.178 µs, 507.845 µs] 121.833 µs (33.3%)
iast_HARDCODED_SECRET_DISABLED 473.913 µs [453.181 µs, 494.644 µs] 108.234 µs (29.6%)
iast_INACTIVE 443.412 µs [422.535 µs, 464.289 µs] 77.734 µs (21.3%)
iast_TELEMETRY_OFF 459.65 µs [439.388 µs, 479.912 µs] 93.971 µs (25.7%)
tracing 438.253 µs [417.698 µs, 458.808 µs] 72.574 µs (19.8%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 358.04 µs [338.623 µs, 377.457 µs] -
iast 470.339 µs [449.51 µs, 491.168 µs] 112.299 µs (31.4%)
iast_FULL 531.901 µs [511.23 µs, 552.572 µs] 173.861 µs (48.6%)
iast_GLOBAL 494.119 µs [472.264 µs, 515.974 µs] 136.078 µs (38.0%)
iast_HARDCODED_SECRET_DISABLED 472.712 µs [451.499 µs, 493.924 µs] 114.671 µs (32.0%)
iast_INACTIVE 441.706 µs [420.756 µs, 462.657 µs] 83.666 µs (23.4%)
iast_TELEMETRY_OFF 474.218 µs [452.782 µs, 495.653 µs] 116.177 µs (32.4%)
tracing 439.672 µs [419.244 µs, 460.1 µs] 81.632 µs (22.8%)
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.32.0-SNAPSHOT~4c2020bb55, baseline=1.32.0-SNAPSHOT~6d2f2adf41
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.346 ms) : 1327, 1365
.   : milestone, 1346,
appsec (1.768 ms) : 1745, 1792
.   : milestone, 1768,
iast (1.519 ms) : 1496, 1543
.   : milestone, 1519,
profiling (1.509 ms) : 1485, 1533
.   : milestone, 1509,
tracing (1.477 ms) : 1454, 1500
.   : milestone, 1477,
section candidate
no_agent (1.347 ms) : 1328, 1366
.   : milestone, 1347,
appsec (1.787 ms) : 1763, 1810
.   : milestone, 1787,
iast (1.512 ms) : 1488, 1535
.   : milestone, 1512,
profiling (1.583 ms) : 1559, 1608
.   : milestone, 1583,
tracing (1.485 ms) : 1461, 1508
.   : milestone, 1485,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.346 ms [1.327 ms, 1.365 ms] -
appsec 1.768 ms [1.745 ms, 1.792 ms] 422.25 µs (31.4%)
iast 1.519 ms [1.496 ms, 1.543 ms] 173.169 µs (12.9%)
profiling 1.509 ms [1.485 ms, 1.533 ms] 162.687 µs (12.1%)
tracing 1.477 ms [1.454 ms, 1.5 ms] 130.513 µs (9.7%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.347 ms [1.328 ms, 1.366 ms] -
appsec 1.787 ms [1.763 ms, 1.81 ms] 439.672 µs (32.6%)
iast 1.512 ms [1.488 ms, 1.535 ms] 164.598 µs (12.2%)
profiling 1.583 ms [1.559 ms, 1.608 ms] 236.295 µs (17.5%)
tracing 1.485 ms [1.461 ms, 1.508 ms] 137.466 µs (10.2%)

@nikita-tkachenko-datadog nikita-tkachenko-datadog marked this pull request as ready for review March 16, 2024 17:41
@nikita-tkachenko-datadog nikita-tkachenko-datadog requested a review from a team as a code owner March 16, 2024 17:41
@nikita-tkachenko-datadog nikita-tkachenko-datadog merged commit d363f03 into master Mar 18, 2024
80 checks passed
@nikita-tkachenko-datadog nikita-tkachenko-datadog deleted the nikita-tkachenko/junit-platform-runner-fix branch March 18, 2024 09:23
@github-actions github-actions bot added this to the 1.32.0 milestone Mar 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@romainkomorndatadog romainkomorndatadog romainkomorndatadog approved these changes

Assignees
No one assigned
Labels
comp: ci visibility Continuous Integration Visibility type: bug
Projects
None yet
Milestone
1.32.0
Development

Successfully merging this pull request may close these issues.
2 participants
@nikita-tkachenko-datadog @romainkomorndatadog