Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix Maven instrumentation to support command-line plugin goals invocation #7430

Merged
merged 2 commits into from
Aug 14, 2024
Merged

Fix Maven instrumentation to support command-line plugin goals invocation #7430

merged 2 commits into from
Aug 14, 2024

Conversation

nikita-tkachenko-datadog
Copy link
Contributor

@nikita-tkachenko-datadog nikita-tkachenko-datadog commented Aug 14, 2024 •
edited by jira bot
Loading

What Does This Do

Fixes Maven instrumentation to support tracing build commands that contain plugin goals (e.g. surefire:test) as opposed to regular lifecycle phases (e.g. test).

Additional Notes

Logic that determines forked JVM path for tests execution is also updated to handle mojo executions that are proxies.

Contributor Checklist

Jira ticket: SDTEST-691

@nikita-tkachenko-datadog nikita-tkachenko-datadog added type: bug comp: ci visibility Continuous Integration Visibility labels Aug 14, 2024
@smola smola mentioned this pull request Aug 14, 2024
Merged
5 tasks
@nikita-tkachenko-datadog nikita-tkachenko-datadog marked this pull request as ready for review August 14, 2024 10:06
@nikita-tkachenko-datadog nikita-tkachenko-datadog requested a review from a team as a code owner August 14, 2024 10:06
@pr-commenter
Copy link

pr-commenter bot commented Aug 14, 2024 •
edited
Loading

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master nikita-tkachenko/maven-cli-plugin-invocation
git_commit_date 1723630498 1723630657
git_commit_sha c39de5e 9143a16
release_version 1.39.0-SNAPSHOT~c39de5ed10 1.39.0-SNAPSHOT~9143a16b20
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1723632864 1723632864
ci_job_id 604696483 604696483
ci_pipeline_id 41743362 41743362
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 48 metrics, 15 unstable metrics.

Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.39.0-SNAPSHOT~9143a16b20, baseline=1.39.0-SNAPSHOT~c39de5ed10

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.053 s) : 0, 1053071
Total [baseline] (8.504 s) : 0, 8504271
Agent [candidate] (1.048 s) : 0, 1047884
Total [candidate] (8.473 s) : 0, 8473380
section iast
Agent [baseline] (1.178 s) : 0, 1178126
Total [baseline] (8.992 s) : 0, 8992269
Agent [candidate] (1.182 s) : 0, 1181743
Total [candidate] (9.008 s) : 0, 9008330
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.182 s) : 0, 1182079
Total [baseline] (8.97 s) : 0, 8969512
Agent [candidate] (1.194 s) : 0, 1193738
Total [candidate] (8.994 s) : 0, 8993877
section iast_TELEMETRY_OFF
Agent [baseline] (1.173 s) : 0, 1172528
Total [baseline] (9.003 s) : 0, 9003474
Agent [candidate] (1.173 s) : 0, 1172942
Total [candidate] (9.017 s) : 0, 9016941
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.053 s -
Agent iast 1.178 s 125.055 ms (11.9%)
Agent iast_HARDCODED_SECRET_DISABLED 1.182 s 129.008 ms (12.3%)
Agent iast_TELEMETRY_OFF 1.173 s 119.457 ms (11.3%)
Total tracing 8.504 s -
Total iast 8.992 s 487.997 ms (5.7%)
Total iast_HARDCODED_SECRET_DISABLED 8.97 s 465.241 ms (5.5%)
Total iast_TELEMETRY_OFF 9.003 s 499.203 ms (5.9%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.048 s -
Agent iast 1.182 s 133.858 ms (12.8%)
Agent iast_HARDCODED_SECRET_DISABLED 1.194 s 145.854 ms (13.9%)
Agent iast_TELEMETRY_OFF 1.173 s 125.058 ms (11.9%)
Total tracing 8.473 s -
Total iast 9.008 s 534.95 ms (6.3%)
Total iast_HARDCODED_SECRET_DISABLED 8.994 s 520.497 ms (6.1%)
Total iast_TELEMETRY_OFF 9.017 s 543.561 ms (6.4%) 
gantt
    title insecure-bank - break down per module: candidate=1.39.0-SNAPSHOT~9143a16b20, baseline=1.39.0-SNAPSHOT~c39de5ed10

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (672.023 ms) : 0, 672023
BytebuddyAgent [candidate] (668.35 ms) : 0, 668350
GlobalTracer [baseline] (308.451 ms) : 0, 308451
GlobalTracer [candidate] (307.096 ms) : 0, 307096
AppSec [baseline] (50.918 ms) : 0, 50918
AppSec [candidate] (50.903 ms) : 0, 50903
Remote Config [baseline] (688.297 µs) : 0, 688
Remote Config [candidate] (681.591 µs) : 0, 682
Telemetry [baseline] (7.493 ms) : 0, 7493
Telemetry [candidate] (7.428 ms) : 0, 7428
section iast
BytebuddyAgent [baseline] (784.167 ms) : 0, 784167
BytebuddyAgent [candidate] (787.683 ms) : 0, 787683
GlobalTracer [baseline] (296.919 ms) : 0, 296919
GlobalTracer [candidate] (296.387 ms) : 0, 296387
AppSec [baseline] (53.474 ms) : 0, 53474
AppSec [candidate] (52.504 ms) : 0, 52504
IAST [baseline] (22.367 ms) : 0, 22367
IAST [candidate] (22.412 ms) : 0, 22412
Remote Config [baseline] (562.617 µs) : 0, 563
Remote Config [candidate] (579.056 µs) : 0, 579
Telemetry [baseline] (7.156 ms) : 0, 7156
Telemetry [candidate] (8.625 ms) : 0, 8625
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (787.038 ms) : 0, 787038
BytebuddyAgent [candidate] (794.624 ms) : 0, 794624
GlobalTracer [baseline] (297.038 ms) : 0, 297038
GlobalTracer [candidate] (300.964 ms) : 0, 300964
AppSec [baseline] (49.503 ms) : 0, 49503
AppSec [candidate] (48.834 ms) : 0, 48834
IAST [baseline] (25.651 ms) : 0, 25651
IAST [candidate] (25.4 ms) : 0, 25400
Remote Config [baseline] (576.563 µs) : 0, 577
Remote Config [candidate] (581.899 µs) : 0, 582
Telemetry [baseline] (8.672 ms) : 0, 8672
Telemetry [candidate] (9.604 ms) : 0, 9604
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (779.295 ms) : 0, 779295
BytebuddyAgent [candidate] (779.955 ms) : 0, 779955
GlobalTracer [baseline] (296.799 ms) : 0, 296799
GlobalTracer [candidate] (296.956 ms) : 0, 296956
AppSec [baseline] (53.145 ms) : 0, 53145
AppSec [candidate] (52.392 ms) : 0, 52392
IAST [baseline] (21.326 ms) : 0, 21326
IAST [candidate] (21.585 ms) : 0, 21585
Remote Config [baseline] (614.124 µs) : 0, 614
Remote Config [candidate] (625.506 µs) : 0, 626
Telemetry [baseline] (7.851 ms) : 0, 7851
Telemetry [candidate] (7.89 ms) : 0, 7890
Loading
Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.39.0-SNAPSHOT~9143a16b20, baseline=1.39.0-SNAPSHOT~c39de5ed10

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.048 s) : 0, 1048446
Total [baseline] (10.334 s) : 0, 10333823
Agent [candidate] (1.055 s) : 0, 1055309
Total [candidate] (10.304 s) : 0, 10304172
section appsec
Agent [baseline] (1.17 s) : 0, 1169951
Total [baseline] (10.49 s) : 0, 10490238
Agent [candidate] (1.176 s) : 0, 1176128
Total [candidate] (10.473 s) : 0, 10473431
section iast
Agent [baseline] (1.195 s) : 0, 1194672
Total [baseline] (10.902 s) : 0, 10901556
Agent [candidate] (1.175 s) : 0, 1175496
Total [candidate] (10.788 s) : 0, 10787619
section profiling
Agent [baseline] (1.253 s) : 0, 1252795
Total [baseline] (10.672 s) : 0, 10671948
Agent [candidate] (1.245 s) : 0, 1245412
Total [candidate] (10.625 s) : 0, 10625192
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.048 s -
Agent appsec 1.17 s 121.505 ms (11.6%)
Agent iast 1.195 s 146.226 ms (13.9%)
Agent profiling 1.253 s 204.349 ms (19.5%)
Total tracing 10.334 s -
Total appsec 10.49 s 156.415 ms (1.5%)
Total iast 10.902 s 567.733 ms (5.5%)
Total profiling 10.672 s 338.125 ms (3.3%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.055 s -
Agent appsec 1.176 s 120.819 ms (11.4%)
Agent iast 1.175 s 120.187 ms (11.4%)
Agent profiling 1.245 s 190.103 ms (18.0%)
Total tracing 10.304 s -
Total appsec 10.473 s 169.259 ms (1.6%)
Total iast 10.788 s 483.447 ms (4.7%)
Total profiling 10.625 s 321.02 ms (3.1%) 
gantt
    title petclinic - break down per module: candidate=1.39.0-SNAPSHOT~9143a16b20, baseline=1.39.0-SNAPSHOT~c39de5ed10

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (668.343 ms) : 0, 668343
BytebuddyAgent [candidate] (673.099 ms) : 0, 673099
GlobalTracer [baseline] (307.193 ms) : 0, 307193
GlobalTracer [candidate] (309.042 ms) : 0, 309042
AppSec [baseline] (51.311 ms) : 0, 51311
AppSec [candidate] (51.388 ms) : 0, 51388
Remote Config [baseline] (683.371 µs) : 0, 683
Remote Config [candidate] (688.345 µs) : 0, 688
Telemetry [baseline] (7.492 ms) : 0, 7492
Telemetry [candidate] (7.54 ms) : 0, 7540
section appsec
BytebuddyAgent [baseline] (679.48 ms) : 0, 679480
BytebuddyAgent [candidate] (681.72 ms) : 0, 681720
GlobalTracer [baseline] (300.342 ms) : 0, 300342
GlobalTracer [candidate] (301.191 ms) : 0, 301191
AppSec [baseline] (157.512 ms) : 0, 157512
AppSec [candidate] (157.912 ms) : 0, 157912
Remote Config [baseline] (600.354 µs) : 0, 600
Remote Config [candidate] (600.881 µs) : 0, 601
Telemetry [baseline] (7.756 ms) : 0, 7756
Telemetry [candidate] (9.657 ms) : 0, 9657
IAST [baseline] (21.558 ms) : 0, 21558
IAST [candidate] (22.958 ms) : 0, 22958
section iast
BytebuddyAgent [baseline] (795.64 ms) : 0, 795640
BytebuddyAgent [candidate] (782.15 ms) : 0, 782150
GlobalTracer [baseline] (300.475 ms) : 0, 300475
GlobalTracer [candidate] (296.669 ms) : 0, 296669
AppSec [baseline] (53.917 ms) : 0, 53917
AppSec [candidate] (53.978 ms) : 0, 53978
Remote Config [baseline] (576.406 µs) : 0, 576
Remote Config [candidate] (593.045 µs) : 0, 593
Telemetry [baseline] (7.907 ms) : 0, 7907
Telemetry [candidate] (7.144 ms) : 0, 7144
IAST [baseline] (22.476 ms) : 0, 22476
IAST [candidate] (21.452 ms) : 0, 21452
section profiling
ProfilingAgent [baseline] (94.649 ms) : 0, 94649
ProfilingAgent [candidate] (94.728 ms) : 0, 94728
BytebuddyAgent [baseline] (667.472 ms) : 0, 667472
BytebuddyAgent [candidate] (662.381 ms) : 0, 662381
GlobalTracer [baseline] (393.233 ms) : 0, 393233
GlobalTracer [candidate] (391.094 ms) : 0, 391094
AppSec [baseline] (51.942 ms) : 0, 51942
AppSec [candidate] (51.96 ms) : 0, 51960
Remote Config [baseline] (702.233 µs) : 0, 702
Remote Config [candidate] (685.793 µs) : 0, 686
Telemetry [baseline] (7.403 ms) : 0, 7403
Telemetry [candidate] (7.385 ms) : 0, 7385
Profiling [baseline] (94.673 ms) : 0, 94673
Profiling [candidate] (94.753 ms) : 0, 94753
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2024-08-14T10:27:07 2024-08-14T10:33:56
git_branch master nikita-tkachenko/maven-cli-plugin-invocation
git_commit_date 1723630498 1723630657
git_commit_sha c39de5e 9143a16
release_version 1.39.0-SNAPSHOT~c39de5ed10 1.39.0-SNAPSHOT~9143a16b20
start_time 2024-08-14T10:26:54 2024-08-14T10:33:43
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1723631980 1723631980
ci_job_id 604696484 604696484
ci_pipeline_id 41743362 41743362
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 17 unstable metrics.

Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.39.0-SNAPSHOT~9143a16b20, baseline=1.39.0-SNAPSHOT~c39de5ed10
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.341 ms) : 1322, 1360
.   : milestone, 1341,
appsec (1.722 ms) : 1698, 1746
.   : milestone, 1722,
appsec_no_iast (1.735 ms) : 1711, 1758
.   : milestone, 1735,
iast (1.46 ms) : 1437, 1483
.   : milestone, 1460,
profiling (1.492 ms) : 1467, 1517
.   : milestone, 1492,
tracing (1.462 ms) : 1438, 1485
.   : milestone, 1462,
section candidate
no_agent (1.343 ms) : 1323, 1362
.   : milestone, 1343,
appsec (1.73 ms) : 1706, 1753
.   : milestone, 1730,
appsec_no_iast (1.695 ms) : 1670, 1719
.   : milestone, 1695,
iast (1.482 ms) : 1460, 1505
.   : milestone, 1482,
profiling (1.484 ms) : 1460, 1509
.   : milestone, 1484,
tracing (1.468 ms) : 1444, 1493
.   : milestone, 1468,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.341 ms [1.322 ms, 1.36 ms] -
appsec 1.722 ms [1.698 ms, 1.746 ms] 381.179 µs (28.4%)
appsec_no_iast 1.735 ms [1.711 ms, 1.758 ms] 393.514 µs (29.3%)
iast 1.46 ms [1.437 ms, 1.483 ms] 119.083 µs (8.9%)
profiling 1.492 ms [1.467 ms, 1.517 ms] 150.678 µs (11.2%)
tracing 1.462 ms [1.438 ms, 1.485 ms] 120.435 µs (9.0%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.343 ms [1.323 ms, 1.362 ms] -
appsec 1.73 ms [1.706 ms, 1.753 ms] 386.781 µs (28.8%)
appsec_no_iast 1.695 ms [1.67 ms, 1.719 ms] 351.975 µs (26.2%)
iast 1.482 ms [1.46 ms, 1.505 ms] 139.276 µs (10.4%)
profiling 1.484 ms [1.46 ms, 1.509 ms] 141.527 µs (10.5%)
tracing 1.468 ms [1.444 ms, 1.493 ms] 125.506 µs (9.3%)
Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.39.0-SNAPSHOT~9143a16b20, baseline=1.39.0-SNAPSHOT~c39de5ed10
    dateFormat X
    axisFormat %s
section baseline
no_agent (367.146 µs) : 348, 386
.   : milestone, 367,
iast (480.414 µs) : 459, 502
.   : milestone, 480,
iast_FULL (549.609 µs) : 527, 572
.   : milestone, 550,
iast_GLOBAL (505.462 µs) : 483, 528
.   : milestone, 505,
iast_HARDCODED_SECRET_DISABLED (487.552 µs) : 466, 509
.   : milestone, 488,
iast_INACTIVE (455.114 µs) : 433, 477
.   : milestone, 455,
iast_TELEMETRY_OFF (473.13 µs) : 452, 494
.   : milestone, 473,
tracing (433.913 µs) : 414, 454
.   : milestone, 434,
section candidate
no_agent (365.425 µs) : 346, 385
.   : milestone, 365,
iast (480.179 µs) : 459, 501
.   : milestone, 480,
iast_FULL (550.423 µs) : 528, 573
.   : milestone, 550,
iast_GLOBAL (503.358 µs) : 481, 526
.   : milestone, 503,
iast_HARDCODED_SECRET_DISABLED (479.471 µs) : 459, 500
.   : milestone, 479,
iast_INACTIVE (456.719 µs) : 434, 479
.   : milestone, 457,
iast_TELEMETRY_OFF (477.711 µs) : 456, 500
.   : milestone, 478,
tracing (439.88 µs) : 419, 460
.   : milestone, 440,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 367.146 µs [347.99 µs, 386.302 µs] -
iast 480.414 µs [459.076 µs, 501.751 µs] 113.268 µs (30.9%)
iast_FULL 549.609 µs [527.38 µs, 571.839 µs] 182.464 µs (49.7%)
iast_GLOBAL 505.462 µs [482.882 µs, 528.043 µs] 138.317 µs (37.7%)
iast_HARDCODED_SECRET_DISABLED 487.552 µs [465.984 µs, 509.12 µs] 120.407 µs (32.8%)
iast_INACTIVE 455.114 µs [432.787 µs, 477.44 µs] 87.968 µs (24.0%)
iast_TELEMETRY_OFF 473.13 µs [451.938 µs, 494.321 µs] 105.984 µs (28.9%)
tracing 433.913 µs [413.668 µs, 454.159 µs] 66.767 µs (18.2%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 365.425 µs [346.056 µs, 384.795 µs] -
iast 480.179 µs [459.201 µs, 501.157 µs] 114.754 µs (31.4%)
iast_FULL 550.423 µs [527.978 µs, 572.869 µs] 184.998 µs (50.6%)
iast_GLOBAL 503.358 µs [480.726 µs, 525.991 µs] 137.933 µs (37.7%)
iast_HARDCODED_SECRET_DISABLED 479.471 µs [458.511 µs, 500.431 µs] 114.045 µs (31.2%)
iast_INACTIVE 456.719 µs [434.349 µs, 479.088 µs] 91.293 µs (25.0%)
iast_TELEMETRY_OFF 477.711 µs [455.806 µs, 499.616 µs] 112.286 µs (30.7%)
tracing 439.88 µs [419.435 µs, 460.325 µs] 74.455 µs (20.4%)

Dacapo

@nikita-tkachenko-datadog nikita-tkachenko-datadog merged commit 0c98a58 into master Aug 14, 2024
80 checks passed
@nikita-tkachenko-datadog nikita-tkachenko-datadog deleted the nikita-tkachenko/maven-cli-plugin-invocation branch August 14, 2024 11:18
@github-actions github-actions bot added this to the 1.39.0 milestone Aug 14, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@romainkomorn-exdatadog romainkomorn-exdatadog romainkomorn-exdatadog approved these changes

@anmarchenko anmarchenko Awaiting requested review from anmarchenko anmarchenko is a code owner automatically assigned from DataDog/ci-app-libraries-java

Assignees
No one assigned
Labels
comp: ci visibility Continuous Integration Visibility type: bug
Projects
None yet
Milestone
1.39.0
Development

Successfully merging this pull request may close these issues.
2 participants
@nikita-tkachenko-datadog @romainkomorn-exdatadog