PoC disable tracing via remote-config

[ygree]

What Does This Do

This PR implements tracing disabling by stopping trace publishing but keeping span creation.

This change satisfies one of the `TestDynamicConfigTracingEnabled' system test scenarios where tracing is initially enabled and then disabled via remote config.

Motivation

In-app APM Service Disablement RFC

Additional Notes

Jira ticket: APMJAVA-1248

[pr-commenter]

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	ygree/disabling-tracing-poc
git_commit_date	1708728504	1708748330
git_commit_sha	6167d5e	eafff70
release_version	1.31.0-SNAPSHOT~6167d5efc0	1.31.0-SNAPSHOT~eafff70116

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1708751234	1708751234
ci_job_id	442453525	442453525
ci_pipeline_id	28862193	28862193
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
module	Agent	Agent
parent	None	None
variant	iast	iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 45 metrics, 9 unstable metrics.

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
end_time	2024-02-24T04:43:30	2024-02-24T05:02:10
git_branch	master	ygree/disabling-tracing-poc
git_commit_date	1708728504	1708748330
git_commit_sha	6167d5e	eafff70
release_version	1.31.0-SNAPSHOT~6167d5efc0	1.31.0-SNAPSHOT~eafff70116
start_time	2024-02-24T04:43:17	2024-02-24T05:01:57

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1708751234	1708751234
ci_job_id	442453525	442453525
ci_pipeline_id	28862193	28862193
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
variant	iast	iast

Summary

Found 0 performance improvements and 1 performance regressions! Performance is the same for 11 metrics, 14 unstable metrics.

scenario	Δ mean http_req_duration	Δ mean throughput	candidate mean http_req_duration	candidate mean throughput	baseline mean http_req_duration	baseline mean throughput
scenario:load:petclinic:profiling	worse [+47.604µs; +100.180µs] or [+3.141%; +6.610%]	unstable [-641.023op/s; +413.103op/s] or [-20.833%; +13.426%]	1.590ms	2962.963op/s	1.516ms	3076.923op/s

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.31.0-SNAPSHOT~eafff70116, baseline=1.31.0-SNAPSHOT~6167d5efc0
    dateFormat X
    axisFormat %s
section baseline
no_agent (374.238 µs) : 353, 395
.   : milestone, 374,
iast (479.522 µs) : 459, 500
.   : milestone, 480,
iast_FULL (550.415 µs) : 530, 571
.   : milestone, 550,
iast_GLOBAL (498.578 µs) : 477, 520
.   : milestone, 499,
iast_HARDCODED_SECRET_DISABLED (484.385 µs) : 463, 506
.   : milestone, 484,
iast_INACTIVE (459.915 µs) : 439, 481
.   : milestone, 460,
iast_TELEMETRY_OFF (473.74 µs) : 453, 494
.   : milestone, 474,
tracing (452.096 µs) : 431, 473
.   : milestone, 452,
section candidate
no_agent (366.548 µs) : 347, 386
.   : milestone, 367,
iast (486.562 µs) : 466, 508
.   : milestone, 487,
iast_FULL (544.36 µs) : 524, 565
.   : milestone, 544,
iast_GLOBAL (506.195 µs) : 484, 528
.   : milestone, 506,
iast_HARDCODED_SECRET_DISABLED (485.474 µs) : 465, 506
.   : milestone, 485,
iast_INACTIVE (458.94 µs) : 438, 480
.   : milestone, 459,
iast_TELEMETRY_OFF (477.911 µs) : 457, 499
.   : milestone, 478,
tracing (451.462 µs) : 431, 472
.   : milestone, 451,

Loading

• baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	374.238 µs [353.024 µs, 395.451 µs]	-
iast	479.522 µs [458.77 µs, 500.274 µs]	105.284 µs (28.1%)
iast_FULL	550.415 µs [529.811 µs, 571.019 µs]	176.177 µs (47.1%)
iast_GLOBAL	498.578 µs [477.274 µs, 519.881 µs]	124.34 µs (33.2%)
iast_HARDCODED_SECRET_DISABLED	484.385 µs [463.067 µs, 505.704 µs]	110.148 µs (29.4%)
iast_INACTIVE	459.915 µs [438.966 µs, 480.863 µs]	85.677 µs (22.9%)
iast_TELEMETRY_OFF	473.74 µs [453.179 µs, 494.302 µs]	99.503 µs (26.6%)
tracing	452.096 µs [431.14 µs, 473.053 µs]	77.858 µs (20.8%)

• candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	366.548 µs [346.741 µs, 386.356 µs]	-
iast	486.562 µs [465.537 µs, 507.587 µs]	120.014 µs (32.7%)
iast_FULL	544.36 µs [523.786 µs, 564.933 µs]	177.811 µs (48.5%)
iast_GLOBAL	506.195 µs [484.112 µs, 528.277 µs]	139.646 µs (38.1%)
iast_HARDCODED_SECRET_DISABLED	485.474 µs [464.501 µs, 506.448 µs]	118.926 µs (32.4%)
iast_INACTIVE	458.94 µs [437.718 µs, 480.163 µs]	92.392 µs (25.2%)
iast_TELEMETRY_OFF	477.911 µs [456.804 µs, 499.019 µs]	111.363 µs (30.4%)
tracing	451.462 µs [430.716 µs, 472.207 µs]	84.913 µs (23.2%)

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.31.0-SNAPSHOT~eafff70116, baseline=1.31.0-SNAPSHOT~6167d5efc0
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.361 ms) : 1342, 1380
.   : milestone, 1361,
appsec (1.775 ms) : 1751, 1798
.   : milestone, 1775,
iast (1.543 ms) : 1520, 1567
.   : milestone, 1543,
profiling (1.516 ms) : 1491, 1540
.   : milestone, 1516,
tracing (1.53 ms) : 1507, 1553
.   : milestone, 1530,
section candidate
no_agent (1.367 ms) : 1348, 1386
.   : milestone, 1367,
appsec (1.795 ms) : 1771, 1819
.   : milestone, 1795,
iast (1.554 ms) : 1531, 1578
.   : milestone, 1554,
profiling (1.59 ms) : 1565, 1614
.   : milestone, 1590,
tracing (1.511 ms) : 1488, 1535
.   : milestone, 1511,

Loading

• baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.361 ms [1.342 ms, 1.38 ms]	-
appsec	1.775 ms [1.751 ms, 1.798 ms]	413.977 µs (30.4%)
iast	1.543 ms [1.52 ms, 1.567 ms]	182.495 µs (13.4%)
profiling	1.516 ms [1.491 ms, 1.54 ms]	154.808 µs (11.4%)
tracing	1.53 ms [1.507 ms, 1.553 ms]	169.082 µs (12.4%)

• candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	1.367 ms [1.348 ms, 1.386 ms]	-
appsec	1.795 ms [1.771 ms, 1.819 ms]	427.875 µs (31.3%)
iast	1.554 ms [1.531 ms, 1.578 ms]	187.199 µs (13.7%)
profiling	1.59 ms [1.565 ms, 1.614 ms]	222.423 µs (16.3%)
tracing	1.511 ms [1.488 ms, 1.535 ms]	144.12 µs (10.5%)

[nayeem-kamal]

These changes have been implemented in #6827