Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

enhancement(vdev): Load compose files and inject network block #17025

Merged
merged 22 commits into from
Apr 7, 2023

Conversation

jonathanpv
Copy link
Contributor

@jonathanpv jonathanpv commented Mar 31, 2023

Problem

A step towards revamping the state management of vdev

Solution

  1. Updated ComposeConfig and ComposeService structs to support serialization. This allows us to inject the networks block into the compose-temp.yaml file.
  2. Removed the networks block from multiple docker compose files.
  3. Adjusted vdev/src/testing/integration.rs to create a temporary compose-temp.yaml file with the injected networks block.

Testing

cargo vdev integration start redis

this will create a temporary valid yaml file that we programmatically inject blocks to

cargo vdev integration stop redis

this will read the file we created and close it as well as delete it

Related Issues

@netlify
Copy link

netlify bot commented Mar 31, 2023

Deploy Preview for vector-project canceled.

Name Link
🔨 Latest commit 4b0260e
🔍 Latest deploy log https://app.netlify.com/sites/vector-project/deploys/642ef282154071000882e721

@github-actions github-actions bot added the domain: ci Anything related to Vector's CI environment label Mar 31, 2023
@jonathanpv jonathanpv added the ci-condition: integration tests enable Run integration tests on this PR label Mar 31, 2023
@jonathanpv jonathanpv self-assigned this Mar 31, 2023
@jonathanpv jonathanpv requested a review from bruceg March 31, 2023 20:20
@jonathanpv jonathanpv marked this pull request as draft March 31, 2023 20:20
@jonathanpv
Copy link
Contributor Author

There is comments that I would like to open as discussion topics located in the diff view of my suggested changes. I read the instructions outlined by @bruceg and believe this is a step closer but realize there might be formatting changes needed and refactoring I should consider.

@datadog-vectordotdev
Copy link

datadog-vectordotdev bot commented Mar 31, 2023

Datadog Report

Branch report: jonathanpv/vdev-running-state
Commit report: 534d458

vector: 0 Failed, 0 New Flaky, 2138 Passed, 0 Skipped, 1h 5m 51.25s Wall Time

@github-actions
Copy link

Regression Detector Results

Run ID: b42e8acc-3039-49de-a478-9e2072938e30
Baseline: 00c0316
Comparison: 9e6de82
Total vector CPUs: 7

Explanation

A regression test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine quickly if vector performance is changed and to what degree by a pull request.

The table below, if present, lists those experiments that have experienced a statistically significant change in mean optimization goal performance between baseline and comparison SHAs with 90.00% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±5.00% change in their mean optimization goal are discarded. An experiment is erratic if its coefficient of variation is greater than 0.1. The abbreviated table will be omitted if no interesting change is observed.

Changes in experiment optimization goals with confidence ≥ 90.00% and |Δ mean %| ≥ 5.00%:

experiment goal Δ mean Δ mean % confidence
http_text_to_http_json ingress throughput 1.42MiB/CPU-s 6.02 100.00%
Fine details of change detection per experiment.
experiment goal Δ mean Δ mean % confidence baseline mean baseline stdev baseline stderr baseline outlier % baseline CoV comparison mean comparison stdev comparison stderr comparison outlier % comparison CoV erratic declared erratic
file_to_blackhole egress throughput 720.86KiB/CPU-s 11.71 99.99% 6.01MiB/CPU-s 4.52MiB/CPU-s 121.57KiB/CPU-s 0.0 0.75156 6.71MiB/CPU-s 4.49MiB/CPU-s 131.22KiB/CPU-s 0.0 0.66766 True True
http_text_to_http_json ingress throughput 1.42MiB/CPU-s 6.02 100.00% 23.68MiB/CPU-s 831.47KiB/CPU-s 10.23KiB/CPU-s 0.0 0.03429 25.1MiB/CPU-s 741.51KiB/CPU-s 9.13KiB/CPU-s 0.0 0.028845 False False
datadog_agent_remap_datadog_logs ingress throughput 661.88KiB/CPU-s 1.99 100.00% 32.41MiB/CPU-s 923.71KiB/CPU-s 11.37KiB/CPU-s 0.0 0.027831 33.06MiB/CPU-s 1022.84KiB/CPU-s 12.58KiB/CPU-s 0.0 0.030215 False False
datadog_agent_remap_datadog_logs_acks ingress throughput 433.4KiB/CPU-s 1.32 100.00% 32.12MiB/CPU-s 1.32MiB/CPU-s 16.68KiB/CPU-s 0.0 0.041224 32.54MiB/CPU-s 1.08MiB/CPU-s 13.56KiB/CPU-s 0.0 0.033074 False False
datadog_agent_remap_blackhole_acks ingress throughput 325.95KiB/CPU-s 1.05 100.00% 30.33MiB/CPU-s 1.38MiB/CPU-s 17.36KiB/CPU-s 0.0 0.045428 30.65MiB/CPU-s 1.64MiB/CPU-s 20.61KiB/CPU-s 0.0 0.053346 False False
socket_to_socket_blackhole ingress throughput 142.45KiB/CPU-s 1.05 100.00% 13.25MiB/CPU-s 324.78KiB/CPU-s 4.0KiB/CPU-s 0.0 0.023944 13.38MiB/CPU-s 290.71KiB/CPU-s 3.58KiB/CPU-s 0.0 0.021209 False False
syslog_loki ingress throughput 10.14KiB/CPU-s 0.12 98.63% 8.5MiB/CPU-s 222.61KiB/CPU-s 2.74KiB/CPU-s 0.0 0.025577 8.51MiB/CPU-s 249.35KiB/CPU-s 3.07KiB/CPU-s 0.0 0.028616 False False
enterprise_http_to_http ingress throughput 4.0KiB/CPU-s 0.03 64.25% 13.62MiB/CPU-s 276.07KiB/CPU-s 3.4KiB/CPU-s 0.0 0.019798 13.62MiB/CPU-s 220.71KiB/CPU-s 2.72KiB/CPU-s 0.0 0.015824 False False
splunk_hec_indexer_ack_blackhole ingress throughput 1.12KiB/CPU-s 0.01 19.54% 13.61MiB/CPU-s 261.92KiB/CPU-s 3.22KiB/CPU-s 0.0 0.018785 13.62MiB/CPU-s 257.42KiB/CPU-s 3.17KiB/CPU-s 0.0 0.018461 False False
splunk_hec_to_splunk_hec_logs_acks ingress throughput 193.17B/CPU-s 0.00 2.34% 13.61MiB/CPU-s 368.94KiB/CPU-s 4.54KiB/CPU-s 0.0 0.026463 13.61MiB/CPU-s 370.59KiB/CPU-s 4.56KiB/CPU-s 0.0 0.026581 False False
fluent_elasticsearch ingress throughput -45.43B/CPU-s -0.00 6.79% 45.41MiB/CPU-s 30.37KiB/CPU-s 378.53B/CPU-s 0.0 0.000653 45.41MiB/CPU-s 30.09KiB/CPU-s 375.09B/CPU-s 0.0 0.000647 False False
http_to_http_json ingress throughput -503.19B/CPU-s -0.00 9.95% 13.62MiB/CPU-s 224.32KiB/CPU-s 2.76KiB/CPU-s 0.0 0.01608 13.62MiB/CPU-s 227.63KiB/CPU-s 2.8KiB/CPU-s 0.0 0.016318 False False
splunk_hec_to_splunk_hec_logs_noack ingress throughput -4.39KiB/CPU-s -0.03 68.39% 13.62MiB/CPU-s 235.52KiB/CPU-s 2.9KiB/CPU-s 0.0 0.016887 13.61MiB/CPU-s 266.69KiB/CPU-s 3.28KiB/CPU-s 0.0 0.019128 False False
http_to_http_noack ingress throughput -5.67KiB/CPU-s -0.04 67.97% 13.62MiB/CPU-s 294.59KiB/CPU-s 3.62KiB/CPU-s 0.0 0.021128 13.61MiB/CPU-s 357.9KiB/CPU-s 4.4KiB/CPU-s 0.0 0.025679 False False
syslog_regex_logs2metric_ddmetrics ingress throughput -15.04KiB/CPU-s -0.40 95.44% 3.71MiB/CPU-s 435.54KiB/CPU-s 5.36KiB/CPU-s 0.0 0.114617 3.7MiB/CPU-s 429.04KiB/CPU-s 5.28KiB/CPU-s 0.0 0.113356 True True
datadog_agent_remap_blackhole ingress throughput -202.95KiB/CPU-s -0.65 100.00% 30.56MiB/CPU-s 1.3MiB/CPU-s 16.43KiB/CPU-s 0.0 0.04268 30.36MiB/CPU-s 1.12MiB/CPU-s 14.15KiB/CPU-s 0.0 0.036975 False False
syslog_log2metric_humio_metrics ingress throughput -50.63KiB/CPU-s -0.82 100.00% 6.02MiB/CPU-s 200.89KiB/CPU-s 2.47KiB/CPU-s 0.0 0.032561 5.98MiB/CPU-s 285.51KiB/CPU-s 3.51KiB/CPU-s 0.0 0.046658 False False
syslog_splunk_hec_logs ingress throughput -82.83KiB/CPU-s -0.91 100.00% 8.93MiB/CPU-s 302.26KiB/CPU-s 3.72KiB/CPU-s 0.0 0.033037 8.85MiB/CPU-s 279.09KiB/CPU-s 3.43KiB/CPU-s 0.0 0.030783 False False
syslog_humio_logs ingress throughput -83.47KiB/CPU-s -0.91 100.00% 8.94MiB/CPU-s 256.86KiB/CPU-s 3.16KiB/CPU-s 0.0 0.028064 8.86MiB/CPU-s 336.05KiB/CPU-s 4.13KiB/CPU-s 0.0 0.037055 False False
splunk_hec_route_s3 ingress throughput -130.87KiB/CPU-s -1.12 100.00% 11.36MiB/CPU-s 602.44KiB/CPU-s 7.41KiB/CPU-s 0.0 0.051781 11.23MiB/CPU-s 586.46KiB/CPU-s 7.22KiB/CPU-s 0.0 0.050981 False False
otlp_grpc_to_blackhole ingress throughput -15.89KiB/CPU-s -1.53 100.00% 1.01MiB/CPU-s 48.61KiB/CPU-s 612.57B/CPU-s 0.0 0.046845 1021.79KiB/CPU-s 47.25KiB/CPU-s 595.53B/CPU-s 0.0 0.046239 False False
http_to_http_acks ingress throughput -83.3KiB/CPU-s -1.55 90.70% 5.25MiB/CPU-s 2.79MiB/CPU-s 35.15KiB/CPU-s 0.0 0.531029 5.17MiB/CPU-s 2.78MiB/CPU-s 34.98KiB/CPU-s 0.0 0.536796 True False
syslog_log2metric_splunk_hec_metrics ingress throughput -205.56KiB/CPU-s -2.17 100.00% 9.23MiB/CPU-s 299.88KiB/CPU-s 3.69KiB/CPU-s 0.0 0.031716 9.03MiB/CPU-s 264.75KiB/CPU-s 3.26KiB/CPU-s 0.0 0.028623 False False
otlp_http_to_blackhole ingress throughput -43.05KiB/CPU-s -2.75 100.00% 1.53MiB/CPU-s 119.84KiB/CPU-s 1.47KiB/CPU-s 0.0 0.076514 1.49MiB/CPU-s 122.98KiB/CPU-s 1.51KiB/CPU-s 0.0 0.08074 False False

vdev/src/testing/config.rs Outdated Show resolved Hide resolved
vdev/src/testing/config.rs Outdated Show resolved Hide resolved
vdev/src/testing/integration.rs Outdated Show resolved Hide resolved
vdev/src/testing/integration.rs Outdated Show resolved Hide resolved
vdev/src/testing/integration.rs Outdated Show resolved Hide resolved
vdev/src/testing/integration.rs Outdated Show resolved Hide resolved
@github-actions
Copy link

Regression Detector Results

Run ID: 32c75a0a-198c-49cb-afa9-f549e1c29f31
Baseline: 00c0316
Comparison: fc15fbd
Total vector CPUs: 7

Explanation

A regression test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine quickly if vector performance is changed and to what degree by a pull request.

The table below, if present, lists those experiments that have experienced a statistically significant change in mean optimization goal performance between baseline and comparison SHAs with 90.00% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±5.00% change in their mean optimization goal are discarded. An experiment is erratic if its coefficient of variation is greater than 0.1. The abbreviated table will be omitted if no interesting change is observed.

No interesting changes in experiment optimization goals with confidence ≥ 90.00% and |Δ mean %| ≥ 5.00%.

Fine details of change detection per experiment.
experiment goal Δ mean Δ mean % confidence baseline mean baseline stdev baseline stderr baseline outlier % baseline CoV comparison mean comparison stdev comparison stderr comparison outlier % comparison CoV erratic declared erratic
file_to_blackhole egress throughput 1.14MiB/CPU-s 19.31 100.00% 5.91MiB/CPU-s 4.74MiB/CPU-s 118.79KiB/CPU-s 0.0 0.801595 7.05MiB/CPU-s 4.02MiB/CPU-s 131.78KiB/CPU-s 0.0 0.569435 True True
http_text_to_http_json ingress throughput 1.03MiB/CPU-s 4.31 100.00% 23.92MiB/CPU-s 622.57KiB/CPU-s 7.66KiB/CPU-s 0.0 0.025415 24.95MiB/CPU-s 720.48KiB/CPU-s 8.87KiB/CPU-s 0.0 0.028198 False False
socket_to_socket_blackhole ingress throughput 316.58KiB/CPU-s 2.37 100.00% 13.07MiB/CPU-s 371.5KiB/CPU-s 4.57KiB/CPU-s 0.0 0.027766 13.37MiB/CPU-s 471.12KiB/CPU-s 5.8KiB/CPU-s 0.0 0.034397 False False
syslog_log2metric_humio_metrics ingress throughput 138.47KiB/CPU-s 2.24 100.00% 6.05MiB/CPU-s 329.7KiB/CPU-s 4.06KiB/CPU-s 0.0 0.053236 6.18MiB/CPU-s 226.48KiB/CPU-s 2.79KiB/CPU-s 0.0 0.035769 False False
datadog_agent_remap_datadog_logs_acks ingress throughput 697.76KiB/CPU-s 2.14 100.00% 31.89MiB/CPU-s 1.18MiB/CPU-s 14.86KiB/CPU-s 0.0 0.036994 32.57MiB/CPU-s 1.07MiB/CPU-s 13.42KiB/CPU-s 0.0 0.03271 False False
datadog_agent_remap_blackhole_acks ingress throughput 609.19KiB/CPU-s 1.98 100.00% 29.97MiB/CPU-s 1.72MiB/CPU-s 21.61KiB/CPU-s 0.0 0.057233 30.57MiB/CPU-s 1.05MiB/CPU-s 13.27KiB/CPU-s 0.0 0.034429 False False
syslog_regex_logs2metric_ddmetrics ingress throughput 57.15KiB/CPU-s 1.58 100.00% 3.54MiB/CPU-s 342.95KiB/CPU-s 4.22KiB/CPU-s 0.0 0.09452 3.6MiB/CPU-s 386.81KiB/CPU-s 4.76KiB/CPU-s 0.0 0.104957 True True
syslog_humio_logs ingress throughput 82.57KiB/CPU-s 0.92 100.00% 8.75MiB/CPU-s 513.45KiB/CPU-s 6.32KiB/CPU-s 0.0 0.057289 8.83MiB/CPU-s 309.94KiB/CPU-s 3.81KiB/CPU-s 0.0 0.034267 False False
splunk_hec_route_s3 ingress throughput 74.75KiB/CPU-s 0.64 100.00% 11.43MiB/CPU-s 653.34KiB/CPU-s 8.04KiB/CPU-s 0.0 0.055798 11.51MiB/CPU-s 603.67KiB/CPU-s 7.43KiB/CPU-s 0.0 0.051229 False False
datadog_agent_remap_datadog_logs ingress throughput 96.65KiB/CPU-s 0.29 100.00% 32.08MiB/CPU-s 1.19MiB/CPU-s 15.02KiB/CPU-s 0.0 0.037176 32.17MiB/CPU-s 1.3MiB/CPU-s 16.35KiB/CPU-s 0.0 0.040345 False False
enterprise_http_to_http ingress throughput 5.44KiB/CPU-s 0.04 86.98% 13.62MiB/CPU-s 249.15KiB/CPU-s 3.07KiB/CPU-s 0.0 0.017864 13.62MiB/CPU-s 152.49KiB/CPU-s 1.88KiB/CPU-s 0.0 0.01093 False False
http_to_http_noack ingress throughput 2.53KiB/CPU-s 0.02 32.85% 13.61MiB/CPU-s 357.65KiB/CPU-s 4.4KiB/CPU-s 0.0 0.025661 13.61MiB/CPU-s 327.94KiB/CPU-s 4.03KiB/CPU-s 0.0 0.023525 False False
splunk_hec_to_splunk_hec_logs_noack ingress throughput -170.16B/CPU-s -0.00 2.86% 13.62MiB/CPU-s 267.49KiB/CPU-s 3.29KiB/CPU-s 0.0 0.019184 13.61MiB/CPU-s 266.2KiB/CPU-s 3.27KiB/CPU-s 0.0 0.019092 False False
fluent_elasticsearch ingress throughput -67.46B/CPU-s -0.00 10.04% 45.41MiB/CPU-s 30.47KiB/CPU-s 379.75B/CPU-s 0.0 0.000655 45.41MiB/CPU-s 30.2KiB/CPU-s 376.39B/CPU-s 0.0 0.000649 False False
splunk_hec_to_splunk_hec_logs_acks ingress throughput -891.47B/CPU-s -0.01 11.29% 13.61MiB/CPU-s 354.54KiB/CPU-s 4.36KiB/CPU-s 0.0 0.025428 13.61MiB/CPU-s 350.5KiB/CPU-s 4.31KiB/CPU-s 0.0 0.025141 False False
splunk_hec_indexer_ack_blackhole ingress throughput -1.59KiB/CPU-s -0.01 28.19% 13.62MiB/CPU-s 248.0KiB/CPU-s 3.05KiB/CPU-s 0.0 0.017784 13.62MiB/CPU-s 258.85KiB/CPU-s 3.18KiB/CPU-s 0.0 0.018564 False False
http_to_http_json ingress throughput -8.49KiB/CPU-s -0.06 91.63% 13.59MiB/CPU-s 273.76KiB/CPU-s 3.37KiB/CPU-s 0.0 0.019675 13.58MiB/CPU-s 290.56KiB/CPU-s 3.57KiB/CPU-s 0.0 0.020895 False False
datadog_agent_remap_blackhole ingress throughput -80.8KiB/CPU-s -0.26 99.98% 30.41MiB/CPU-s 1.24MiB/CPU-s 15.68KiB/CPU-s 0.0 0.040935 30.33MiB/CPU-s 1.16MiB/CPU-s 14.64KiB/CPU-s 0.0 0.038314 False False
syslog_splunk_hec_logs ingress throughput -52.72KiB/CPU-s -0.58 100.00% 8.9MiB/CPU-s 238.93KiB/CPU-s 2.94KiB/CPU-s 0.0 0.026207 8.85MiB/CPU-s 197.53KiB/CPU-s 2.43KiB/CPU-s 0.0 0.021793 False False
syslog_loki ingress throughput -92.23KiB/CPU-s -1.09 100.00% 8.28MiB/CPU-s 343.03KiB/CPU-s 4.22KiB/CPU-s 0.0 0.04045 8.19MiB/CPU-s 354.27KiB/CPU-s 4.36KiB/CPU-s 0.0 0.042234 False False
syslog_log2metric_splunk_hec_metrics ingress throughput -124.74KiB/CPU-s -1.33 100.00% 9.17MiB/CPU-s 340.17KiB/CPU-s 4.19KiB/CPU-s 0.0 0.036204 9.05MiB/CPU-s 257.25KiB/CPU-s 3.17KiB/CPU-s 0.0 0.027748 False False
otlp_grpc_to_blackhole ingress throughput -15.15KiB/CPU-s -1.45 100.00% 1.02MiB/CPU-s 45.69KiB/CPU-s 575.83B/CPU-s 0.0 0.043678 1.01MiB/CPU-s 43.52KiB/CPU-s 548.49B/CPU-s 0.0 0.04221 False False
http_to_http_acks ingress throughput -82.29KiB/CPU-s -1.53 91.26% 5.26MiB/CPU-s 2.7MiB/CPU-s 34.08KiB/CPU-s 0.0 0.514138 5.18MiB/CPU-s 2.7MiB/CPU-s 34.01KiB/CPU-s 0.0 0.520891 True False
otlp_http_to_blackhole ingress throughput -46.49KiB/CPU-s -2.95 100.00% 1.54MiB/CPU-s 122.58KiB/CPU-s 1.51KiB/CPU-s 0.0 0.077832 1.49MiB/CPU-s 128.21KiB/CPU-s 1.58KiB/CPU-s 0.0 0.083884 False False

@github-actions
Copy link

github-actions bot commented Apr 2, 2023

Regression Detector Results

Run ID: adfa25e3-5c44-4201-9f5f-59b9dc71e0ad
Baseline: 00c0316
Comparison: 537049b
Total vector CPUs: 7

Explanation

A regression test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine quickly if vector performance is changed and to what degree by a pull request.

The table below, if present, lists those experiments that have experienced a statistically significant change in mean optimization goal performance between baseline and comparison SHAs with 90.00% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±5.00% change in their mean optimization goal are discarded. An experiment is erratic if its coefficient of variation is greater than 0.1. The abbreviated table will be omitted if no interesting change is observed.

Changes in experiment optimization goals with confidence ≥ 90.00% and |Δ mean %| ≥ 5.00%:

experiment goal Δ mean Δ mean % confidence
http_text_to_http_json ingress throughput 1.39MiB/CPU-s 5.92 100.00%
Fine details of change detection per experiment.
experiment goal Δ mean Δ mean % confidence baseline mean baseline stdev baseline stderr baseline outlier % baseline CoV comparison mean comparison stdev comparison stderr comparison outlier % comparison CoV erratic declared erratic
http_text_to_http_json ingress throughput 1.39MiB/CPU-s 5.92 100.00% 23.44MiB/CPU-s 750.06KiB/CPU-s 9.23KiB/CPU-s 0.0 0.031251 24.82MiB/CPU-s 603.66KiB/CPU-s 7.43KiB/CPU-s 0.0 0.023746 False False
syslog_log2metric_humio_metrics ingress throughput 125.3KiB/CPU-s 2.03 100.00% 6.02MiB/CPU-s 144.83KiB/CPU-s 1.78KiB/CPU-s 0.0 0.023509 6.14MiB/CPU-s 215.68KiB/CPU-s 2.65KiB/CPU-s 0.0 0.03431 False False
datadog_agent_remap_datadog_logs ingress throughput 460.06KiB/CPU-s 1.39 100.00% 32.23MiB/CPU-s 1.06MiB/CPU-s 13.35KiB/CPU-s 0.0 0.032873 32.68MiB/CPU-s 1001.61KiB/CPU-s 12.32KiB/CPU-s 0.0 0.029927 False False
syslog_loki ingress throughput 100.04KiB/CPU-s 1.17 100.00% 8.32MiB/CPU-s 236.88KiB/CPU-s 2.91KiB/CPU-s 0.0 0.027802 8.42MiB/CPU-s 224.5KiB/CPU-s 2.76KiB/CPU-s 0.0 0.026043 False False
file_to_blackhole egress throughput 56.48KiB/CPU-s 0.94 26.56% 5.87MiB/CPU-s 4.65MiB/CPU-s 116.53KiB/CPU-s 0.0 0.791713 5.92MiB/CPU-s 4.48MiB/CPU-s 118.89KiB/CPU-s 0.0 0.755756 True True
socket_to_socket_blackhole ingress throughput 116.57KiB/CPU-s 0.87 100.00% 13.14MiB/CPU-s 337.38KiB/CPU-s 4.15KiB/CPU-s 0.0 0.025074 13.25MiB/CPU-s 410.82KiB/CPU-s 5.05KiB/CPU-s 0.0 0.03027 False False
datadog_agent_remap_datadog_logs_acks ingress throughput 228.14KiB/CPU-s 0.69 100.00% 32.39MiB/CPU-s 955.51KiB/CPU-s 11.76KiB/CPU-s 0.0 0.028805 32.61MiB/CPU-s 954.09KiB/CPU-s 11.74KiB/CPU-s 0.0 0.028566 False False
enterprise_http_to_http ingress throughput 5.52KiB/CPU-s 0.04 81.82% 13.62MiB/CPU-s 276.51KiB/CPU-s 3.4KiB/CPU-s 0.0 0.019829 13.62MiB/CPU-s 190.24KiB/CPU-s 2.34KiB/CPU-s 0.0 0.013637 False False
http_to_http_noack ingress throughput 3.74KiB/CPU-s 0.03 50.29% 13.61MiB/CPU-s 337.15KiB/CPU-s 4.15KiB/CPU-s 0.0 0.024187 13.62MiB/CPU-s 294.67KiB/CPU-s 3.63KiB/CPU-s 0.0 0.021134 False False
splunk_hec_to_splunk_hec_logs_acks ingress throughput 54.39B/CPU-s 0.00 0.65% 13.61MiB/CPU-s 372.1KiB/CPU-s 4.58KiB/CPU-s 0.0 0.02669 13.61MiB/CPU-s 378.53KiB/CPU-s 4.66KiB/CPU-s 0.0 0.027151 False False
fluent_elasticsearch ingress throughput -213.13B/CPU-s -0.00 30.50% 45.41MiB/CPU-s 30.06KiB/CPU-s 374.63B/CPU-s 0.0 0.000646 45.41MiB/CPU-s 30.13KiB/CPU-s 393.8B/CPU-s 0.0 0.000648 False False
http_to_http_json ingress throughput -549.17B/CPU-s -0.00 10.71% 13.62MiB/CPU-s 227.77KiB/CPU-s 2.8KiB/CPU-s 0.0 0.016328 13.62MiB/CPU-s 230.26KiB/CPU-s 2.83KiB/CPU-s 0.0 0.016508 False False
splunk_hec_indexer_ack_blackhole ingress throughput -1.68KiB/CPU-s -0.01 29.76% 13.62MiB/CPU-s 247.05KiB/CPU-s 3.04KiB/CPU-s 0.0 0.017716 13.62MiB/CPU-s 257.91KiB/CPU-s 3.17KiB/CPU-s 0.0 0.018497 False False
splunk_hec_to_splunk_hec_logs_noack ingress throughput -1.09KiB/CPU-s -0.01 18.29% 13.61MiB/CPU-s 266.46KiB/CPU-s 3.28KiB/CPU-s 0.0 0.019111 13.61MiB/CPU-s 273.1KiB/CPU-s 3.36KiB/CPU-s 0.0 0.019588 False False
http_to_http_acks ingress throughput -2.01KiB/CPU-s -0.04 3.31% 5.26MiB/CPU-s 2.7MiB/CPU-s 34.02KiB/CPU-s 0.0 0.513371 5.26MiB/CPU-s 2.75MiB/CPU-s 34.61KiB/CPU-s 0.0 0.52244 True False
syslog_humio_logs ingress throughput -17.98KiB/CPU-s -0.20 99.98% 8.93MiB/CPU-s 295.57KiB/CPU-s 3.64KiB/CPU-s 0.0 0.03233 8.91MiB/CPU-s 256.49KiB/CPU-s 3.15KiB/CPU-s 0.0 0.028111 False False
syslog_splunk_hec_logs ingress throughput -28.57KiB/CPU-s -0.31 100.00% 8.94MiB/CPU-s 224.7KiB/CPU-s 2.76KiB/CPU-s 0.0 0.024537 8.91MiB/CPU-s 214.3KiB/CPU-s 2.64KiB/CPU-s 0.0 0.023475 False False
splunk_hec_route_s3 ingress throughput -47.39KiB/CPU-s -0.40 100.00% 11.53MiB/CPU-s 564.65KiB/CPU-s 6.95KiB/CPU-s 0.0 0.047825 11.48MiB/CPU-s 603.15KiB/CPU-s 7.42KiB/CPU-s 0.0 0.051292 False False
syslog_log2metric_splunk_hec_metrics ingress throughput -52.25KiB/CPU-s -0.56 100.00% 9.19MiB/CPU-s 366.87KiB/CPU-s 4.52KiB/CPU-s 0.0 0.038995 9.14MiB/CPU-s 297.84KiB/CPU-s 3.66KiB/CPU-s 0.0 0.031834 False False
datadog_agent_remap_blackhole_acks ingress throughput -206.15KiB/CPU-s -0.65 100.00% 30.82MiB/CPU-s 1.33MiB/CPU-s 16.76KiB/CPU-s 0.0 0.043154 30.62MiB/CPU-s 1023.77KiB/CPU-s 12.6KiB/CPU-s 0.0 0.032646 False False
otlp_http_to_blackhole ingress throughput -22.8KiB/CPU-s -1.45 100.00% 1.54MiB/CPU-s 115.33KiB/CPU-s 1.42KiB/CPU-s 0.0 0.073284 1.51MiB/CPU-s 117.45KiB/CPU-s 1.45KiB/CPU-s 0.0 0.07573 False False
otlp_grpc_to_blackhole ingress throughput -16.15KiB/CPU-s -1.55 100.00% 1.02MiB/CPU-s 50.08KiB/CPU-s 630.95B/CPU-s 0.0 0.048057 1.0MiB/CPU-s 53.15KiB/CPU-s 669.64B/CPU-s 0.0 0.051803 False False
syslog_regex_logs2metric_ddmetrics ingress throughput -68.28KiB/CPU-s -1.83 100.00% 3.65MiB/CPU-s 374.9KiB/CPU-s 4.61KiB/CPU-s 0.0 0.100286 3.58MiB/CPU-s 365.62KiB/CPU-s 4.5KiB/CPU-s 0.0 0.099622 True True
datadog_agent_remap_blackhole ingress throughput -665.24KiB/CPU-s -2.11 100.00% 30.8MiB/CPU-s 1.09MiB/CPU-s 13.78KiB/CPU-s 0.0 0.035533 30.15MiB/CPU-s 1.35MiB/CPU-s 16.99KiB/CPU-s 0.0 0.044723 False False

@github-actions
Copy link

github-actions bot commented Apr 2, 2023

Regression Detector Results

Run ID: d399911f-b7a7-4a69-bb04-52d938797982
Baseline: 00c0316
Comparison: 06433ab
Total vector CPUs: 7

Explanation

A regression test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine quickly if vector performance is changed and to what degree by a pull request.

The table below, if present, lists those experiments that have experienced a statistically significant change in mean optimization goal performance between baseline and comparison SHAs with 90.00% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±5.00% change in their mean optimization goal are discarded. An experiment is erratic if its coefficient of variation is greater than 0.1. The abbreviated table will be omitted if no interesting change is observed.

No interesting changes in experiment optimization goals with confidence ≥ 90.00% and |Δ mean %| ≥ 5.00%.

Fine details of change detection per experiment.
experiment goal Δ mean Δ mean % confidence baseline mean baseline stdev baseline stderr baseline outlier % baseline CoV comparison mean comparison stdev comparison stderr comparison outlier % comparison CoV erratic declared erratic
http_text_to_http_json ingress throughput 1.15MiB/CPU-s 4.83 100.00% 23.74MiB/CPU-s 576.27KiB/CPU-s 7.09KiB/CPU-s 0.0 0.023707 24.88MiB/CPU-s 644.26KiB/CPU-s 7.93KiB/CPU-s 0.0 0.025283 False False
http_to_http_acks ingress throughput 79.59KiB/CPU-s 1.50 89.86% 5.17MiB/CPU-s 2.71MiB/CPU-s 34.18KiB/CPU-s 0.0 0.52494 5.24MiB/CPU-s 2.74MiB/CPU-s 34.53KiB/CPU-s 0.0 0.522516 True False
datadog_agent_remap_datadog_logs_acks ingress throughput 479.02KiB/CPU-s 1.45 100.00% 32.21MiB/CPU-s 1.08MiB/CPU-s 13.58KiB/CPU-s 0.0 0.033473 32.68MiB/CPU-s 926.59KiB/CPU-s 11.4KiB/CPU-s 0.0 0.027689 False False
datadog_agent_remap_datadog_logs ingress throughput 388.95KiB/CPU-s 1.18 100.00% 32.29MiB/CPU-s 1.1MiB/CPU-s 13.8KiB/CPU-s 0.0 0.03391 32.67MiB/CPU-s 1.06MiB/CPU-s 13.32KiB/CPU-s 0.0 0.032357 False False
socket_to_socket_blackhole ingress throughput 91.71KiB/CPU-s 0.68 100.00% 13.17MiB/CPU-s 290.91KiB/CPU-s 3.58KiB/CPU-s 0.0 0.021577 13.26MiB/CPU-s 395.93KiB/CPU-s 4.87KiB/CPU-s 0.0 0.029167 False False
syslog_splunk_hec_logs ingress throughput 39.9KiB/CPU-s 0.44 100.00% 8.81MiB/CPU-s 307.43KiB/CPU-s 3.78KiB/CPU-s 0.0 0.034063 8.85MiB/CPU-s 202.57KiB/CPU-s 2.49KiB/CPU-s 0.0 0.022346 False False
splunk_hec_route_s3 ingress throughput 37.25KiB/CPU-s 0.32 99.96% 11.48MiB/CPU-s 609.2KiB/CPU-s 7.49KiB/CPU-s 0.0 0.051821 11.52MiB/CPU-s 598.71KiB/CPU-s 7.37KiB/CPU-s 0.0 0.050768 False False
http_to_http_json ingress throughput 17.88KiB/CPU-s 0.13 100.00% 13.6MiB/CPU-s 258.82KiB/CPU-s 3.18KiB/CPU-s 0.0 0.018579 13.62MiB/CPU-s 230.25KiB/CPU-s 2.83KiB/CPU-s 0.0 0.016507 False False
enterprise_http_to_http ingress throughput 9.42KiB/CPU-s 0.07 97.85% 13.61MiB/CPU-s 296.57KiB/CPU-s 3.65KiB/CPU-s 0.0 0.021271 13.62MiB/CPU-s 151.11KiB/CPU-s 1.86KiB/CPU-s 0.0 0.01083 False False
http_to_http_noack ingress throughput 7.15KiB/CPU-s 0.05 75.26% 13.61MiB/CPU-s 388.26KiB/CPU-s 4.78KiB/CPU-s 0.0 0.027865 13.61MiB/CPU-s 318.68KiB/CPU-s 3.92KiB/CPU-s 0.0 0.02286 False False
splunk_hec_indexer_ack_blackhole ingress throughput -156.61B/CPU-s -0.00 2.64% 13.61MiB/CPU-s 263.71KiB/CPU-s 3.24KiB/CPU-s 0.0 0.018915 13.61MiB/CPU-s 266.72KiB/CPU-s 3.28KiB/CPU-s 0.0 0.019131 False False
splunk_hec_to_splunk_hec_logs_acks ingress throughput 92.39B/CPU-s 0.00 1.16% 13.61MiB/CPU-s 354.11KiB/CPU-s 4.36KiB/CPU-s 0.0 0.0254 13.61MiB/CPU-s 357.19KiB/CPU-s 4.39KiB/CPU-s 0.0 0.025621 False False
splunk_hec_to_splunk_hec_logs_noack ingress throughput -2.09KiB/CPU-s -0.01 35.36% 13.62MiB/CPU-s 252.99KiB/CPU-s 3.11KiB/CPU-s 0.0 0.018142 13.61MiB/CPU-s 269.58KiB/CPU-s 3.32KiB/CPU-s 0.0 0.019335 False False
fluent_elasticsearch ingress throughput -3.74KiB/CPU-s -0.01 83.45% 45.41MiB/CPU-s 30.04KiB/CPU-s 374.16B/CPU-s 0.0 0.000646 45.41MiB/CPU-s 219.69KiB/CPU-s 2.67KiB/CPU-s 0.0 0.004724 False False
datadog_agent_remap_blackhole ingress throughput -34.63KiB/CPU-s -0.11 88.20% 30.27MiB/CPU-s 1.34MiB/CPU-s 16.92KiB/CPU-s 0.0 0.044378 30.24MiB/CPU-s 1.13MiB/CPU-s 14.3KiB/CPU-s 0.0 0.037525 False False
datadog_agent_remap_blackhole_acks ingress throughput -44.72KiB/CPU-s -0.14 98.74% 31.01MiB/CPU-s 1.11MiB/CPU-s 14.03KiB/CPU-s 0.0 0.03589 30.97MiB/CPU-s 907.07KiB/CPU-s 11.16KiB/CPU-s 0.0 0.028601 False False
syslog_humio_logs ingress throughput -88.4KiB/CPU-s -0.96 100.00% 9.01MiB/CPU-s 183.77KiB/CPU-s 2.26KiB/CPU-s 0.0 0.019919 8.92MiB/CPU-s 258.01KiB/CPU-s 3.17KiB/CPU-s 0.0 0.028237 False False
otlp_grpc_to_blackhole ingress throughput -11.87KiB/CPU-s -1.14 100.00% 1.02MiB/CPU-s 49.86KiB/CPU-s 628.19B/CPU-s 0.0 0.047825 1.01MiB/CPU-s 47.04KiB/CPU-s 592.7B/CPU-s 0.0 0.04564 False False
syslog_loki ingress throughput -142.12KiB/CPU-s -1.67 100.00% 8.32MiB/CPU-s 264.8KiB/CPU-s 3.26KiB/CPU-s 0.0 0.031063 8.19MiB/CPU-s 320.14KiB/CPU-s 3.94KiB/CPU-s 0.0 0.038192 False False
otlp_http_to_blackhole ingress throughput -28.79KiB/CPU-s -1.82 100.00% 1.55MiB/CPU-s 113.33KiB/CPU-s 1.39KiB/CPU-s 0.0 0.071612 1.52MiB/CPU-s 117.58KiB/CPU-s 1.45KiB/CPU-s 0.0 0.075673 False False
syslog_log2metric_humio_metrics ingress throughput -118.46KiB/CPU-s -1.89 100.00% 6.13MiB/CPU-s 248.81KiB/CPU-s 3.06KiB/CPU-s 0.0 0.039633 6.01MiB/CPU-s 195.08KiB/CPU-s 2.4KiB/CPU-s 0.0 0.031672 False False
syslog_log2metric_splunk_hec_metrics ingress throughput -235.3KiB/CPU-s -2.49 100.00% 9.24MiB/CPU-s 257.39KiB/CPU-s 3.17KiB/CPU-s 0.0 0.02721 9.01MiB/CPU-s 362.48KiB/CPU-s 4.46KiB/CPU-s 0.0 0.039298 False False
file_to_blackhole egress throughput -418.89KiB/CPU-s -6.13 97.73% 6.68MiB/CPU-s 4.23MiB/CPU-s 132.09KiB/CPU-s 0.185874 0.633338 6.27MiB/CPU-s 4.55MiB/CPU-s 127.76KiB/CPU-s 0.0 0.725563 True True
syslog_regex_logs2metric_ddmetrics ingress throughput -294.0KiB/CPU-s -7.94 100.00% 3.62MiB/CPU-s 354.55KiB/CPU-s 4.36KiB/CPU-s 0.0 0.095692 3.33MiB/CPU-s 402.73KiB/CPU-s 4.96KiB/CPU-s 0.0 0.118066 True True

vdev/src/testing/config.rs Outdated Show resolved Hide resolved
@netlify
Copy link

netlify bot commented Apr 4, 2023

Deploy Preview for vrl-playground failed.

Name Link
🔨 Latest commit 4b0260e
🔍 Latest deploy log https://app.netlify.com/sites/vrl-playground/deploys/642ef282e740d6000896076a

@github-actions
Copy link

github-actions bot commented Apr 4, 2023

Regression Detector Results

Run ID: 80b8054f-f81a-49c9-bc55-7021a9d8dcc7
Baseline: c854fc6
Comparison: 1a64739
Total vector CPUs: 7

Explanation

A regression test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine quickly if vector performance is changed and to what degree by a pull request.

The table below, if present, lists those experiments that have experienced a statistically significant change in mean optimization goal performance between baseline and comparison SHAs with 90.00% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±5.00% change in their mean optimization goal are discarded. An experiment is erratic if its coefficient of variation is greater than 0.1. The abbreviated table will be omitted if no interesting change is observed.

No interesting changes in experiment optimization goals with confidence ≥ 90.00% and |Δ mean %| ≥ 5.00%.

Fine details of change detection per experiment.
experiment goal Δ mean Δ mean % confidence baseline mean baseline stdev baseline stderr baseline outlier % baseline CoV comparison mean comparison stdev comparison stderr comparison outlier % comparison CoV erratic declared erratic
file_to_blackhole egress throughput 955.3KiB/CPU-s 14.45 100.00% 6.46MiB/CPU-s 4.42MiB/CPU-s 128.33KiB/CPU-s 0.0 0.683718 7.39MiB/CPU-s 3.97MiB/CPU-s 135.33KiB/CPU-s 0.0 0.536497 True True
datadog_agent_remap_datadog_logs ingress throughput 657.42KiB/CPU-s 1.96 100.00% 32.7MiB/CPU-s 1.33MiB/CPU-s 16.81KiB/CPU-s 0.0 0.040812 33.34MiB/CPU-s 1000.08KiB/CPU-s 12.3KiB/CPU-s 0.0 0.029291 False False
datadog_agent_remap_blackhole ingress throughput 450.75KiB/CPU-s 1.46 100.00% 30.07MiB/CPU-s 1.86MiB/CPU-s 23.37KiB/CPU-s 0.0 0.061713 30.51MiB/CPU-s 1.32MiB/CPU-s 16.63KiB/CPU-s 0.0 0.043263 False False
http_text_to_http_json ingress throughput 365.37KiB/CPU-s 1.42 100.00% 25.13MiB/CPU-s 796.05KiB/CPU-s 9.8KiB/CPU-s 0.0 0.030927 25.49MiB/CPU-s 593.69KiB/CPU-s 7.31KiB/CPU-s 0.0 0.022742 False False
syslog_humio_logs ingress throughput 102.86KiB/CPU-s 1.10 100.00% 9.15MiB/CPU-s 243.83KiB/CPU-s 3.0KiB/CPU-s 0.0 0.02601 9.25MiB/CPU-s 208.41KiB/CPU-s 2.56KiB/CPU-s 0.0 0.021991 False False
http_to_http_json ingress throughput 143.3KiB/CPU-s 1.04 100.00% 13.47MiB/CPU-s 388.24KiB/CPU-s 4.78KiB/CPU-s 0.0 0.028144 13.61MiB/CPU-s 261.86KiB/CPU-s 3.22KiB/CPU-s 0.0 0.018787 False False
datadog_agent_remap_blackhole_acks ingress throughput 317.9KiB/CPU-s 1.00 100.00% 31.19MiB/CPU-s 1.28MiB/CPU-s 16.11KiB/CPU-s 0.0 0.041003 31.5MiB/CPU-s 1.0MiB/CPU-s 12.66KiB/CPU-s 0.0 0.031885 False False
socket_to_socket_blackhole ingress throughput 127.23KiB/CPU-s 0.93 100.00% 13.4MiB/CPU-s 410.44KiB/CPU-s 5.05KiB/CPU-s 0.0 0.029903 13.53MiB/CPU-s 291.84KiB/CPU-s 3.59KiB/CPU-s 0.0 0.021067 False False
http_to_http_acks ingress throughput 12.43KiB/CPU-s 0.23 20.37% 5.23MiB/CPU-s 2.71MiB/CPU-s 34.17KiB/CPU-s 0.0 0.518658 5.24MiB/CPU-s 2.69MiB/CPU-s 33.92KiB/CPU-s 0.0 0.51358 True False
enterprise_http_to_http ingress throughput 3.73KiB/CPU-s 0.03 74.39% 13.62MiB/CPU-s 220.53KiB/CPU-s 2.71KiB/CPU-s 0.0 0.01581 13.62MiB/CPU-s 150.82KiB/CPU-s 1.86KiB/CPU-s 0.0 0.01081 False False
splunk_hec_indexer_ack_blackhole ingress throughput 1.12KiB/CPU-s 0.01 19.45% 13.61MiB/CPU-s 265.84KiB/CPU-s 3.27KiB/CPU-s 0.0 0.019067 13.62MiB/CPU-s 258.34KiB/CPU-s 3.18KiB/CPU-s 0.0 0.018528 False False
splunk_hec_to_splunk_hec_logs_acks ingress throughput 787.73B/CPU-s 0.01 10.15% 13.61MiB/CPU-s 349.38KiB/CPU-s 4.3KiB/CPU-s 0.0 0.02506 13.61MiB/CPU-s 343.97KiB/CPU-s 4.23KiB/CPU-s 0.0 0.02467 False False
http_to_http_noack ingress throughput 1.9KiB/CPU-s 0.01 24.50% 13.61MiB/CPU-s 360.5KiB/CPU-s 4.43KiB/CPU-s 0.0 0.025866 13.61MiB/CPU-s 338.84KiB/CPU-s 4.17KiB/CPU-s 0.0 0.024308 False False
fluent_elasticsearch ingress throughput -19.2B/CPU-s -0.00 2.87% 45.41MiB/CPU-s 30.14KiB/CPU-s 375.65B/CPU-s 0.0 0.000648 45.41MiB/CPU-s 30.41KiB/CPU-s 379.1B/CPU-s 0.0 0.000654 False False
splunk_hec_to_splunk_hec_logs_noack ingress throughput -2.12KiB/CPU-s -0.02 34.94% 13.62MiB/CPU-s 262.65KiB/CPU-s 3.23KiB/CPU-s 0.0 0.018837 13.61MiB/CPU-s 275.02KiB/CPU-s 3.38KiB/CPU-s 0.0 0.019727 False False
splunk_hec_route_s3 ingress throughput -7.87KiB/CPU-s -0.06 61.53% 11.95MiB/CPU-s 527.71KiB/CPU-s 6.49KiB/CPU-s 0.0 0.04313 11.94MiB/CPU-s 512.36KiB/CPU-s 6.3KiB/CPU-s 0.0 0.041902 False False
syslog_splunk_hec_logs ingress throughput -9.88KiB/CPU-s -0.11 98.30% 9.06MiB/CPU-s 244.64KiB/CPU-s 3.01KiB/CPU-s 0.0 0.026375 9.05MiB/CPU-s 230.88KiB/CPU-s 2.84KiB/CPU-s 0.0 0.024918 False False
otlp_http_to_blackhole ingress throughput -4.11KiB/CPU-s -0.25 98.34% 1.58MiB/CPU-s 104.51KiB/CPU-s 1.29KiB/CPU-s 0.0 0.064543 1.58MiB/CPU-s 92.51KiB/CPU-s 1.14KiB/CPU-s 0.0 0.057278 False False
syslog_regex_logs2metric_ddmetrics ingress throughput -30.22KiB/CPU-s -0.81 100.00% 3.66MiB/CPU-s 429.45KiB/CPU-s 5.28KiB/CPU-s 0.0 0.114679 3.63MiB/CPU-s 394.6KiB/CPU-s 4.86KiB/CPU-s 0.0 0.106231 True True
syslog_loki ingress throughput -74.62KiB/CPU-s -0.87 100.00% 8.39MiB/CPU-s 348.61KiB/CPU-s 4.29KiB/CPU-s 0.0 0.040591 8.31MiB/CPU-s 363.15KiB/CPU-s 4.47KiB/CPU-s 0.0 0.042655 False False
otlp_grpc_to_blackhole ingress throughput -9.88KiB/CPU-s -0.91 100.00% 1.06MiB/CPU-s 56.18KiB/CPU-s 707.6B/CPU-s 0.0 0.051893 1.05MiB/CPU-s 46.8KiB/CPU-s 589.7B/CPU-s 0.0 0.043628 False False
syslog_log2metric_splunk_hec_metrics ingress throughput -90.68KiB/CPU-s -0.94 100.00% 9.46MiB/CPU-s 215.53KiB/CPU-s 2.65KiB/CPU-s 0.0 0.022243 9.37MiB/CPU-s 341.0KiB/CPU-s 4.2KiB/CPU-s 0.0 0.035525 False False
datadog_agent_remap_datadog_logs_acks ingress throughput -545.19KiB/CPU-s -1.64 100.00% 32.52MiB/CPU-s 1.22MiB/CPU-s 15.33KiB/CPU-s 0.0 0.037418 31.99MiB/CPU-s 1.33MiB/CPU-s 16.7KiB/CPU-s 0.0 0.041445 False False
syslog_log2metric_humio_metrics ingress throughput -201.74KiB/CPU-s -3.02 100.00% 6.52MiB/CPU-s 239.33KiB/CPU-s 2.95KiB/CPU-s 0.0 0.03583 6.33MiB/CPU-s 210.83KiB/CPU-s 2.59KiB/CPU-s 0.0 0.032546 False False

@github-actions
Copy link

github-actions bot commented Apr 4, 2023

Regression Detector Results

Run ID: 96b23aa7-ea2d-4743-9a55-58904d6a0cc8
Baseline: c854fc6
Comparison: ba1f01f
Total vector CPUs: 7

Explanation

A regression test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine quickly if vector performance is changed and to what degree by a pull request.

The table below, if present, lists those experiments that have experienced a statistically significant change in mean optimization goal performance between baseline and comparison SHAs with 90.00% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±5.00% change in their mean optimization goal are discarded. An experiment is erratic if its coefficient of variation is greater than 0.1. The abbreviated table will be omitted if no interesting change is observed.

No interesting changes in experiment optimization goals with confidence ≥ 90.00% and |Δ mean %| ≥ 5.00%.

Fine details of change detection per experiment.
experiment goal Δ mean Δ mean % confidence baseline mean baseline stdev baseline stderr baseline outlier % baseline CoV comparison mean comparison stdev comparison stderr comparison outlier % comparison CoV erratic declared erratic
file_to_blackhole egress throughput 1.66MiB/CPU-s 30.19 100.00% 5.49MiB/CPU-s 4.67MiB/CPU-s 121.64KiB/CPU-s 0.0 0.849937 7.15MiB/CPU-s 3.97MiB/CPU-s 134.69KiB/CPU-s 0.0 0.554473 True True
syslog_regex_logs2metric_ddmetrics ingress throughput 220.66KiB/CPU-s 6.43 100.00% 3.35MiB/CPU-s 443.86KiB/CPU-s 5.46KiB/CPU-s 0.0 0.129245 3.57MiB/CPU-s 489.56KiB/CPU-s 6.03KiB/CPU-s 0.0 0.133945 True True
socket_to_socket_blackhole ingress throughput 360.92KiB/CPU-s 2.70 100.00% 13.05MiB/CPU-s 552.43KiB/CPU-s 6.8KiB/CPU-s 0.0 0.041325 13.41MiB/CPU-s 378.82KiB/CPU-s 4.66KiB/CPU-s 0.0 0.027594 False False
syslog_humio_logs ingress throughput 226.43KiB/CPU-s 2.49 100.00% 8.9MiB/CPU-s 337.07KiB/CPU-s 4.15KiB/CPU-s 0.0 0.03699 9.12MiB/CPU-s 212.22KiB/CPU-s 2.61KiB/CPU-s 0.0 0.022725 False False
http_text_to_http_json ingress throughput 491.63KiB/CPU-s 1.95 100.00% 24.64MiB/CPU-s 859.64KiB/CPU-s 10.58KiB/CPU-s 0.0 0.034064 25.12MiB/CPU-s 627.63KiB/CPU-s 7.73KiB/CPU-s 0.0 0.024396 False False
http_to_http_json ingress throughput 204.31KiB/CPU-s 1.49 100.00% 13.37MiB/CPU-s 479.78KiB/CPU-s 5.9KiB/CPU-s 0.0 0.035035 13.57MiB/CPU-s 295.62KiB/CPU-s 3.64KiB/CPU-s 0.0 0.02127 False False
splunk_hec_route_s3 ingress throughput 162.73KiB/CPU-s 1.39 100.00% 11.43MiB/CPU-s 647.89KiB/CPU-s 7.97KiB/CPU-s 0.0 0.055352 11.59MiB/CPU-s 639.14KiB/CPU-s 7.86KiB/CPU-s 0.0 0.053855 False False
otlp_http_to_blackhole ingress throughput 7.46KiB/CPU-s 0.47 99.98% 1.54MiB/CPU-s 123.36KiB/CPU-s 1.52KiB/CPU-s 0.0 0.07801 1.55MiB/CPU-s 109.67KiB/CPU-s 1.35KiB/CPU-s 0.0 0.06903 False False
datadog_agent_remap_blackhole_acks ingress throughput 127.47KiB/CPU-s 0.40 100.00% 31.27MiB/CPU-s 1.3MiB/CPU-s 16.37KiB/CPU-s 0.0 0.041549 31.4MiB/CPU-s 1.1MiB/CPU-s 13.86KiB/CPU-s 0.0 0.035017 False False
enterprise_http_to_http ingress throughput 7.49KiB/CPU-s 0.05 94.86% 13.62MiB/CPU-s 272.54KiB/CPU-s 3.35KiB/CPU-s 0.0 0.019544 13.62MiB/CPU-s 153.02KiB/CPU-s 1.88KiB/CPU-s 0.0 0.010967 False False
http_to_http_noack ingress throughput 2.03KiB/CPU-s 0.01 25.26% 13.61MiB/CPU-s 371.1KiB/CPU-s 4.56KiB/CPU-s 0.0 0.02663 13.61MiB/CPU-s 352.16KiB/CPU-s 4.33KiB/CPU-s 0.0 0.025267 False False
splunk_hec_to_splunk_hec_logs_acks ingress throughput 135.21B/CPU-s 0.00 1.81% 13.61MiB/CPU-s 330.84KiB/CPU-s 4.07KiB/CPU-s 0.0 0.023731 13.61MiB/CPU-s 339.68KiB/CPU-s 4.18KiB/CPU-s 0.0 0.024365 False False
fluent_elasticsearch ingress throughput -952.49B/CPU-s -0.00 74.02% 45.41MiB/CPU-s 29.93KiB/CPU-s 372.95B/CPU-s 0.0 0.000644 45.41MiB/CPU-s 60.87KiB/CPU-s 758.55B/CPU-s 0.0 0.001309 False False
splunk_hec_indexer_ack_blackhole ingress throughput -1.21KiB/CPU-s -0.01 21.43% 13.62MiB/CPU-s 254.55KiB/CPU-s 3.13KiB/CPU-s 0.0 0.018254 13.62MiB/CPU-s 257.58KiB/CPU-s 3.17KiB/CPU-s 0.0 0.018473 False False
splunk_hec_to_splunk_hec_logs_noack ingress throughput -1.65KiB/CPU-s -0.01 27.76% 13.62MiB/CPU-s 262.58KiB/CPU-s 3.23KiB/CPU-s 0.0 0.018832 13.61MiB/CPU-s 271.45KiB/CPU-s 3.34KiB/CPU-s 0.0 0.01947 False False
datadog_agent_remap_datadog_logs_acks ingress throughput -24.58KiB/CPU-s -0.07 70.89% 32.31MiB/CPU-s 1.32MiB/CPU-s 16.63KiB/CPU-s 0.0 0.040857 32.28MiB/CPU-s 1.29MiB/CPU-s 16.3KiB/CPU-s 0.0 0.040062 False False
http_to_http_acks ingress throughput -9.95KiB/CPU-s -0.19 16.17% 5.23MiB/CPU-s 2.76MiB/CPU-s 34.77KiB/CPU-s 0.0 0.528012 5.22MiB/CPU-s 2.71MiB/CPU-s 34.2KiB/CPU-s 0.0 0.520223 True False
syslog_log2metric_splunk_hec_metrics ingress throughput -26.45KiB/CPU-s -0.28 99.98% 9.25MiB/CPU-s 392.76KiB/CPU-s 4.83KiB/CPU-s 0.0 0.041459 9.22MiB/CPU-s 416.41KiB/CPU-s 5.12KiB/CPU-s 0.0 0.04408 False False
datadog_agent_remap_blackhole ingress throughput -107.41KiB/CPU-s -0.35 99.98% 30.4MiB/CPU-s 1.61MiB/CPU-s 20.26KiB/CPU-s 0.0 0.052898 30.3MiB/CPU-s 1.58MiB/CPU-s 19.95KiB/CPU-s 0.0 0.052248 False False
otlp_grpc_to_blackhole ingress throughput -4.25KiB/CPU-s -0.40 100.00% 1.04MiB/CPU-s 51.19KiB/CPU-s 644.99B/CPU-s 0.0 0.047866 1.04MiB/CPU-s 46.33KiB/CPU-s 583.88B/CPU-s 0.0 0.043501 False False
syslog_splunk_hec_logs ingress throughput -37.5KiB/CPU-s -0.41 100.00% 8.94MiB/CPU-s 363.49KiB/CPU-s 4.47KiB/CPU-s 0.0 0.039705 8.9MiB/CPU-s 360.42KiB/CPU-s 4.43KiB/CPU-s 0.0 0.039532 False False
datadog_agent_remap_datadog_logs ingress throughput -747.61KiB/CPU-s -2.17 100.00% 33.58MiB/CPU-s 1.03MiB/CPU-s 12.98KiB/CPU-s 0.0 0.03067 32.85MiB/CPU-s 1.19MiB/CPU-s 15.05KiB/CPU-s 0.0 0.036357 False False
syslog_loki ingress throughput -241.89KiB/CPU-s -2.79 100.00% 8.46MiB/CPU-s 309.0KiB/CPU-s 3.8KiB/CPU-s 0.0 0.035664 8.22MiB/CPU-s 382.5KiB/CPU-s 4.7KiB/CPU-s 0.0 0.045414 False False
syslog_log2metric_humio_metrics ingress throughput -228.23KiB/CPU-s -3.59 100.00% 6.21MiB/CPU-s 416.76KiB/CPU-s 5.13KiB/CPU-s 0.0 0.065482 5.99MiB/CPU-s 275.21KiB/CPU-s 3.39KiB/CPU-s 0.0 0.04485 False False

@github-actions
Copy link

github-actions bot commented Apr 5, 2023

Regression Detector Results

Run ID: 40d933f2-ebfb-4cc7-80ae-03fbb6435578
Baseline: 5074d82
Comparison: e69623b
Total vector CPUs: 7

Explanation

A regression test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine quickly if vector performance is changed and to what degree by a pull request.

The table below, if present, lists those experiments that have experienced a statistically significant change in mean optimization goal performance between baseline and comparison SHAs with 90.00% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±5.00% change in their mean optimization goal are discarded. An experiment is erratic if its coefficient of variation is greater than 0.1. The abbreviated table will be omitted if no interesting change is observed.

Changes in experiment optimization goals with confidence ≥ 90.00% and |Δ mean %| ≥ 5.00%:

experiment goal Δ mean Δ mean % confidence
syslog_log2metric_humio_metrics ingress throughput -342.26KiB/CPU-s -5.21 100.00%
Fine details of change detection per experiment.
experiment goal Δ mean Δ mean % confidence baseline mean baseline stdev baseline stderr baseline outlier % baseline CoV comparison mean comparison stdev comparison stderr comparison outlier % comparison CoV erratic declared erratic
datadog_agent_remap_blackhole ingress throughput 967.51KiB/CPU-s 3.16 100.00% 29.89MiB/CPU-s 1.94MiB/CPU-s 24.44KiB/CPU-s 0.0 0.064913 30.83MiB/CPU-s 967.4KiB/CPU-s 11.9KiB/CPU-s 0.0 0.030639 False False
syslog_loki ingress throughput 211.96KiB/CPU-s 2.48 100.00% 8.35MiB/CPU-s 344.48KiB/CPU-s 4.24KiB/CPU-s 0.0 0.040267 8.56MiB/CPU-s 189.56KiB/CPU-s 2.33KiB/CPU-s 0.0 0.021622 False False
syslog_humio_logs ingress throughput 216.98KiB/CPU-s 2.37 100.00% 8.94MiB/CPU-s 273.62KiB/CPU-s 3.37KiB/CPU-s 0.0 0.029896 9.15MiB/CPU-s 218.51KiB/CPU-s 2.69KiB/CPU-s 0.0 0.023322 False False
syslog_log2metric_splunk_hec_metrics ingress throughput 213.5KiB/CPU-s 2.29 100.00% 9.1MiB/CPU-s 421.71KiB/CPU-s 5.19KiB/CPU-s 0.0 0.045241 9.31MiB/CPU-s 347.77KiB/CPU-s 4.28KiB/CPU-s 0.0 0.036474 False False
syslog_regex_logs2metric_ddmetrics ingress throughput 56.26KiB/CPU-s 1.51 100.00% 3.64MiB/CPU-s 371.25KiB/CPU-s 4.57KiB/CPU-s 0.0 0.099596 3.69MiB/CPU-s 400.36KiB/CPU-s 4.93KiB/CPU-s 0.0 0.105809 True True
http_text_to_http_json ingress throughput 320.46KiB/CPU-s 1.27 100.00% 24.56MiB/CPU-s 688.54KiB/CPU-s 8.47KiB/CPU-s 0.0 0.027375 24.87MiB/CPU-s 937.23KiB/CPU-s 11.54KiB/CPU-s 0.0 0.036793 False False
splunk_hec_route_s3 ingress throughput 144.42KiB/CPU-s 1.22 100.00% 11.59MiB/CPU-s 548.84KiB/CPU-s 6.75KiB/CPU-s 0.0 0.046235 11.73MiB/CPU-s 543.88KiB/CPU-s 6.69KiB/CPU-s 0.0 0.045267 False False
http_to_http_acks ingress throughput 58.83KiB/CPU-s 1.10 77.55% 5.22MiB/CPU-s 2.7MiB/CPU-s 34.0KiB/CPU-s 0.0 0.516947 5.28MiB/CPU-s 2.74MiB/CPU-s 34.5KiB/CPU-s 0.0 0.519074 True False
syslog_splunk_hec_logs ingress throughput 99.32KiB/CPU-s 1.09 100.00% 8.87MiB/CPU-s 265.24KiB/CPU-s 3.26KiB/CPU-s 0.0 0.029185 8.97MiB/CPU-s 192.99KiB/CPU-s 2.37KiB/CPU-s 0.0 0.021005 False False
datadog_agent_remap_datadog_logs ingress throughput 269.84KiB/CPU-s 0.81 100.00% 32.63MiB/CPU-s 1.09MiB/CPU-s 13.78KiB/CPU-s 0.0 0.033521 32.89MiB/CPU-s 969.04KiB/CPU-s 11.92KiB/CPU-s 0.0 0.028771 False False
otlp_grpc_to_blackhole ingress throughput 6.5KiB/CPU-s 0.61 100.00% 1.04MiB/CPU-s 51.4KiB/CPU-s 647.72B/CPU-s 0.0 0.04805 1.05MiB/CPU-s 32.52KiB/CPU-s 409.92B/CPU-s 0.0 0.030217 False False
enterprise_http_to_http ingress throughput 5.87KiB/CPU-s 0.04 84.51% 13.62MiB/CPU-s 275.25KiB/CPU-s 3.39KiB/CPU-s 0.0 0.019739 13.62MiB/CPU-s 191.05KiB/CPU-s 2.35KiB/CPU-s 0.0 0.013695 False False
http_to_http_noack ingress throughput 3.67KiB/CPU-s 0.03 49.10% 13.61MiB/CPU-s 335.74KiB/CPU-s 4.13KiB/CPU-s 0.0 0.024087 13.61MiB/CPU-s 301.87KiB/CPU-s 3.71KiB/CPU-s 0.0 0.021651 False False
splunk_hec_indexer_ack_blackhole ingress throughput -690.81B/CPU-s -0.00 11.90% 13.62MiB/CPU-s 258.27KiB/CPU-s 3.18KiB/CPU-s 0.0 0.018522 13.62MiB/CPU-s 259.77KiB/CPU-s 3.2KiB/CPU-s 0.0 0.01863 False False
fluent_elasticsearch ingress throughput -551.8B/CPU-s -0.00 65.63% 45.41MiB/CPU-s 29.96KiB/CPU-s 373.48B/CPU-s 0.0 0.000644 45.41MiB/CPU-s 35.89KiB/CPU-s 447.36B/CPU-s 0.0 0.000772 False False
splunk_hec_to_splunk_hec_logs_acks ingress throughput 510.98B/CPU-s 0.00 6.70% 13.61MiB/CPU-s 345.96KiB/CPU-s 4.25KiB/CPU-s 0.0 0.024814 13.61MiB/CPU-s 336.89KiB/CPU-s 4.14KiB/CPU-s 0.0 0.024163 False False
splunk_hec_to_splunk_hec_logs_noack ingress throughput -2.03KiB/CPU-s -0.01 34.04% 13.62MiB/CPU-s 259.59KiB/CPU-s 3.19KiB/CPU-s 0.0 0.018616 13.61MiB/CPU-s 269.55KiB/CPU-s 3.32KiB/CPU-s 0.0 0.019334 False False
socket_to_socket_blackhole ingress throughput -5.35KiB/CPU-s -0.04 71.06% 13.39MiB/CPU-s 313.73KiB/CPU-s 3.86KiB/CPU-s 0.0 0.022887 13.38MiB/CPU-s 264.46KiB/CPU-s 3.25KiB/CPU-s 0.0 0.0193 False False
datadog_agent_remap_datadog_logs_acks ingress throughput -40.35KiB/CPU-s -0.12 97.04% 32.63MiB/CPU-s 1004.92KiB/CPU-s 12.36KiB/CPU-s 0.0 0.030078 32.59MiB/CPU-s 1.1MiB/CPU-s 13.82KiB/CPU-s 0.0 0.033665 False False
http_to_http_json ingress throughput -116.56KiB/CPU-s -0.84 100.00% 13.62MiB/CPU-s 228.54KiB/CPU-s 2.81KiB/CPU-s 0.0 0.016384 13.51MiB/CPU-s 388.59KiB/CPU-s 4.78KiB/CPU-s 0.0 0.028092 False False
datadog_agent_remap_blackhole_acks ingress throughput -352.95KiB/CPU-s -1.11 100.00% 31.07MiB/CPU-s 1.01MiB/CPU-s 12.67KiB/CPU-s 0.0 0.032368 30.72MiB/CPU-s 1.29MiB/CPU-s 16.31KiB/CPU-s 0.0 0.042136 False False
otlp_http_to_blackhole ingress throughput -26.37KiB/CPU-s -1.62 100.00% 1.59MiB/CPU-s 92.02KiB/CPU-s 1.13KiB/CPU-s 0.0 0.056558 1.56MiB/CPU-s 100.82KiB/CPU-s 1.24KiB/CPU-s 0.0 0.06299 False False
syslog_log2metric_humio_metrics ingress throughput -342.26KiB/CPU-s -5.21 100.00% 6.41MiB/CPU-s 209.21KiB/CPU-s 2.58KiB/CPU-s 0.0 0.031847 6.08MiB/CPU-s 261.64KiB/CPU-s 3.22KiB/CPU-s 0.0 0.042019 False False
file_to_blackhole egress throughput -1.57MiB/CPU-s -20.68 100.00% 7.57MiB/CPU-s 4.17MiB/CPU-s 140.09KiB/CPU-s 3.759398 0.551097 6.0MiB/CPU-s 4.64MiB/CPU-s 119.23KiB/CPU-s 0.0 0.772729 True True

@github-actions
Copy link

github-actions bot commented Apr 5, 2023

Regression Detector Results

Run ID: db85eb0e-23a8-4fab-9ab5-25e5f6b42a39
Baseline: 5074d82
Comparison: 6a7e226
Total vector CPUs: 7

Explanation

A regression test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine quickly if vector performance is changed and to what degree by a pull request.

The table below, if present, lists those experiments that have experienced a statistically significant change in mean optimization goal performance between baseline and comparison SHAs with 90.00% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±5.00% change in their mean optimization goal are discarded. An experiment is erratic if its coefficient of variation is greater than 0.1. The abbreviated table will be omitted if no interesting change is observed.

No interesting changes in experiment optimization goals with confidence ≥ 90.00% and |Δ mean %| ≥ 5.00%.

Fine details of change detection per experiment.
experiment goal Δ mean Δ mean % confidence baseline mean baseline stdev baseline stderr baseline outlier % baseline CoV comparison mean comparison stdev comparison stderr comparison outlier % comparison CoV erratic declared erratic
file_to_blackhole egress throughput 476.83KiB/CPU-s 6.85 98.54% 6.8MiB/CPU-s 4.34MiB/CPU-s 134.52KiB/CPU-s 3.948577 0.637244 7.27MiB/CPU-s 4.23MiB/CPU-s 141.32KiB/CPU-s 0.0 0.58144 True True
http_text_to_http_json ingress throughput 676.99KiB/CPU-s 2.71 100.00% 24.37MiB/CPU-s 705.02KiB/CPU-s 8.67KiB/CPU-s 0.0 0.028252 25.03MiB/CPU-s 584.51KiB/CPU-s 7.19KiB/CPU-s 0.0 0.022804 False False
syslog_log2metric_splunk_hec_metrics ingress throughput 124.87KiB/CPU-s 1.31 100.00% 9.29MiB/CPU-s 287.98KiB/CPU-s 3.54KiB/CPU-s 0.0 0.030277 9.41MiB/CPU-s 229.22KiB/CPU-s 2.82KiB/CPU-s 0.0 0.023787 False False
syslog_loki ingress throughput 86.93KiB/CPU-s 1.01 100.00% 8.39MiB/CPU-s 318.43KiB/CPU-s 3.92KiB/CPU-s 0.0 0.037063 8.47MiB/CPU-s 199.3KiB/CPU-s 2.45KiB/CPU-s 0.0 0.022964 False False
syslog_splunk_hec_logs ingress throughput 84.62KiB/CPU-s 0.92 100.00% 9.0MiB/CPU-s 253.4KiB/CPU-s 3.12KiB/CPU-s 0.0 0.027509 9.08MiB/CPU-s 181.34KiB/CPU-s 2.23KiB/CPU-s 0.0 0.019507 False False
datadog_agent_remap_datadog_logs ingress throughput 280.62KiB/CPU-s 0.84 100.00% 32.48MiB/CPU-s 1.06MiB/CPU-s 13.36KiB/CPU-s 0.0 0.032637 32.75MiB/CPU-s 984.04KiB/CPU-s 12.11KiB/CPU-s 0.0 0.029341 False False
socket_to_socket_blackhole ingress throughput 114.04KiB/CPU-s 0.83 100.00% 13.36MiB/CPU-s 238.85KiB/CPU-s 2.94KiB/CPU-s 0.0 0.017458 13.47MiB/CPU-s 253.83KiB/CPU-s 3.12KiB/CPU-s 0.0 0.0184 False False
syslog_humio_logs ingress throughput 60.0KiB/CPU-s 0.65 100.00% 9.02MiB/CPU-s 266.19KiB/CPU-s 3.28KiB/CPU-s 0.0 0.02881 9.08MiB/CPU-s 277.21KiB/CPU-s 3.41KiB/CPU-s 0.0 0.029809 False False
datadog_agent_remap_datadog_logs_acks ingress throughput 164.33KiB/CPU-s 0.50 100.00% 32.18MiB/CPU-s 1.29MiB/CPU-s 16.23KiB/CPU-s 0.0 0.040014 32.34MiB/CPU-s 1.02MiB/CPU-s 12.87KiB/CPU-s 0.0 0.031584 False False
datadog_agent_remap_blackhole_acks ingress throughput 134.18KiB/CPU-s 0.42 100.00% 31.21MiB/CPU-s 904.39KiB/CPU-s 11.13KiB/CPU-s 0.0 0.0283 31.34MiB/CPU-s 833.26KiB/CPU-s 10.25KiB/CPU-s 0.0 0.025965 False False
enterprise_http_to_http ingress throughput 7.54KiB/CPU-s 0.05 91.96% 13.62MiB/CPU-s 294.89KiB/CPU-s 3.63KiB/CPU-s 0.0 0.02115 13.62MiB/CPU-s 189.53KiB/CPU-s 2.33KiB/CPU-s 0.0 0.013586 False False
http_to_http_noack ingress throughput 4.12KiB/CPU-s 0.03 49.32% 13.61MiB/CPU-s 375.9KiB/CPU-s 4.62KiB/CPU-s 0.0 0.026975 13.61MiB/CPU-s 336.5KiB/CPU-s 4.14KiB/CPU-s 0.0 0.02414 False False
splunk_hec_to_splunk_hec_logs_acks ingress throughput -198.04B/CPU-s -0.00 2.55% 13.61MiB/CPU-s 353.54KiB/CPU-s 4.35KiB/CPU-s 0.0 0.025358 13.61MiB/CPU-s 343.22KiB/CPU-s 4.22KiB/CPU-s 0.0 0.024619 False False
splunk_hec_indexer_ack_blackhole ingress throughput -85.44B/CPU-s -0.00 1.48% 13.62MiB/CPU-s 260.72KiB/CPU-s 3.21KiB/CPU-s 0.0 0.018699 13.62MiB/CPU-s 257.65KiB/CPU-s 3.17KiB/CPU-s 0.0 0.018479 False False
fluent_elasticsearch ingress throughput -168.2B/CPU-s -0.00 24.74% 45.41MiB/CPU-s 30.41KiB/CPU-s 379.07B/CPU-s 0.0 0.000654 45.41MiB/CPU-s 30.13KiB/CPU-s 375.53B/CPU-s 0.0 0.000648 False False
http_to_http_json ingress throughput -504.37B/CPU-s -0.00 9.96% 13.62MiB/CPU-s 226.17KiB/CPU-s 2.78KiB/CPU-s 0.0 0.016214 13.62MiB/CPU-s 226.36KiB/CPU-s 2.78KiB/CPU-s 0.0 0.016228 False False
splunk_hec_to_splunk_hec_logs_noack ingress throughput -2.33KiB/CPU-s -0.02 38.49% 13.62MiB/CPU-s 258.62KiB/CPU-s 3.18KiB/CPU-s 0.0 0.018547 13.61MiB/CPU-s 273.61KiB/CPU-s 3.37KiB/CPU-s 0.0 0.019625 False False
syslog_regex_logs2metric_ddmetrics ingress throughput -8.57KiB/CPU-s -0.23 77.96% 3.7MiB/CPU-s 392.89KiB/CPU-s 4.83KiB/CPU-s 0.0 0.103665 3.69MiB/CPU-s 410.63KiB/CPU-s 5.05KiB/CPU-s 0.0 0.108591 True True
splunk_hec_route_s3 ingress throughput -101.21KiB/CPU-s -0.85 100.00% 11.69MiB/CPU-s 543.86KiB/CPU-s 6.69KiB/CPU-s 0.0 0.045436 11.59MiB/CPU-s 663.38KiB/CPU-s 8.16KiB/CPU-s 0.0 0.055893 False False
http_to_http_acks ingress throughput -54.61KiB/CPU-s -1.01 73.81% 5.26MiB/CPU-s 2.71MiB/CPU-s 34.18KiB/CPU-s 0.0 0.516063 5.2MiB/CPU-s 2.75MiB/CPU-s 34.64KiB/CPU-s 0.0 0.528273 True False
otlp_grpc_to_blackhole ingress throughput -12.76KiB/CPU-s -1.17 100.00% 1.06MiB/CPU-s 49.43KiB/CPU-s 622.74B/CPU-s 0.0 0.045445 1.05MiB/CPU-s 38.89KiB/CPU-s 490.06B/CPU-s 0.0 0.036173 False False
syslog_log2metric_humio_metrics ingress throughput -127.07KiB/CPU-s -1.93 100.00% 6.43MiB/CPU-s 209.13KiB/CPU-s 2.57KiB/CPU-s 0.0 0.031767 6.3MiB/CPU-s 147.15KiB/CPU-s 1.81KiB/CPU-s 0.0 0.022791 False False
datadog_agent_remap_blackhole ingress throughput -709.0KiB/CPU-s -2.23 100.00% 31.09MiB/CPU-s 1.14MiB/CPU-s 14.4KiB/CPU-s 0.0 0.036761 30.4MiB/CPU-s 1.67MiB/CPU-s 21.07KiB/CPU-s 0.0 0.055001 False False
otlp_http_to_blackhole ingress throughput -67.59KiB/CPU-s -4.13 100.00% 1.6MiB/CPU-s 92.58KiB/CPU-s 1.14KiB/CPU-s 0.0 0.056607 1.53MiB/CPU-s 118.15KiB/CPU-s 1.45KiB/CPU-s 0.0 0.075355 False False

@jonathanpv jonathanpv marked this pull request as ready for review April 5, 2023 08:20
tempfile = "3.4.0"
tempfile = "3.5.0"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm curious why this needed updating? It looks like it didn't change the actual crate version, so it's just reflecting what we're already using.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I could be wrong but when I tried a merge master it used tempfile 3.5.0. So I tried matching the version. I assume I need to have these in the vdev/Cargo.toml match the one in the repo level.

vdev/src/testing/integration.rs Outdated Show resolved Hide resolved
@jonathanpv
Copy link
Contributor Author

Making this work on Windows will require removing the conditional compilation on loading, parsing, and creation of the NamedTempFile for the original compose.yamlfile.

@github-actions
Copy link

github-actions bot commented Apr 5, 2023

Regression Detector Results

Run ID: c6eb591c-567a-48e0-a984-5b8a895f4c36
Baseline: 88d6aec
Comparison: eacf027
Total vector CPUs: 7

Explanation

A regression test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine quickly if vector performance is changed and to what degree by a pull request.

The table below, if present, lists those experiments that have experienced a statistically significant change in mean optimization goal performance between baseline and comparison SHAs with 90.00% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±5.00% change in their mean optimization goal are discarded. An experiment is erratic if its coefficient of variation is greater than 0.1. The abbreviated table will be omitted if no interesting change is observed.

No interesting changes in experiment optimization goals with confidence ≥ 90.00% and |Δ mean %| ≥ 5.00%.

Fine details of change detection per experiment.
experiment goal Δ mean Δ mean % confidence baseline mean baseline stdev baseline stderr baseline outlier % baseline CoV comparison mean comparison stdev comparison stderr comparison outlier % comparison CoV erratic declared erratic
http_text_to_http_json ingress throughput 901.05KiB/CPU-s 3.58 100.00% 24.61MiB/CPU-s 562.39KiB/CPU-s 6.92KiB/CPU-s 0.0 0.022317 25.49MiB/CPU-s 532.24KiB/CPU-s 6.55KiB/CPU-s 0.0 0.020391 False False
syslog_log2metric_humio_metrics ingress throughput 123.9KiB/CPU-s 1.99 100.00% 6.08MiB/CPU-s 308.06KiB/CPU-s 3.79KiB/CPU-s 0.0 0.049442 6.21MiB/CPU-s 251.15KiB/CPU-s 3.09KiB/CPU-s 0.0 0.039523 False False
syslog_splunk_hec_logs ingress throughput 79.29KiB/CPU-s 0.85 100.00% 9.12MiB/CPU-s 222.91KiB/CPU-s 2.74KiB/CPU-s 0.0 0.023879 9.19MiB/CPU-s 149.14KiB/CPU-s 1.84KiB/CPU-s 0.0 0.015842 False False
datadog_agent_remap_datadog_logs_acks ingress throughput 209.28KiB/CPU-s 0.62 100.00% 33.02MiB/CPU-s 1.01MiB/CPU-s 12.71KiB/CPU-s 0.0 0.030565 33.22MiB/CPU-s 894.32KiB/CPU-s 11.0KiB/CPU-s 0.0 0.026288 False False
datadog_agent_remap_datadog_logs ingress throughput 200.94KiB/CPU-s 0.59 100.00% 33.33MiB/CPU-s 956.72KiB/CPU-s 11.77KiB/CPU-s 0.0 0.028029 33.53MiB/CPU-s 752.88KiB/CPU-s 9.26KiB/CPU-s 0.0 0.021928 False False
syslog_loki ingress throughput 24.16KiB/CPU-s 0.28 100.00% 8.42MiB/CPU-s 281.02KiB/CPU-s 3.46KiB/CPU-s 0.0 0.032581 8.45MiB/CPU-s 292.64KiB/CPU-s 3.6KiB/CPU-s 0.0 0.033834 False False
http_to_http_json ingress throughput 35.5KiB/CPU-s 0.26 100.00% 13.59MiB/CPU-s 317.83KiB/CPU-s 3.91KiB/CPU-s 0.0 0.022845 13.62MiB/CPU-s 229.88KiB/CPU-s 2.83KiB/CPU-s 0.0 0.016481 False False
splunk_hec_route_s3 ingress throughput 21.68KiB/CPU-s 0.19 94.34% 11.4MiB/CPU-s 681.04KiB/CPU-s 8.38KiB/CPU-s 0.0 0.058355 11.42MiB/CPU-s 624.84KiB/CPU-s 7.69KiB/CPU-s 0.0 0.05344 False False
http_to_http_acks ingress throughput 4.26KiB/CPU-s 0.08 7.00% 5.22MiB/CPU-s 2.7MiB/CPU-s 34.01KiB/CPU-s 0.0 0.516641 5.23MiB/CPU-s 2.74MiB/CPU-s 34.52KiB/CPU-s 0.0 0.523894 True False
enterprise_http_to_http ingress throughput 5.56KiB/CPU-s 0.04 82.55% 13.62MiB/CPU-s 274.79KiB/CPU-s 3.38KiB/CPU-s 0.0 0.019706 13.62MiB/CPU-s 188.04KiB/CPU-s 2.31KiB/CPU-s 0.0 0.01348 False False
http_to_http_noack ingress throughput -14.85B/CPU-s -0.00 0.21% 13.61MiB/CPU-s 322.09KiB/CPU-s 3.96KiB/CPU-s 0.0 0.023104 13.61MiB/CPU-s 320.99KiB/CPU-s 3.95KiB/CPU-s 0.0 0.023025 False False
splunk_hec_indexer_ack_blackhole ingress throughput 340.68B/CPU-s 0.00 5.89% 13.61MiB/CPU-s 261.85KiB/CPU-s 3.22KiB/CPU-s 0.0 0.01878 13.62MiB/CPU-s 255.61KiB/CPU-s 3.14KiB/CPU-s 0.0 0.018332 False False
splunk_hec_to_splunk_hec_logs_acks ingress throughput -285.82B/CPU-s -0.00 3.64% 13.61MiB/CPU-s 358.21KiB/CPU-s 4.41KiB/CPU-s 0.0 0.025694 13.61MiB/CPU-s 344.21KiB/CPU-s 4.23KiB/CPU-s 0.0 0.02469 False False
fluent_elasticsearch ingress throughput -184.55B/CPU-s -0.00 26.99% 45.41MiB/CPU-s 30.33KiB/CPU-s 378.02B/CPU-s 0.0 0.000652 45.41MiB/CPU-s 30.37KiB/CPU-s 378.6B/CPU-s 0.0 0.000653 False False
splunk_hec_to_splunk_hec_logs_noack ingress throughput -988.61B/CPU-s -0.01 16.03% 13.61MiB/CPU-s 270.86KiB/CPU-s 3.33KiB/CPU-s 0.0 0.019428 13.61MiB/CPU-s 277.82KiB/CPU-s 3.42KiB/CPU-s 0.0 0.019928 False False
syslog_humio_logs ingress throughput -2.22KiB/CPU-s -0.02 43.78% 9.18MiB/CPU-s 205.54KiB/CPU-s 2.53KiB/CPU-s 0.0 0.021866 9.18MiB/CPU-s 233.96KiB/CPU-s 2.88KiB/CPU-s 0.0 0.024896 False False
syslog_log2metric_splunk_hec_metrics ingress throughput -28.27KiB/CPU-s -0.30 100.00% 9.27MiB/CPU-s 322.87KiB/CPU-s 3.97KiB/CPU-s 0.0 0.034019 9.24MiB/CPU-s 328.96KiB/CPU-s 4.05KiB/CPU-s 0.0 0.034764 False False
datadog_agent_remap_blackhole_acks ingress throughput -151.24KiB/CPU-s -0.48 100.00% 30.99MiB/CPU-s 1.35MiB/CPU-s 17.06KiB/CPU-s 0.0 0.043675 30.84MiB/CPU-s 1.09MiB/CPU-s 13.78KiB/CPU-s 0.0 0.035452 False False
socket_to_socket_blackhole ingress throughput -70.75KiB/CPU-s -0.51 100.00% 13.58MiB/CPU-s 244.74KiB/CPU-s 3.01KiB/CPU-s 0.0 0.017601 13.51MiB/CPU-s 291.79KiB/CPU-s 3.59KiB/CPU-s 0.0 0.021092 False False
syslog_regex_logs2metric_ddmetrics ingress throughput -23.48KiB/CPU-s -0.63 99.96% 3.62MiB/CPU-s 360.3KiB/CPU-s 4.43KiB/CPU-s 0.0 0.097243 3.6MiB/CPU-s 406.86KiB/CPU-s 5.01KiB/CPU-s 0.0 0.11051 True True
otlp_grpc_to_blackhole ingress throughput -16.94KiB/CPU-s -1.56 100.00% 1.06MiB/CPU-s 48.86KiB/CPU-s 615.62B/CPU-s 0.0 0.045134 1.04MiB/CPU-s 45.37KiB/CPU-s 571.78B/CPU-s 0.0 0.042576 False False
otlp_http_to_blackhole ingress throughput -33.92KiB/CPU-s -2.07 100.00% 1.6MiB/CPU-s 90.95KiB/CPU-s 1.12KiB/CPU-s 0.0 0.055418 1.57MiB/CPU-s 100.03KiB/CPU-s 1.23KiB/CPU-s 0.0 0.062234 False False
datadog_agent_remap_blackhole ingress throughput -836.86KiB/CPU-s -2.62 100.00% 31.2MiB/CPU-s 1.22MiB/CPU-s 15.32KiB/CPU-s 0.0 0.03898 30.39MiB/CPU-s 1.41MiB/CPU-s 17.78KiB/CPU-s 0.0 0.046445 False False
file_to_blackhole egress throughput -417.44KiB/CPU-s -5.75 97.02% 7.09MiB/CPU-s 4.35MiB/CPU-s 138.22KiB/CPU-s 3.182257 0.61275 6.68MiB/CPU-s 4.33MiB/CPU-s 133.3KiB/CPU-s 8.605072 0.646928 True True

@github-actions
Copy link

github-actions bot commented Apr 6, 2023

Regression Detector Results

Run ID: c8a0fccb-1119-4418-8bb0-1daefc356271
Baseline: b25a0fb
Comparison: cd1e267
Total vector CPUs: 7

Explanation

A regression test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine quickly if vector performance is changed and to what degree by a pull request.

The table below, if present, lists those experiments that have experienced a statistically significant change in mean optimization goal performance between baseline and comparison SHAs with 90.00% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±5.00% change in their mean optimization goal are discarded. An experiment is erratic if its coefficient of variation is greater than 0.1. The abbreviated table will be omitted if no interesting change is observed.

No interesting changes in experiment optimization goals with confidence ≥ 90.00% and |Δ mean %| ≥ 5.00%.

Fine details of change detection per experiment.
experiment goal Δ mean Δ mean % confidence baseline mean baseline stdev baseline stderr baseline outlier % baseline CoV comparison mean comparison stdev comparison stderr comparison outlier % comparison CoV erratic declared erratic
file_to_blackhole egress throughput 541.23KiB/CPU-s 7.91 99.39% 6.68MiB/CPU-s 4.41MiB/CPU-s 138.09KiB/CPU-s 10.177404 0.660517 7.21MiB/CPU-s 4.05MiB/CPU-s 140.55KiB/CPU-s 0.0 0.561739 True True
http_text_to_http_json ingress throughput 797.98KiB/CPU-s 3.22 100.00% 24.2MiB/CPU-s 674.6KiB/CPU-s 8.71KiB/CPU-s 0.0 0.027223 24.98MiB/CPU-s 650.27KiB/CPU-s 8.39KiB/CPU-s 0.0 0.025423 False False
datadog_agent_remap_blackhole_acks ingress throughput 814.61KiB/CPU-s 2.67 100.00% 29.76MiB/CPU-s 1.97MiB/CPU-s 25.99KiB/CPU-s 0.0 0.06608 30.56MiB/CPU-s 1.07MiB/CPU-s 14.09KiB/CPU-s 0.0 0.034875 False False
splunk_hec_route_s3 ingress throughput 174.61KiB/CPU-s 1.48 100.00% 11.53MiB/CPU-s 618.55KiB/CPU-s 7.98KiB/CPU-s 0.0 0.052403 11.7MiB/CPU-s 528.41KiB/CPU-s 6.82KiB/CPU-s 0.0 0.044114 False False
http_to_http_acks ingress throughput 52.65KiB/CPU-s 0.99 70.06% 5.19MiB/CPU-s 2.68MiB/CPU-s 35.45KiB/CPU-s 0.0 0.516931 5.24MiB/CPU-s 2.75MiB/CPU-s 36.29KiB/CPU-s 0.0 0.524036 True False
syslog_log2metric_splunk_hec_metrics ingress throughput 67.08KiB/CPU-s 0.71 100.00% 9.2MiB/CPU-s 307.83KiB/CPU-s 3.97KiB/CPU-s 0.0 0.03268 9.26MiB/CPU-s 399.86KiB/CPU-s 5.16KiB/CPU-s 0.0 0.04215 False False
syslog_regex_logs2metric_ddmetrics ingress throughput 24.81KiB/CPU-s 0.69 99.87% 3.52MiB/CPU-s 378.57KiB/CPU-s 4.89KiB/CPU-s 0.0 0.105064 3.54MiB/CPU-s 463.33KiB/CPU-s 5.98KiB/CPU-s 0.0 0.127708 True True
syslog_humio_logs ingress throughput 39.58KiB/CPU-s 0.42 100.00% 9.12MiB/CPU-s 215.19KiB/CPU-s 2.78KiB/CPU-s 0.0 0.023036 9.16MiB/CPU-s 232.46KiB/CPU-s 3.0KiB/CPU-s 0.0 0.02478 False False
enterprise_http_to_http ingress throughput 10.32KiB/CPU-s 0.07 98.21% 13.61MiB/CPU-s 301.71KiB/CPU-s 3.89KiB/CPU-s 0.0 0.02164 13.62MiB/CPU-s 152.0KiB/CPU-s 1.96KiB/CPU-s 0.0 0.010894 False False
http_to_http_noack ingress throughput 5.77KiB/CPU-s 0.04 62.44% 13.61MiB/CPU-s 381.37KiB/CPU-s 4.92KiB/CPU-s 0.0 0.02737 13.61MiB/CPU-s 331.16KiB/CPU-s 4.27KiB/CPU-s 0.0 0.023757 False False
syslog_splunk_hec_logs ingress throughput 2.3KiB/CPU-s 0.03 25.29% 8.84MiB/CPU-s 346.01KiB/CPU-s 4.46KiB/CPU-s 0.0 0.038218 8.84MiB/CPU-s 432.66KiB/CPU-s 5.58KiB/CPU-s 0.0 0.047777 False False
socket_to_socket_blackhole ingress throughput 4.22KiB/CPU-s 0.03 38.45% 13.11MiB/CPU-s 500.7KiB/CPU-s 6.46KiB/CPU-s 0.0 0.037306 13.11MiB/CPU-s 415.1KiB/CPU-s 5.36KiB/CPU-s 0.0 0.030919 False False
splunk_hec_indexer_ack_blackhole ingress throughput 999.89B/CPU-s 0.01 16.19% 13.61MiB/CPU-s 264.3KiB/CPU-s 3.41KiB/CPU-s 0.0 0.018957 13.62MiB/CPU-s 259.73KiB/CPU-s 3.35KiB/CPU-s 0.0 0.018628 False False
splunk_hec_to_splunk_hec_logs_acks ingress throughput -328.33B/CPU-s -0.00 4.01% 13.61MiB/CPU-s 365.18KiB/CPU-s 4.71KiB/CPU-s 0.0 0.026194 13.61MiB/CPU-s 333.88KiB/CPU-s 4.31KiB/CPU-s 0.0 0.023949 False False
fluent_elasticsearch ingress throughput 266.02B/CPU-s 0.00 36.25% 45.41MiB/CPU-s 30.01KiB/CPU-s 392.18B/CPU-s 0.0 0.000645 45.41MiB/CPU-s 31.08KiB/CPU-s 406.04B/CPU-s 0.0 0.000668 False False
splunk_hec_to_splunk_hec_logs_noack ingress throughput -964.68B/CPU-s -0.01 15.57% 13.62MiB/CPU-s 260.37KiB/CPU-s 3.36KiB/CPU-s 0.0 0.018673 13.62MiB/CPU-s 265.37KiB/CPU-s 3.42KiB/CPU-s 0.0 0.019033 False False
http_to_http_json ingress throughput -43.29KiB/CPU-s -0.31 100.00% 13.6MiB/CPU-s 257.07KiB/CPU-s 3.32KiB/CPU-s 0.0 0.018451 13.56MiB/CPU-s 309.02KiB/CPU-s 3.99KiB/CPU-s 0.0 0.022249 False False
syslog_loki ingress throughput -32.84KiB/CPU-s -0.38 100.00% 8.5MiB/CPU-s 236.23KiB/CPU-s 3.05KiB/CPU-s 0.0 0.027153 8.46MiB/CPU-s 228.19KiB/CPU-s 2.94KiB/CPU-s 0.0 0.026328 False False
datadog_agent_remap_datadog_logs ingress throughput -279.17KiB/CPU-s -0.84 100.00% 32.27MiB/CPU-s 1.48MiB/CPU-s 19.61KiB/CPU-s 0.0 0.04599 32.0MiB/CPU-s 1.31MiB/CPU-s 17.3KiB/CPU-s 0.0 0.040905 False False
otlp_grpc_to_blackhole ingress throughput -10.22KiB/CPU-s -0.96 100.00% 1.04MiB/CPU-s 54.46KiB/CPU-s 719.64B/CPU-s 0.0 0.051274 1.03MiB/CPU-s 52.74KiB/CPU-s 696.88B/CPU-s 0.0 0.050131 False False
datadog_agent_remap_blackhole ingress throughput -319.62KiB/CPU-s -1.00 100.00% 31.07MiB/CPU-s 1.1MiB/CPU-s 14.56KiB/CPU-s 0.0 0.035473 30.75MiB/CPU-s 1.33MiB/CPU-s 17.55KiB/CPU-s 0.0 0.043186 False False
datadog_agent_remap_datadog_logs_acks ingress throughput -398.73KiB/CPU-s -1.20 100.00% 32.5MiB/CPU-s 995.27KiB/CPU-s 12.84KiB/CPU-s 0.0 0.029903 32.11MiB/CPU-s 1.01MiB/CPU-s 13.33KiB/CPU-s 0.0 0.031423 False False
otlp_http_to_blackhole ingress throughput -27.64KiB/CPU-s -1.73 100.00% 1.56MiB/CPU-s 112.96KiB/CPU-s 1.46KiB/CPU-s 0.0 0.070543 1.54MiB/CPU-s 122.82KiB/CPU-s 1.58KiB/CPU-s 0.0 0.078052 False False
syslog_log2metric_humio_metrics ingress throughput -283.5KiB/CPU-s -4.37 100.00% 6.33MiB/CPU-s 326.71KiB/CPU-s 4.22KiB/CPU-s 0.0 0.050378 6.06MiB/CPU-s 343.36KiB/CPU-s 4.43KiB/CPU-s 0.0 0.055366 False False

vdev/src/testing/integration.rs Outdated Show resolved Hide resolved
@github-actions
Copy link

github-actions bot commented Apr 6, 2023

Regression Detector Results

Run ID: adf70898-abae-4ed0-bc75-54e666e16723
Baseline: 9cdee35
Comparison: 4b0260e
Total vector CPUs: 7

Explanation

A regression test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine quickly if vector performance is changed and to what degree by a pull request.

The table below, if present, lists those experiments that have experienced a statistically significant change in mean optimization goal performance between baseline and comparison SHAs with 90.00% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±5.00% change in their mean optimization goal are discarded. An experiment is erratic if its coefficient of variation is greater than 0.1. The abbreviated table will be omitted if no interesting change is observed.

No interesting changes in experiment optimization goals with confidence ≥ 90.00% and |Δ mean %| ≥ 5.00%.

Fine details of change detection per experiment.
experiment goal Δ mean Δ mean % confidence baseline mean baseline stdev baseline stderr baseline outlier % baseline CoV comparison mean comparison stdev comparison stderr comparison outlier % comparison CoV erratic declared erratic
http_text_to_http_json ingress throughput 847.92KiB/CPU-s 3.40 100.00% 24.36MiB/CPU-s 608.51KiB/CPU-s 7.85KiB/CPU-s 0.0 0.024388 25.19MiB/CPU-s 553.78KiB/CPU-s 7.15KiB/CPU-s 0.0 0.021465 False False
splunk_hec_route_s3 ingress throughput 85.61KiB/CPU-s 0.73 100.00% 11.51MiB/CPU-s 535.92KiB/CPU-s 6.92KiB/CPU-s 0.0 0.045464 11.59MiB/CPU-s 617.53KiB/CPU-s 7.97KiB/CPU-s 0.0 0.052008 False False
datadog_agent_remap_datadog_logs ingress throughput 234.0KiB/CPU-s 0.70 100.00% 32.74MiB/CPU-s 1.26MiB/CPU-s 16.63KiB/CPU-s 0.0 0.038444 32.97MiB/CPU-s 974.36KiB/CPU-s 12.58KiB/CPU-s 0.0 0.028861 False False
datadog_agent_remap_datadog_logs_acks ingress throughput 180.13KiB/CPU-s 0.55 100.00% 32.16MiB/CPU-s 1.29MiB/CPU-s 17.0KiB/CPU-s 0.0 0.039991 32.34MiB/CPU-s 1.06MiB/CPU-s 14.05KiB/CPU-s 0.0 0.032887 False False
syslog_loki ingress throughput 43.16KiB/CPU-s 0.50 100.00% 8.44MiB/CPU-s 297.08KiB/CPU-s 3.83KiB/CPU-s 0.0 0.034387 8.48MiB/CPU-s 270.71KiB/CPU-s 3.49KiB/CPU-s 0.0 0.03118 False False
enterprise_http_to_http ingress throughput 6.1KiB/CPU-s 0.04 88.80% 13.62MiB/CPU-s 255.38KiB/CPU-s 3.3KiB/CPU-s 0.0 0.018312 13.62MiB/CPU-s 152.74KiB/CPU-s 1.97KiB/CPU-s 0.0 0.010947 False False
splunk_hec_indexer_ack_blackhole ingress throughput 3.01KiB/CPU-s 0.02 48.27% 13.61MiB/CPU-s 262.52KiB/CPU-s 3.39KiB/CPU-s 0.0 0.018829 13.62MiB/CPU-s 246.38KiB/CPU-s 3.18KiB/CPU-s 0.0 0.017668 False False
fluent_elasticsearch ingress throughput -341.92B/CPU-s -0.00 45.67% 45.41MiB/CPU-s 30.24KiB/CPU-s 395.28B/CPU-s 0.0 0.00065 45.41MiB/CPU-s 30.62KiB/CPU-s 400.16B/CPU-s 0.0 0.000658 False False
splunk_hec_to_splunk_hec_logs_acks ingress throughput -131.94B/CPU-s -0.00 1.48% 13.61MiB/CPU-s 396.54KiB/CPU-s 5.12KiB/CPU-s 0.0 0.028442 13.61MiB/CPU-s 362.8KiB/CPU-s 4.68KiB/CPU-s 0.0 0.026022 False False
http_to_http_noack ingress throughput 117.32B/CPU-s 0.00 1.24% 13.6MiB/CPU-s 404.75KiB/CPU-s 5.22KiB/CPU-s 0.0 0.029052 13.6MiB/CPU-s 403.99KiB/CPU-s 5.21KiB/CPU-s 0.0 0.028998 False False
splunk_hec_to_splunk_hec_logs_noack ingress throughput -2.56KiB/CPU-s -0.02 42.20% 13.62MiB/CPU-s 243.77KiB/CPU-s 3.15KiB/CPU-s 0.0 0.017479 13.62MiB/CPU-s 260.65KiB/CPU-s 3.36KiB/CPU-s 0.0 0.018693 False False
http_to_http_json ingress throughput -22.07KiB/CPU-s -0.16 100.00% 13.62MiB/CPU-s 229.43KiB/CPU-s 2.96KiB/CPU-s 0.0 0.016448 13.6MiB/CPU-s 264.56KiB/CPU-s 3.41KiB/CPU-s 0.0 0.018997 False False
syslog_log2metric_splunk_hec_metrics ingress throughput -16.31KiB/CPU-s -0.17 99.89% 9.44MiB/CPU-s 269.1KiB/CPU-s 3.47KiB/CPU-s 0.0 0.027849 9.42MiB/CPU-s 277.22KiB/CPU-s 3.58KiB/CPU-s 0.0 0.028737 False False
http_to_http_acks ingress throughput -15.85KiB/CPU-s -0.29 24.21% 5.27MiB/CPU-s 2.75MiB/CPU-s 36.29KiB/CPU-s 0.0 0.521566 5.25MiB/CPU-s 2.76MiB/CPU-s 36.42KiB/CPU-s 0.0 0.524919 True False
file_to_blackhole egress throughput -23.95KiB/CPU-s -0.32 9.78% 7.23MiB/CPU-s 3.86MiB/CPU-s 135.07KiB/CPU-s 0.0 0.533733 7.21MiB/CPU-s 4.02MiB/CPU-s 140.46KiB/CPU-s 0.0 0.557125 True True
syslog_humio_logs ingress throughput -33.33KiB/CPU-s -0.36 100.00% 9.13MiB/CPU-s 295.43KiB/CPU-s 3.81KiB/CPU-s 0.0 0.031601 9.1MiB/CPU-s 347.38KiB/CPU-s 4.48KiB/CPU-s 0.0 0.037291 False False
datadog_agent_remap_blackhole ingress throughput -286.42KiB/CPU-s -0.92 100.00% 30.3MiB/CPU-s 1.25MiB/CPU-s 16.54KiB/CPU-s 0.0 0.041312 30.02MiB/CPU-s 1.22MiB/CPU-s 16.15KiB/CPU-s 0.0 0.040707 False False
otlp_grpc_to_blackhole ingress throughput -12.57KiB/CPU-s -1.16 100.00% 1.06MiB/CPU-s 47.8KiB/CPU-s 631.65B/CPU-s 0.0 0.044219 1.04MiB/CPU-s 40.31KiB/CPU-s 532.81B/CPU-s 0.0 0.037729 False False
socket_to_socket_blackhole ingress throughput -172.69KiB/CPU-s -1.25 100.00% 13.51MiB/CPU-s 275.99KiB/CPU-s 3.56KiB/CPU-s 0.0 0.019951 13.34MiB/CPU-s 410.23KiB/CPU-s 5.29KiB/CPU-s 0.0 0.03003 False False
datadog_agent_remap_blackhole_acks ingress throughput -408.87KiB/CPU-s -1.29 100.00% 30.98MiB/CPU-s 1.14MiB/CPU-s 15.06KiB/CPU-s 0.0 0.036778 30.58MiB/CPU-s 964.66KiB/CPU-s 12.45KiB/CPU-s 0.0 0.0308 False False
syslog_splunk_hec_logs ingress throughput -120.26KiB/CPU-s -1.32 100.00% 8.92MiB/CPU-s 264.49KiB/CPU-s 3.41KiB/CPU-s 0.0 0.028966 8.8MiB/CPU-s 343.6KiB/CPU-s 4.43KiB/CPU-s 0.0 0.038131 False False
syslog_regex_logs2metric_ddmetrics ingress throughput -50.58KiB/CPU-s -1.39 100.00% 3.56MiB/CPU-s 467.98KiB/CPU-s 6.04KiB/CPU-s 0.0 0.12848 3.51MiB/CPU-s 410.42KiB/CPU-s 5.3KiB/CPU-s 0.0 0.114265 True True
otlp_http_to_blackhole ingress throughput -39.23KiB/CPU-s -2.43 100.00% 1.58MiB/CPU-s 99.84KiB/CPU-s 1.29KiB/CPU-s 0.0 0.061794 1.54MiB/CPU-s 114.35KiB/CPU-s 1.48KiB/CPU-s 0.0 0.072536 False False
syslog_log2metric_humio_metrics ingress throughput -251.14KiB/CPU-s -4.02 100.00% 6.1MiB/CPU-s 300.11KiB/CPU-s 3.87KiB/CPU-s 0.0 0.048048 5.85MiB/CPU-s 334.63KiB/CPU-s 4.32KiB/CPU-s 0.0 0.05582 False False

pub struct ComposeService {
#[serde(default, skip_serializing_if = "Option::is_none")]
pub image: Option<String>,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was under the assumption that image was required, in what case can it be None?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This might have been an artifact of me being copy pasta happy with that line above and putting it over even fields that are required or guaranteed to be included.

Copy link
Contributor Author

@jonathanpv jonathanpv Apr 7, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah but in this case it was added because cargo vdev integration start dnstap would fail if making the image field a String instead of an option

scripts/integration/dnstap/compose.yaml

version: "3"

services:
  dnstap:
    build:
      context: ../../../tests/data/dnstap
    container_name: vector_dnstap
    hostname: ns.example.com
    volumes:
      - dnstap-sockets:/bind1/etc/bind/socket
      - dnstap-sockets:/bind2/etc/bind/socket
      - dnstap-sockets:/bind3/etc/bind/socket

volumes:
  dnstap-sockets: {}

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

noting that image isn't required when defining the test runner service, which builds from the local source.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But it is required to capture and recreate the image parameter already existing in the compose files.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah, yeah I was just adding a case example to Spencer's earlier question, could have quoted that

I was under the assumption that image was required, in what case can it be None?

@jonathanpv jonathanpv merged commit 5d88655 into master Apr 7, 2023
@jonathanpv jonathanpv deleted the jonathanpv/vdev-running-state branch April 7, 2023 19:43
aholmberg pushed a commit to aholmberg/vector that referenced this pull request Feb 14, 2024
# [1.13.0](answerbook/vector@v1.12.1...v1.13.0) (2023-09-13)

### Bug Fixes

* **appsignal sink**: Add TLS config option (vectordotdev#17122) [198068c](answerbook/vector@198068c) - GitHub
* **buffers**: correctly handle partial writes in reader seek during initialization (vectordotdev#17099) [a791595](answerbook/vector@a791595) - GitHub
* **config**: recurse through schema refs when determining eligibility for unevaluated properties (vectordotdev#17150) [71d1bf6](answerbook/vector@71d1bf6) - GitHub
* **docker_logs source**: Support tcp schema [e1c0c02](answerbook/vector@e1c0c02) - GitHub
* **elasticsearch sink**:  Elasticsearch sink with api_version set to "auto" does not recognize the API version of ES6 as V6 (vectordotdev#17226) (vectordotdev#17227) [9b6ef24](answerbook/vector@9b6ef24) - GitHub
* **gcp_stackdriver_metrics sink**: Call function to regenerate auth token (vectordotdev#17297) [bf7904b](answerbook/vector@bf7904b) - GitHub
* **influxdb_logs**: encode influx line when no tags present (vectordotdev#17029) [c3aa14f](answerbook/vector@c3aa14f) - GitHub
* **reduce transform**: Revert flushing on interval change to `expire_metrics_ms` (vectordotdev#17084) [e86b155](answerbook/vector@e86b155) - GitHub
* **releasing**: Fix globbing of release artifacts for GitHub (vectordotdev#17114) [7fe089c](answerbook/vector@7fe089c) - GitHub
* **schemas**: Dont panic with non object field kinds (vectordotdev#17140) [1e43208](answerbook/vector@1e43208) - GitHub

### Chores

* (syslog source): add source_ip to some syslog tests (vectordotdev#17235) [29c34c0](answerbook/vector@29c34c0) - GitHub
* add note to DEVELOPING.md re panics (vectordotdev#17277) [03e905e](answerbook/vector@03e905e) - GitHub
* Add UX note about encoding of log_schema keys (vectordotdev#17266) [dc6e54c](answerbook/vector@dc6e54c) - GitHub
* **administration**: add `appsignal` to codeowners (vectordotdev#17127) [7b15d19](answerbook/vector@7b15d19) - GitHub
* **buffer**: tidy up some of the module level docs for `disk_v2` (vectordotdev#17093) [edaa612](answerbook/vector@edaa612) - GitHub
* **ci**: bump docker/metadata-action from 4.3.0 to 4.4.0 (vectordotdev#17170) [854d71e](answerbook/vector@854d71e) - GitHub
* **ci**: Disable `appsignal` integration test until CA issues are resolved (vectordotdev#17109) [f3b5d42](answerbook/vector@f3b5d42) - GitHub
* **ci**: Disable scheduled runs of Baseline Timings workflow (vectordotdev#17281) [4335b0a](answerbook/vector@4335b0a) - GitHub
* **ci**: Fix event assertions for `aws_ec2_metadata` transform (vectordotdev#17413) [da36fb6](answerbook/vector@da36fb6) - GitHub
* **ci**: Increase timeout for integration tests (vectordotdev#17326) [e1f125a](answerbook/vector@e1f125a) - GitHub
* **ci**: Increase timeout for integration tests to 30m (vectordotdev#17350) [5d3f619](answerbook/vector@5d3f619) - GitHub
* **ci**: re-enable `appsignal` integration test (vectordotdev#17111) [48fc574](answerbook/vector@48fc574) - GitHub
* **ci**: Remove ci-sweep tasks (vectordotdev#17415) [5c33f99](answerbook/vector@5c33f99) - GitHub
* **ci**: remove unnecessary dep install (vectordotdev#17128) [f56d1ef](answerbook/vector@f56d1ef) - GitHub
* **ci**: Try to fix apt retries (vectordotdev#17393) [6b3db04](answerbook/vector@6b3db04) - GitHub
* **ci**: update unsupported ubuntu version runners (vectordotdev#17113) [e7c4815](answerbook/vector@e7c4815) - GitHub
* **ci**: use python v3.8 in ubuntu 20.04 runner (vectordotdev#17116) [7a40c81](answerbook/vector@7a40c81) - GitHub
* **config**: begin laying out primitives for programmatically querying schema (vectordotdev#17130) [aad8115](answerbook/vector@aad8115) - GitHub
* **config**: emit human-friendly version of enum variant/property names in schema (vectordotdev#17171) [3b38ba8](answerbook/vector@3b38ba8) - GitHub
* **config**: improve config schema output with more precise `unevaluatedProperties` + schema ref flattening (vectordotdev#17026) [2d72f82](answerbook/vector@2d72f82) - GitHub
* **deps**: Add 3rd party license file and CI checks (vectordotdev#17344) [7350e1a](answerbook/vector@7350e1a) - GitHub
* **deps**: bump anyhow from 1.0.70 to 1.0.71 (vectordotdev#17300) [6a5af3b](answerbook/vector@6a5af3b) - GitHub
* **deps**: bump assert_cmd from 2.0.10 to 2.0.11 (vectordotdev#17290) [c4784fd](answerbook/vector@c4784fd) - GitHub
* **deps**: bump async-compression from 0.3.15 to 0.4.0 (vectordotdev#17365) [b9aac47](answerbook/vector@b9aac47) - GitHub
* **deps**: bump async-graphql from 5.0.7 to 5.0.8 (vectordotdev#17357) [05a4f17](answerbook/vector@05a4f17) - GitHub
* **deps**: bump async-graphql-warp from 5.0.7 to 5.0.8 (vectordotdev#17367) [693584e](answerbook/vector@693584e) - GitHub
* **deps**: bump async-stream from 0.3.4 to 0.3.5 (vectordotdev#17076) [c29c817](answerbook/vector@c29c817) - GitHub
* **deps**: bump aws-sigv4 from 0.55.0 to 0.55.1 (vectordotdev#17138) [dbb3f25](answerbook/vector@dbb3f25) - GitHub
* **deps**: bump axum from 0.6.12 to 0.6.18 (vectordotdev#17257) [41ac76e](answerbook/vector@41ac76e) - GitHub
* **deps**: bump cached from 0.42.0 to 0.43.0 (vectordotdev#17118) [f90b3b3](answerbook/vector@f90b3b3) - GitHub
* **deps**: bump chrono-tz from 0.8.1 to 0.8.2 (vectordotdev#17088) [623b838](answerbook/vector@623b838) - GitHub
* **deps**: bump clap_complete from 4.2.0 to 4.2.1 (vectordotdev#17229) [d286d16](answerbook/vector@d286d16) - GitHub
* **deps**: bump clap_complete from 4.2.1 to 4.2.2 (vectordotdev#17359) [565668e](answerbook/vector@565668e) - GitHub
* **deps**: bump clap_complete from 4.2.2 to 4.2.3 (vectordotdev#17383) [111cd07](answerbook/vector@111cd07) - GitHub
* **deps**: bump console-subscriber from 0.1.8 to 0.1.9 (vectordotdev#17358) [97b862c](answerbook/vector@97b862c) - GitHub
* **deps**: bump directories from 5.0.0 to 5.0.1 (vectordotdev#17271) [be69f5f](answerbook/vector@be69f5f) - GitHub
* **deps**: bump dunce from 1.0.3 to 1.0.4 (vectordotdev#17244) [cfc387d](answerbook/vector@cfc387d) - GitHub
* **deps**: bump enumflags2 from 0.7.5 to 0.7.6 (vectordotdev#17079) [cbc17be](answerbook/vector@cbc17be) - GitHub
* **deps**: bump enumflags2 from 0.7.6 to 0.7.7 (vectordotdev#17206) [c80c5eb](answerbook/vector@c80c5eb) - GitHub
* **deps**: bump flate2 from 1.0.25 to 1.0.26 (vectordotdev#17320) [ef13370](answerbook/vector@ef13370) - GitHub
* **deps**: bump getrandom from 0.2.8 to 0.2.9 (vectordotdev#17101) [d53240b](answerbook/vector@d53240b) - GitHub
* **deps**: bump h2 from 0.3.18 to 0.3.19 (vectordotdev#17388) [6088abd](answerbook/vector@6088abd) - GitHub
* **deps**: bump hashlink from 0.8.1 to 0.8.2 (vectordotdev#17419) [01b3cd7](answerbook/vector@01b3cd7) - GitHub
* **deps**: bump hyper from 0.14.25 to 0.14.26 (vectordotdev#17347) [c43dcfd](answerbook/vector@c43dcfd) - GitHub
* **deps**: bump inventory from 0.3.5 to 0.3.6 (vectordotdev#17401) [5b5ad16](answerbook/vector@5b5ad16) - GitHub
* **deps**: bump libc from 0.2.140 to 0.2.141 (vectordotdev#17104) [dd9608a](answerbook/vector@dd9608a) - GitHub
* **deps**: bump libc from 0.2.141 to 0.2.142 (vectordotdev#17273) [bc618a2](answerbook/vector@bc618a2) - GitHub
* **deps**: bump libc from 0.2.142 to 0.2.143 (vectordotdev#17338) [6afe206](answerbook/vector@6afe206) - GitHub
* **deps**: bump libc from 0.2.143 to 0.2.144 (vectordotdev#17346) [99b8dc1](answerbook/vector@99b8dc1) - GitHub
* **deps**: bump memmap2 from 0.5.10 to 0.6.0 (vectordotdev#17355) [dae0c6a](answerbook/vector@dae0c6a) - GitHub
* **deps**: bump memmap2 from 0.6.0 to 0.6.1 (vectordotdev#17364) [58ba741](answerbook/vector@58ba741) - GitHub
* **deps**: bump metrics, metrics-tracing-context, metrics-util (vectordotdev#17336) [9a723e3](answerbook/vector@9a723e3) - GitHub
* **deps**: bump mlua from 0.8.8 to 0.8.9 (vectordotdev#17423) [57f8bd4](answerbook/vector@57f8bd4) - GitHub
* **deps**: bump mock_instant from 0.2.1 to 0.3.0 (vectordotdev#17210) [40c9afc](answerbook/vector@40c9afc) - GitHub
* **deps**: bump mongodb from 2.4.0 to 2.5.0 (vectordotdev#17337) [64f4f69](answerbook/vector@64f4f69) - GitHub
* **deps**: bump nkeys from 0.2.0 to 0.3.0 (vectordotdev#17421) [3320eda](answerbook/vector@3320eda) - GitHub
* **deps**: bump notify from 5.1.0 to 6.0.0 (vectordotdev#17422) [58603b9](answerbook/vector@58603b9) - GitHub
* **deps**: bump num_enum from 0.5.11 to 0.6.0 (vectordotdev#17106) [42f298b](answerbook/vector@42f298b) - GitHub
* **deps**: bump num_enum from 0.6.0 to 0.6.1 (vectordotdev#17272) [f696e7b](answerbook/vector@f696e7b) - GitHub
* **deps**: bump opendal from 0.30.5 to 0.31.0 (vectordotdev#17119) [8762563](answerbook/vector@8762563) - GitHub
* **deps**: bump opendal from 0.31.0 to 0.33.2 (vectordotdev#17286) [3d41931](answerbook/vector@3d41931) - GitHub
* **deps**: bump opendal from 0.33.2 to 0.34.0 (vectordotdev#17354) [ae602da](answerbook/vector@ae602da) - GitHub
* **deps**: bump openssl from 0.10.48 to 0.10.50 (vectordotdev#17087) [9a56ed8](answerbook/vector@9a56ed8) - GitHub
* **deps**: bump openssl from 0.10.50 to 0.10.52 (vectordotdev#17299) [0ecceb3](answerbook/vector@0ecceb3) - GitHub
* **deps**: bump pin-project from 1.0.12 to 1.1.0 (vectordotdev#17385) [e8d3002](answerbook/vector@e8d3002) - GitHub
* **deps**: bump prettydiff from 0.6.2 to 0.6.4 (vectordotdev#17089) [e090610](answerbook/vector@e090610) - GitHub
* **deps**: bump prettydiff from 0.6.2 to 0.6.4 (vectordotdev#17315) [a1ec68d](answerbook/vector@a1ec68d) - GitHub
* **deps**: bump proc-macro2 from 1.0.55 to 1.0.56 (vectordotdev#17103) [6f74523](answerbook/vector@6f74523) - GitHub
* **deps**: bump proc-macro2 from 1.0.56 to 1.0.57 (vectordotdev#17400) [a6e1ae7](answerbook/vector@a6e1ae7) - GitHub
* **deps**: bump prost-build from 0.11.8 to 0.11.9 (vectordotdev#17260) [a88aba4](answerbook/vector@a88aba4) - GitHub
* **deps**: bump quote from 1.0.26 to 1.0.27 (vectordotdev#17348) [f81ff18](answerbook/vector@f81ff18) - GitHub
* **deps**: bump rdkafka from 0.29.0 to 0.30.0 (vectordotdev#17387) [9703188](answerbook/vector@9703188) - GitHub
* **deps**: bump regex from 1.7.3 to 1.8.1 (vectordotdev#17222) [410aa3c](answerbook/vector@410aa3c) - GitHub
* **deps**: bump reqwest from 0.11.16 to 0.11.17 (vectordotdev#17316) [09176ec](answerbook/vector@09176ec) - GitHub
* **deps**: bump security-framework from 2.8.2 to 2.9.0 (vectordotdev#17386) [1287168](answerbook/vector@1287168) - GitHub
* **deps**: bump serde from 1.0.159 to 1.0.160 (vectordotdev#17270) [036ad4a](answerbook/vector@036ad4a) - GitHub
* **deps**: bump serde from 1.0.160 to 1.0.162 (vectordotdev#17317) [79e97a2](answerbook/vector@79e97a2) - GitHub
* **deps**: bump serde from 1.0.162 to 1.0.163 (vectordotdev#17366) [9852c17](answerbook/vector@9852c17) - GitHub
* **deps**: bump serde_json from 1.0.95 to 1.0.96 (vectordotdev#17258) [7570bb3](answerbook/vector@7570bb3) - GitHub
* **deps**: bump serde_with from 2.3.1 to 2.3.2 (vectordotdev#17090) [adbf4d5](answerbook/vector@adbf4d5) - GitHub
* **deps**: bump serde_yaml from 0.9.19 to 0.9.21 (vectordotdev#17120) [d6f2625](answerbook/vector@d6f2625) - GitHub
* **deps**: bump socket2 from 0.4.7 to 0.5.2 (vectordotdev#17121) [db39d83](answerbook/vector@db39d83) - GitHub
* **deps**: bump socket2 from 0.5.2 to 0.5.3 (vectordotdev#17384) [ac51b8a](answerbook/vector@ac51b8a) - GitHub
* **deps**: bump syslog from 6.0.1 to 6.1.0 (vectordotdev#17301) [61e6154](answerbook/vector@61e6154) - GitHub
* **deps**: bump tokio from 1.27.0 to 1.28.0 (vectordotdev#17231) [8067f84](answerbook/vector@8067f84) - GitHub
* **deps**: bump tokio from 1.28.0 to 1.28.1 (vectordotdev#17368) [ae6a51b](answerbook/vector@ae6a51b) - GitHub
* **deps**: bump tokio-stream from 0.1.12 to 0.1.14 (vectordotdev#17339) [80c8247](answerbook/vector@80c8247) - GitHub
* **deps**: bump tokio-tungstenite from 0.18.0 to 0.19.0 (vectordotdev#17404) [ae1dd6e](answerbook/vector@ae1dd6e) - GitHub
* **deps**: bump tonic from 0.8.3 to 0.9.1 (vectordotdev#17077) [eafba69](answerbook/vector@eafba69) - GitHub
* **deps**: bump tonic from 0.9.1 to 0.9.2 (vectordotdev#17221) [aa9cbd0](answerbook/vector@aa9cbd0) - GitHub
* **deps**: bump tonic-build from 0.8.4 to 0.9.2 (vectordotdev#17274) [e0a07c6](answerbook/vector@e0a07c6) - GitHub
* **deps**: bump tracing-subscriber from 0.3.16 to 0.3.17 (vectordotdev#17268) [1406c08](answerbook/vector@1406c08) - GitHub
* **deps**: bump typetag from 0.2.7 to 0.2.8 (vectordotdev#17302) [c8e0e5f](answerbook/vector@c8e0e5f) - GitHub
* **deps**: bump uuid from 1.3.0 to 1.3.1 (vectordotdev#17091) [9cc2f1d](answerbook/vector@9cc2f1d) - GitHub
* **deps**: bump uuid from 1.3.0 to 1.3.2 (vectordotdev#17256) [bc6f7fd](answerbook/vector@bc6f7fd) - GitHub
* **deps**: bump uuid from 1.3.2 to 1.3.3 (vectordotdev#17403) [3a3fe63](answerbook/vector@3a3fe63) - GitHub
* **deps**: bump warp from 0.3.4 to 0.3.5 (vectordotdev#17288) [d8c1f12](answerbook/vector@d8c1f12) - GitHub
* **deps**: bump wasm-bindgen from 0.2.84 to 0.2.85 (vectordotdev#17356) [ea24b4d](answerbook/vector@ea24b4d) - GitHub
* **deps**: bump wasm-bindgen from 0.2.85 to 0.2.86 (vectordotdev#17402) [0518176](answerbook/vector@0518176) - GitHub
* **deps**: bump wiremock from 0.5.17 to 0.5.18 (vectordotdev#17092) [51312aa](answerbook/vector@51312aa) - GitHub
* **deps**: Fix up missing license (vectordotdev#17379) [a2b8903](answerbook/vector@a2b8903) - GitHub
* **deps**: Reset dependencies bumped by a61dea1 (vectordotdev#17100) [887d6d7](answerbook/vector@887d6d7) - GitHub
* **deps**: true up cargo.lock (vectordotdev#17149) [10fce65](answerbook/vector@10fce65) - GitHub
* **deps**: Update h2 (vectordotdev#17189) [a2882f3](answerbook/vector@a2882f3) - GitHub
* **deps**: Upgrade cue to 0.5.0 (vectordotdev#17204) [d396320](answerbook/vector@d396320) - GitHub
* **deps**: Upgrade Debian to bullseye for distroless image (vectordotdev#17160) [c304a8c](answerbook/vector@c304a8c) - GitHub
* **deps**: Upgrade rust to 1.69.0 (vectordotdev#17194) [ef15696](answerbook/vector@ef15696) - GitHub
* **dev**: Add note about generating licenses to CONTRIBUTING.md (vectordotdev#17410) [539f379](answerbook/vector@539f379) - GitHub
* **dev**: ignore `.helix` dir (vectordotdev#17203) [32a935b](answerbook/vector@32a935b) - GitHub
* **dev**: Install the correct `mold` based on CPU architecture (vectordotdev#17248) [4b80c71](answerbook/vector@4b80c71) - GitHub
* **dev**: remove editors from gitignore (vectordotdev#17267) [61c0d76](answerbook/vector@61c0d76) - GitHub
* **docs**: Add Enterprise link and update Support link (vectordotdev#17408) [5184d50](answerbook/vector@5184d50) - GitHub
* **docs**: Add missing 0.28.2 version [38607cd](answerbook/vector@38607cd) - Jesse Szwedko
* **docs**: Clarify `key_field` for `sample` and `throttle` transforms (vectordotdev#17372) [d1e5588](answerbook/vector@d1e5588) - GitHub
* **docs**: Document event type conditions (vectordotdev#17311) [a9c8dc8](answerbook/vector@a9c8dc8) - GitHub
* **docs**: make doc style edits (vectordotdev#17155) [65a8856](answerbook/vector@65a8856) - GitHub
* **docs**: Remove trailing, unmatched quote (vectordotdev#17163) [3c92556](answerbook/vector@3c92556) - GitHub
* **docs**: Remove unneeded `yaml` dependency from website (vectordotdev#17215) [752d424](answerbook/vector@752d424) - GitHub
* **docs**: Update component statuses 2023Q2 (vectordotdev#17362) [22cda94](answerbook/vector@22cda94) - GitHub
* **docs**: update the `v0.28.0` upgrade guide with note about `datadog_logs` sink `hostname` key (vectordotdev#17156) [c169131](answerbook/vector@c169131) - GitHub
* **external docs**: correctly mark some sinks as stateful (vectordotdev#17085) [64d560d](answerbook/vector@64d560d) - GitHub
* **loki sink**: warn on label expansions and collisions (vectordotdev#17052) [f06692b](answerbook/vector@f06692b) - GitHub
* **pulsar**: pulsar-rs bump to v5.1.1 (vectordotdev#17159) [68b54a9](answerbook/vector@68b54a9) - GitHub
* Re-add transform definitions (vectordotdev#17152) [9031d0f](answerbook/vector@9031d0f) - GitHub
* Regen docs for sample and throttle (vectordotdev#17390) [6c57ca0](answerbook/vector@6c57ca0) - GitHub
* **releasing**: Add known issues fixed by 0.29.1 (vectordotdev#17218) [40d543a](answerbook/vector@40d543a) - GitHub
* **releasing**: Bump Vector version to 0.30.0 (vectordotdev#17134) [3834612](answerbook/vector@3834612) - GitHub
* **releasing**: Fix homebrew release script (vectordotdev#17131) [cfbf233](answerbook/vector@cfbf233) - Jesse Szwedko
* **releasing**: Fix release channels (vectordotdev#17133) [58b44e8](answerbook/vector@58b44e8) - Jesse Szwedko
* **releasing**: Prepare v0.28.2 release [a61dea1](answerbook/vector@a61dea1) - Jesse Szwedko
* **releasing**: Prepare v0.29.0 release [4bf6805](answerbook/vector@4bf6805) - Jesse Szwedko
* **releasing**: Prepare v0.30.0 release [38c3f0b](answerbook/vector@38c3f0b) - Jesse Szwedko
* **releasing**: Regenerate Kubernetes manifests for 0.21.2 (vectordotdev#17108) [fd13d64](answerbook/vector@fd13d64) - GitHub
* **releasing**: Regenerate manifests for 0.21.1 chart (vectordotdev#17187) [1f0de6b](answerbook/vector@1f0de6b) - GitHub
* **releasing**: Regenerate manifests for 0.22.0 chart (vectordotdev#17135) [e7ea0a8](answerbook/vector@e7ea0a8) - GitHub
* **releasing**: update patch release template with extra step details [27c3526](answerbook/vector@27c3526) - GitHub
* Remove skaffold from project (vectordotdev#17145) [d245927](answerbook/vector@d245927) - GitHub
* remove transform type coercion (vectordotdev#17411) [b6c7e0a](answerbook/vector@b6c7e0a) - GitHub
* Revert transform definitions (vectordotdev#17146) [05a3f44](answerbook/vector@05a3f44) - GitHub
* **socket source**: Remove deprecated `max_length` setting from `tcp` and `unix` modes. (vectordotdev#17162) [9ecfc8c](answerbook/vector@9ecfc8c) - GitHub
* **syslog source**: remove the remove of source_ip (vectordotdev#17184) [5dff0ed](answerbook/vector@5dff0ed) - GitHub
* **topology**: Transform outputs hash table of OutputId -> Definition (vectordotdev#17059) [1bdb24d](answerbook/vector@1bdb24d) - GitHub
* Upgrade `VRL` to `0.3.0` (vectordotdev#17325) [4911d36](answerbook/vector@4911d36) - GitHub

### Features

* adds 'metric_name' field to internal logs for the tag_cardinality_limit component (vectordotdev#17295) [4317340](answerbook/vector@4317340) - GitHub
* **codecs**: Add full codec support to AWS S3 source/sink (vectordotdev#17098) [d648c86](answerbook/vector@d648c86) - GitHub
* **kubernetes_logs**: use kube-apiserver cache for list requests (vectordotdev#17095) [e46fae7](answerbook/vector@e46fae7) - GitHub
* Merge with upstream v0.30.0 [6b93177](answerbook/vector@6b93177) - GitHub [LOG-17997](https://logdna.atlassian.net/browse/LOG-17997)
* **new sink**: Initial `datadog_events` sink (vectordotdev#7678) [fef3810](answerbook/vector@fef3810) - Jesse Szwedko
* Update VRL library [6ace1e6](answerbook/vector@6ace1e6) - Jorge Bay

### Miscellaneous

* Merge branch 'master' [d4b35bb](answerbook/vector@d4b35bb) - Jorge Bay
* Merge tag 'v0.30.0' into update-upstream [ee2f300](answerbook/vector@ee2f300) - Jorge Bay
* Merge commit '9031d0faa2811187874364e1b5a3305c9ed0c0da' into update-upstream [c12faae](answerbook/vector@c12faae) - Jorge Bay
* Improve tokio::select behavior for kafka source and finalizers (vectordotdev#17380) [d4df21c](answerbook/vector@d4df21c) - GitHub
* Prepare v0.29.1 release [21beed7](answerbook/vector@21beed7) - Kyle Criddle
* Add a quickfix to handle special capitalization cases (vectordotdev#17141) [ba63e21](answerbook/vector@ba63e21) - GitHub
* Adjust doc comment locations (vectordotdev#17154) [730c938](answerbook/vector@730c938) - GitHub
* **amqp sink**: Support AMQ Properties (content-type) in AMQP sink (vectordotdev#17174) [c10d30b](answerbook/vector@c10d30b) - GitHub
* **aws provider**: Let `region` be configured for default authentication (vectordotdev#17414) [c81ad30](answerbook/vector@c81ad30) - GitHub
* **core**: add unit test (ignored) for support for encoding special chars in `ProxyConfig` (vectordotdev#17148) [5247972](answerbook/vector@5247972) - GitHub
* **loki sink**: Fix formatting in labels example (vectordotdev#17396) [f3734e8](answerbook/vector@f3734e8) - GitHub
* **observability**: Log underlying error for unhandled HTTP errors (vectordotdev#17327) [bf8376c](answerbook/vector@bf8376c) - GitHub
* **observability**: Update internal log rate limiting messages (vectordotdev#17394) [1951535](answerbook/vector@1951535) - GitHub
* **pulsar sink**: Refactor to use StreamSink (vectordotdev#14345) [1e97a2f](answerbook/vector@1e97a2f) - GitHub
* **topology**: Add source id to metadata (vectordotdev#17369) [c683999](answerbook/vector@c683999) - GitHub
* **vdev**: Load compose files and inject network block (vectordotdev#17025) [5d88655](answerbook/vector@5d88655) - GitHub
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ci-condition: integration tests enable Run integration tests on this PR domain: ci Anything related to Vector's CI environment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants