Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

enhancement(pulsar sink): Refactor to use StreamSink #14345

Merged
merged 58 commits into from
Apr 11, 2023

Conversation

addisonj
Copy link
Contributor

@addisonj addisonj commented Sep 9, 2022

This commit heavily refactors the Pulsar Sink to use the StreamSink interface and is modeled after the Kafka Sink.

It also adds additional features that bring it in line with Kafka Sink feature set.

This includes:

  • Refactoring to use StreamSink instead of Sink interace. See Rewrite sinks into the "new-style" #9261
  • Supports dynamic topics using a topic template
  • Refactor configurations in advance of adding Pulsar source
  • Rework message parsing to support logs and metrics, with support for dynamic keys and properties

This work is heavily modeled after Kafka sink. This means there has been some duplication of some utility code. However, it has not been refactored to remove the duplication as there wasn't a clear pattern of where such shared code should be put.

Additionally, this refactor seems to be much simpler by using StreamSink but does require some workarounds limitations in the Pulsar client library by wrapping certain resources in Arc that may have performance implications. I am not famaliar enough to know if there might be some efficiencies by structuring this differently.

Remaining work:

  • Add a few more tests

This commit heavily refactors the Pulsar Sink to use the StreamSink
interface and is modeled after the Kafka Sink.

It also adds additional features that bring it in line with
Kafka Sink feature set.

This includes:
* Refactoring to use StreamSink instead of Sink interace. See vectordotdev#9261
* Supports dynamic topics using a topic template
* Refactor configurations in advance of adding Pulsar source
* Rework message parsing to support logs and metrics, with support for
  dynamic keys and properties

This work is heavily modeled after Kafka sink. This means there has been
some duplication of some utility code. However, it has not been
refactored to remove the duplication as there wasn't a clear pattern of
where such shared code should be put.

Additionally, this refactor seems to be much simpler by using StreamSink
but does require some workarounds limitations in the Pulsar client
library by wrapping certain resources in Arc<Mutex> that *may* have
performance implications. I am not famaliar enough to know if there
might be some efficiencies by structuring this differently.

Remaining work:
* Add a few more tests
@bits-bot
Copy link

bits-bot commented Sep 9, 2022

CLA assistant check
All committers have signed the CLA.

@netlify
Copy link

netlify bot commented Sep 9, 2022

Deploy Preview for vector-project canceled.

Name Link
🔨 Latest commit f1a45e8
🔍 Latest deploy log https://app.netlify.com/sites/vector-project/deploys/6435b29c2c97df0008e67bf6

@github-actions github-actions bot added the domain: sinks Anything related to the Vector's sinks label Sep 9, 2022
@github-actions
Copy link

github-actions bot commented Sep 9, 2022

Soak Test Results

Baseline: 3c8f8de
Comparison: 08688d9
Total Vector CPUs: 4

Explanation

A soak test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their throughput performance between baseline and comparision SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±8.87% change in mean throughput are discarded. An experiment is erratic if its coefficient of variation is greater than 0.3. The abbreviated table will be omitted if no interesting changes are observed.

No interesting changes in throughput with confidence ≥ 90.00% and absolute Δ mean >= ±8.87%:

Fine details of change detection per experiment.
experiment Δ mean Δ mean % confidence baseline mean baseline stdev baseline stderr baseline outlier % baseline CoV comparison mean comparison stdev comparison stderr comparison outlier % comparison CoV erratic declared erratic
socket_to_socket_blackhole 615.23KiB 2.6 100.00% 23.11MiB 206.25KiB 4.21KiB 0 0.00871315 23.71MiB 219.34KiB 4.48KiB 0 0.00903127 False False
syslog_log2metric_splunk_hec_metrics 289.78KiB 1.59 100.00% 17.78MiB 930.74KiB 18.98KiB 0 0.0511022 18.07MiB 683.3KiB 13.92KiB 0 0.0369288 False False
datadog_agent_remap_blackhole 439.47KiB 0.71 100.00% 60.12MiB 3.88MiB 80.81KiB 0 0.0644819 60.55MiB 3.28MiB 68.39KiB 0 0.0541524 False False
datadog_agent_remap_datadog_logs_acks 415.89KiB 0.66 99.98% 61.21MiB 3.22MiB 67.33KiB 0 0.0526297 61.62MiB 4.37MiB 90.86KiB 0 0.070824 False False
datadog_agent_remap_blackhole_acks 398.22KiB 0.63 99.99% 61.36MiB 4.2MiB 87.41KiB 0 0.0684081 61.75MiB 2.26MiB 47.22KiB 0 0.036523 False False
syslog_log2metric_humio_metrics 70.89KiB 0.55 100.00% 12.67MiB 334.35KiB 6.83KiB 0 0.0257604 12.74MiB 491.15KiB 10.0KiB 0 0.0376363 False False
datadog_agent_remap_datadog_logs 267.98KiB 0.43 99.74% 61.48MiB 268.87KiB 5.51KiB 0 0.00426958 61.75MiB 4.25MiB 88.57KiB 0 0.0688787 False False
syslog_humio_logs 53.47KiB 0.32 99.99% 16.18MiB 488.48KiB 9.98KiB 0 0.0294776 16.23MiB 447.39KiB 9.16KiB 0 0.0269108 False False
http_pipelines_blackhole_acks 3.53KiB 0.28 82.20% 1.23MiB 112.94KiB 2.3KiB 0 0.0893742 1.24MiB 61.4KiB 1.25KiB 0 0.0484527 False False
syslog_regex_logs2metric_ddmetrics 34.32KiB 0.27 96.46% 12.59MiB 578.91KiB 11.79KiB 0 0.0448842 12.63MiB 552.37KiB 11.26KiB 0 0.0427128 False False
splunk_hec_to_splunk_hec_logs_noack 24.58KiB 0.1 94.81% 23.81MiB 521.15KiB 10.63KiB 0 0.0213666 23.84MiB 335.13KiB 6.84KiB 0 0.0137262 False False
splunk_hec_indexer_ack_blackhole 19.06KiB 0.08 51.76% 23.73MiB 979.16KiB 19.91KiB 0 0.0402804 23.75MiB 905.83KiB 18.43KiB 0 0.0372346 False False
syslog_splunk_hec_logs 3.88KiB 0.02 16.67% 16.28MiB 674.46KiB 13.74KiB 0 0.0404382 16.29MiB 603.39KiB 12.3KiB 0 0.0361687 False False
enterprise_http_to_http -1.37KiB -0.01 14.93% 23.85MiB 250.66KiB 5.12KiB 0 0.0102629 23.85MiB 251.48KiB 5.14KiB 0 0.0102968 False False
splunk_hec_to_splunk_hec_logs_acks -5.03KiB -0.02 15.73% 23.75MiB 866.63KiB 17.62KiB 0 0.0356258 23.75MiB 894.91KiB 18.2KiB 0 0.0367958 False False
file_to_blackhole -36.26KiB -0.04 26.12% 95.33MiB 3.59MiB 74.48KiB 0 0.0376796 95.3MiB 3.81MiB 79.21KiB 0 0.0399382 False False
http_to_http_json -21.62KiB -0.09 94.57% 23.85MiB 329.79KiB 6.73KiB 0 0.0135014 23.83MiB 439.27KiB 8.99KiB 0 0.0179996 False False
splunk_hec_route_s3 -60.11KiB -0.31 64.39% 18.73MiB 2.23MiB 46.37KiB 0 0.118826 18.67MiB 2.19MiB 45.74KiB 0 0.117204 False False
http_to_http_noack -96.96KiB -0.4 100.00% 23.84MiB 277.87KiB 5.69KiB 0 0.0113785 23.75MiB 1.05MiB 21.96KiB 0 0.044347 False False
fluent_elasticsearch -395.66KiB -0.49 100.00% 79.47MiB 52.53KiB 1.06KiB 0 0.000645394 79.09MiB 4.52MiB 92.95KiB 0 0.0571084 False False
http_to_http_acks -117.83KiB -0.66 38.21% 17.41MiB 7.83MiB 163.69KiB 0 0.449464 17.3MiB 8.16MiB 170.24KiB 0 0.471907 True True
http_pipelines_blackhole -20.25KiB -1.16 100.00% 1.7MiB 42.2KiB 882.75B 0 0.0242323 1.68MiB 97.19KiB 1.98KiB 0 0.0564691 False False
syslog_loki -248.0KiB -1.66 100.00% 14.63MiB 341.74KiB 6.99KiB 0 0.0228041 14.39MiB 710.11KiB 14.44KiB 0 0.048183 False False
http_text_to_http_json -801.81KiB -2.06 100.00% 37.98MiB 1.12MiB 23.32KiB 0 0.0293724 37.2MiB 1.05MiB 22.02KiB 0 0.0283029 False False
http_pipelines_no_grok_blackhole -281.29KiB -2.51 100.00% 10.96MiB 34.8KiB 727.95B 0 0.00310119 10.68MiB 1.09MiB 22.65KiB 0 0.101861 False False

Copy link
Contributor

@spencergilbert spencergilbert left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Partial feedback, mostly on the configurable. Will need to spend a chunk of time reviewing the bulk of the changes.

src/internal_events/pulsar.rs Outdated Show resolved Hide resolved
src/sinks/pulsar/config.rs Outdated Show resolved Hide resolved
src/sinks/pulsar/config.rs Outdated Show resolved Hide resolved
src/sinks/pulsar/config.rs Outdated Show resolved Hide resolved
src/sinks/pulsar/config.rs Outdated Show resolved Hide resolved
src/sinks/pulsar/config.rs Outdated Show resolved Hide resolved
src/sinks/pulsar/config.rs Show resolved Hide resolved
src/sinks/pulsar/config.rs Outdated Show resolved Hide resolved
addisonj and others added 6 commits September 12, 2022 10:45
Co-authored-by: Spencer Gilbert <Spencer.Gilbert@gmail.com>
Co-authored-by: Spencer Gilbert <Spencer.Gilbert@gmail.com>
Co-authored-by: Spencer Gilbert <Spencer.Gilbert@gmail.com>
Co-authored-by: Spencer Gilbert <Spencer.Gilbert@gmail.com>
Co-authored-by: Spencer Gilbert <Spencer.Gilbert@gmail.com>
Co-authored-by: Spencer Gilbert <Spencer.Gilbert@gmail.com>
@fuchsnj fuchsnj self-requested a review September 13, 2022 17:09
@github-actions
Copy link

Soak Test Results

Baseline: 28113af
Comparison: 63a7965
Total Vector CPUs: 4

Explanation

A soak test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their throughput performance between baseline and comparision SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±8.87% change in mean throughput are discarded. An experiment is erratic if its coefficient of variation is greater than 0.3. The abbreviated table will be omitted if no interesting changes are observed.

No interesting changes in throughput with confidence ≥ 90.00% and absolute Δ mean >= ±8.87%:

Fine details of change detection per experiment.
experiment Δ mean Δ mean % confidence baseline mean baseline stdev baseline stderr baseline outlier % baseline CoV comparison mean comparison stdev comparison stderr comparison outlier % comparison CoV erratic declared erratic
socket_to_socket_blackhole 1.08MiB 4.66 100.00% 23.09MiB 166.86KiB 3.41KiB 0 0.00705623 24.16MiB 163.89KiB 3.35KiB 0 0.00662204 False False
datadog_agent_remap_blackhole 2.01MiB 3.57 100.00% 56.45MiB 4.46MiB 93.02KiB 0 0.0790063 58.46MiB 3.81MiB 79.42KiB 0 0.065102 False False
datadog_agent_remap_datadog_logs_acks 2.05MiB 3.57 100.00% 57.32MiB 4.38MiB 91.58KiB 0 0.0763965 59.36MiB 5.23MiB 108.79KiB 0 0.088019 False False
syslog_log2metric_splunk_hec_metrics 508.05KiB 2.89 100.00% 17.14MiB 1.06MiB 22.21KiB 0 0.0620584 17.64MiB 1.06MiB 22.17KiB 0 0.0602797 False False
splunk_hec_route_s3 344.77KiB 1.97 100.00% 17.12MiB 2.69MiB 55.98KiB 0 0.157009 17.45MiB 2.66MiB 55.62KiB 0 0.152224 False False
syslog_splunk_hec_logs 261.43KiB 1.62 100.00% 15.81MiB 777.51KiB 15.83KiB 0 0.048027 16.06MiB 822.71KiB 16.77KiB 0 0.0500111 False False
syslog_humio_logs 264.73KiB 1.56 100.00% 16.62MiB 109.79KiB 2.24KiB 0 0.00644762 16.88MiB 126.59KiB 2.59KiB 0 0.00732072 False False
syslog_regex_logs2metric_ddmetrics 171.1KiB 1.36 100.00% 12.28MiB 505.41KiB 10.3KiB 0 0.0401824 12.45MiB 531.74KiB 10.84KiB 0 0.0417085 False False
http_text_to_http_json 491.28KiB 1.26 100.00% 37.93MiB 840.94KiB 17.17KiB 0 0.021644 38.41MiB 828.4KiB 16.92KiB 0 0.021055 False False
datadog_agent_remap_datadog_logs 587.95KiB 1 99.94% 57.24MiB 5.41MiB 113.47KiB 0 0.0945792 57.81MiB 6.11MiB 127.13KiB 0 0.1056 False False
datadog_agent_remap_blackhole_acks 561.11KiB 0.92 99.97% 59.78MiB 5.54MiB 115.49KiB 0 0.0927283 60.33MiB 4.87MiB 101.97KiB 0 0.0807395 False False
http_to_http_acks 50.99KiB 0.28 16.91% 17.51MiB 8.15MiB 170.41KiB 0 0.465359 17.56MiB 8.01MiB 167.21KiB 0 0.456182 True True
syslog_log2metric_humio_metrics 25.44KiB 0.2 72.87% 12.18MiB 711.45KiB 14.52KiB 0 0.0570408 12.2MiB 884.3KiB 18.0KiB 0 0.0707549 False False
syslog_loki 19.22KiB 0.13 73.29% 14.55MiB 414.55KiB 8.49KiB 0 0.0278202 14.57MiB 742.7KiB 15.1KiB 0 0.0497774 False False
splunk_hec_to_splunk_hec_logs_noack 10.86KiB 0.04 66.79% 23.83MiB 434.83KiB 8.88KiB 0 0.0178167 23.84MiB 334.47KiB 6.83KiB 0 0.0136985 False False
splunk_hec_to_splunk_hec_logs_acks 5.8KiB 0.02 18.67% 23.75MiB 866.61KiB 17.63KiB 0 0.0356253 23.76MiB 840.08KiB 17.09KiB 0 0.0345263 False False
enterprise_http_to_http 358.15B 0 3.89% 23.84MiB 247.34KiB 5.05KiB 0 0.0101276 23.85MiB 248.29KiB 5.08KiB 0 0.0101664 False False
splunk_hec_indexer_ack_blackhole -413.11B -0 1.12% 23.73MiB 999.78KiB 20.32KiB 0 0.0411378 23.73MiB 1000.72KiB 20.34KiB 0 0.0411771 False False
file_to_blackhole -44.05KiB -0.05 29.84% 95.34MiB 3.72MiB 77.1KiB 0 0.0390052 95.29MiB 4.1MiB 85.31KiB 0 0.0430426 False False
http_to_http_json -38.15KiB -0.16 99.74% 23.85MiB 332.48KiB 6.79KiB 0 0.0136111 23.81MiB 523.79KiB 10.7KiB 0 0.0214763 False False
fluent_elasticsearch -198.27KiB -0.24 100.00% 79.47MiB 52.74KiB 1.07KiB 0 0.000647954 79.28MiB 1.85MiB 38.16KiB 0 0.0233603 False False
http_to_http_noack -113.21KiB -0.46 100.00% 23.85MiB 251.13KiB 5.14KiB 0 0.0102826 23.73MiB 1.15MiB 24.02KiB 0 0.0485305 False False
http_pipelines_blackhole_acks -9.94KiB -0.79 99.99% 1.23MiB 104.22KiB 2.12KiB 0 0.0825419 1.22MiB 73.46KiB 1.5KiB 0 0.0586436 False False
http_pipelines_no_grok_blackhole -129.28KiB -1.23 100.00% 10.28MiB 541.41KiB 11.05KiB 0 0.0514208 10.15MiB 1.08MiB 22.56KiB 0 0.106661 False False
http_pipelines_blackhole -23.35KiB -1.33 100.00% 1.72MiB 11.67KiB 244.23B 0 0.00662535 1.7MiB 122.83KiB 2.5KiB 0 0.0706426 False False

@github-actions
Copy link

Soak Test Results

Baseline: fd9e733
Comparison: fb244fa
Total Vector CPUs: 4

Explanation

A soak test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine, quickly, if vector performance is changed and to what degree by a pull request. Where appropriate units are scaled per-core.

The table below, if present, lists those experiments that have experienced a statistically significant change in their throughput performance between baseline and comparision SHAs, with 90.0% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±8.87% change in mean throughput are discarded. An experiment is erratic if its coefficient of variation is greater than 0.3. The abbreviated table will be omitted if no interesting changes are observed.

No interesting changes in throughput with confidence ≥ 90.00% and absolute Δ mean >= ±8.87%:

Fine details of change detection per experiment.
experiment Δ mean Δ mean % confidence baseline mean baseline stdev baseline stderr baseline outlier % baseline CoV comparison mean comparison stdev comparison stderr comparison outlier % comparison CoV erratic declared erratic
socket_to_socket_blackhole 742.45KiB 3.11 100.00% 23.29MiB 330.78KiB 6.75KiB 0 0.0138655 24.02MiB 131.94KiB 2.69KiB 0 0.00536376 False False
http_text_to_http_json 1.02MiB 2.76 100.00% 36.88MiB 1.32MiB 27.67KiB 0 0.035892 37.9MiB 1.24MiB 25.91KiB 0 0.0326759 False False
syslog_loki 239.11KiB 1.61 100.00% 14.54MiB 374.78KiB 7.67KiB 0 0.0251676 14.77MiB 736.06KiB 14.96KiB 0 0.0486478 False False
syslog_log2metric_splunk_hec_metrics 287.23KiB 1.58 100.00% 17.77MiB 498.25KiB 10.16KiB 0 0.027374 18.05MiB 795.01KiB 16.18KiB 0 0.0429997 False False
syslog_splunk_hec_logs 215.83KiB 1.3 100.00% 16.27MiB 736.13KiB 14.98KiB 0 0.0441855 16.48MiB 690.43KiB 14.07KiB 0 0.0409126 False False
syslog_regex_logs2metric_ddmetrics 130.35KiB 1.03 100.00% 12.39MiB 568.96KiB 11.59KiB 0 0.0448271 12.52MiB 540.99KiB 11.03KiB 0 0.0421901 False False
syslog_humio_logs 130.17KiB 0.77 100.00% 16.44MiB 175.69KiB 3.59KiB 0 0.0104334 16.57MiB 132.3KiB 2.71KiB 0 0.00779607 False False
splunk_hec_route_s3 139.58KiB 0.73 96.67% 18.61MiB 2.25MiB 46.95KiB 0 0.121056 18.74MiB 2.19MiB 45.77KiB 0 0.116691 False False
datadog_agent_remap_blackhole 362.33KiB 0.61 99.98% 58.26MiB 3.84MiB 80.11KiB 0 0.0659793 58.61MiB 2.72MiB 56.85KiB 0 0.0464693 False False
http_pipelines_blackhole_acks 5.23KiB 0.42 96.86% 1.22MiB 97.9KiB 1.99KiB 0 0.0784629 1.22MiB 67.88KiB 1.38KiB 0 0.0541761 False False
splunk_hec_to_splunk_hec_logs_noack 12.27KiB 0.05 73.01% 23.83MiB 430.58KiB 8.8KiB 0 0.0176447 23.84MiB 333.13KiB 6.8KiB 0 0.0136444 False False
splunk_hec_to_splunk_hec_logs_acks 9.85KiB 0.04 31.40% 23.75MiB 867.87KiB 17.65KiB 0 0.0356799 23.76MiB 824.97KiB 16.79KiB 0 0.0339025 False False
enterprise_http_to_http -1003.95B -0 10.76% 23.85MiB 250.84KiB 5.12KiB 0 0.0102703 23.85MiB 250.7KiB 5.13KiB 0 0.0102649 False False
file_to_blackhole -51.31KiB -0.05 37.60% 95.34MiB 3.46MiB 71.64KiB 0 0.036243 95.29MiB 3.67MiB 76.31KiB 0 0.0385303 False False
splunk_hec_indexer_ack_blackhole -11.7KiB -0.05 36.20% 23.77MiB 841.3KiB 17.13KiB 0 0.0345574 23.76MiB 886.46KiB 18.04KiB 0 0.0364299 False False
http_to_http_json -43.35KiB -0.18 99.90% 23.85MiB 331.05KiB 6.76KiB 0 0.0135535 23.81MiB 554.2KiB 11.31KiB 0 0.0227299 False False
datadog_agent_remap_datadog_logs_acks -142.0KiB -0.23 78.88% 60.71MiB 3.37MiB 70.31KiB 0 0.0554458 60.57MiB 4.28MiB 89.18KiB 0 0.0707177 False False
syslog_log2metric_humio_metrics -32.15KiB -0.24 97.30% 12.87MiB 378.88KiB 7.73KiB 0 0.0287341 12.84MiB 604.53KiB 12.3KiB 0 0.0459598 False False
fluent_elasticsearch -242.4KiB -0.3 100.00% 79.47MiB 54.83KiB 1.11KiB 0 0.000673642 79.24MiB 2.56MiB 52.56KiB 0 0.0323164 False False
http_to_http_noack -100.03KiB -0.41 100.00% 23.84MiB 262.13KiB 5.37KiB 0 0.0107332 23.75MiB 1.07MiB 22.23KiB 0 0.0448891 False False
http_to_http_acks -84.42KiB -0.47 27.37% 17.49MiB 8.23MiB 171.96KiB 0 0.470192 17.41MiB 8.1MiB 169.11KiB 0 0.464862 True True
datadog_agent_remap_datadog_logs -421.18KiB -0.69 100.00% 59.94MiB 374.66KiB 7.66KiB 0 0.00610267 59.53MiB 3.96MiB 82.54KiB 0 0.0665714 False False
http_pipelines_blackhole -13.75KiB -0.79 100.00% 1.71MiB 12.11KiB 253.2B 0 0.00691508 1.7MiB 131.29KiB 2.67KiB 0 0.0755895 False False
http_pipelines_no_grok_blackhole -192.45KiB -1.74 100.00% 10.78MiB 131.78KiB 2.69KiB 0 0.0119332 10.59MiB 961.7KiB 19.57KiB 0 0.0886333 False False
datadog_agent_remap_blackhole_acks -1.69MiB -2.79 100.00% 60.51MiB 4.6MiB 95.87KiB 0 0.076061 58.82MiB 4.79MiB 100.06KiB 0 0.0813499 False False

Copy link
Contributor

@neuronull neuronull left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey thanks for the contribution!
It looks like this PR is getting close.

This doesn't happen very often but we actually have two PRs that are touching the same component, and in this case one PR would be overwritten by your changes.

The good news is I believe it should be fairly straightforward to satisfy the other PR's logic within this PR of yours, if you are open to it.

The other conflicting PR is #14779
, where there is a need to expose retry logic settings.

Luckily we have provisions to do this in the new style sink framework.

I've left a couple comments below showing where that would be added.
If you could include that in your PR it would be awesome! 🙏

src/sinks/pulsar/config.rs Show resolved Hide resolved
src/sinks/pulsar/sink.rs Outdated Show resolved Hide resolved
@addisonj
Copy link
Contributor Author

Hey @neuronull, sorry I lost the thread on this one!

Yes I would like to get this in, working on it right now

@netlify
Copy link

netlify bot commented Nov 18, 2022

Deploy Preview for vrl-playground ready!

Name Link
🔨 Latest commit 906f6a1
🔍 Latest deploy log https://app.netlify.com/sites/vrl-playground/deploys/641cd250825f3e000831bc88
😎 Deploy Preview https://deploy-preview-14345--vrl-playground.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site settings.

addisonj and others added 2 commits November 17, 2022 18:07
Co-authored-by: neuronull <kyle.criddle@datadoghq.com>
This changes the Pulsar sink to use the new helpers like:
* Request Builder
* Tower Request / Retry Settings
* Improved size_of

However, it is not currently complining and I need some help
@addisonj
Copy link
Contributor Author

Hey @neuronull,

I have worked on refactoring this to use the tower request and also use some of the newer mechanisms for request builder, retry logic, and the byte counting...

However, I have a snag... and some of my rust inexperience is showing.

To keep this PR clear, I haven't pushed the in-progress changes here, but instead to https://github.com/addisonj/vector/tree/pulsar_sink_improve_help

I think I am pretty close to have it compiling... but hitting a snag that I am struggling to make sense of:

error[E0277]: the trait bound `tower::timeout::Timeout<PulsarService<TokioExecutor>>: Clone` is not satisfied
   --> src/sinks/pulsar/sink.rs:152:26
    |
152 |             .into_driver(service);
    |              ----------- ^^^^^^^ expected an implementor of trait `Clone`
    |              |
    |              required by a bound introduced by this call
    |
    = note: required because of the requirements on the impl of `Clone` for `tower::timeout::Timeout<PulsarService<TokioExecutor>>`
    = note: required because of the requirements on the impl of `tower::Service<PulsarRequest>` for `tower::retry::Retry<retries::FixedRetryPolicy<PulsarRetryLogic>, tower::timeout::Timeout<PulsarService<TokioExecutor>>>`
    = note: 1 redundant requirement hidden
    = note: required because of the requirements on the impl of `tower::Service<PulsarRequest>` for `sinks::util::adaptive_concurrency::service::AdaptiveConcurrencyLimit<tower::retry::Retry<retries::FixedRetryPolicy<PulsarRetryLogic>, tower::timeout::Timeout<PulsarService<TokioExecutor>>>, PulsarRetryLogic>`
help: consider borrowing here
    |
152 |             .into_driver(&service);
    |                          +

error[E0599]: the method `run` exists for struct `vector_core::stream::Driver<futures_util::stream::FilterMap<ConcurrentMap<futures_util::stream::FilterMap<Pin<Box<dyn futures_util::Stream<Item = vector_core::event::Event> + std::marker::Send>>, std::future::Ready<std::option::Option<PulsarEvent>>, [closure@src/sinks/pulsar/sink.rs:141:25: 141:32]>, std::result::Result<PulsarRequest, std::io::Error>>, impl futures_util::Future<Output = std::option::Option<PulsarRequest>>, [closure@src/sinks/pulsar/sink.rs:143:25: 143:34]>, RateLimit<sinks::util::adaptive_concurrency::service::AdaptiveConcurrencyLimit<tower::retry::Retry<retries::FixedRetryPolicy<PulsarRetryLogic>, tower::timeout::Timeout<PulsarService<TokioExecutor>>>, PulsarRetryLogic>>>`, but its trait bounds were not satisfied
   --> src/sinks/pulsar/sink.rs:153:14
    |
153 |           sink.run().await
    |                ^^^ method cannot be called on `vector_core::stream::Driver<futures_util::stream::FilterMap<ConcurrentMap<futures_util::stream::FilterMap<Pin<Box<dyn futures_util::Stream<Item = vector_core::event::Event> + std::marker::Send>>, std::future::Ready<std::option::Option<PulsarEvent>>, [closure@src/sinks/pulsar/sink.rs:141:25: 141:32]>, std::result::Result<PulsarRequest, std::io::Error>>, impl futures_util::Future<Output = std::option::Option<PulsarRequest>>, [closure@src/sinks/pulsar/sink.rs:143:25: 143:34]>, RateLimit<sinks::util::adaptive_concurrency::service::AdaptiveConcurrencyLimit<tower::retry::Retry<retries::FixedRetryPolicy<PulsarRetryLogic>, tower::timeout::Timeout<PulsarService<TokioExecutor>>>, PulsarRetryLogic>>>` due to unsatisfied trait bounds
    |
   ::: src/sinks/util/adaptive_concurrency/service.rs:20:1
    |
20  |   pub struct AdaptiveConcurrencyLimit<S, L> {
    |   ----------------------------------------- doesn't satisfy `_: tower::Service<PulsarRequest>`
    |
   ::: /home/addison/.cargo/registry/src/gh.neting.cc-1ecc6299db9ec823/tower-0.4.13/src/limit/rate/service.rs:14:1
    |
14  |   pub struct RateLimit<T> {
    |   ----------------------- doesn't satisfy `_: tower::Service<PulsarRequest>`
    |
   ::: /home/addison/.cargo/registry/src/gh.neting.cc-1ecc6299db9ec823/tower-0.4.13/src/timeout/mod.rs:19:1
    |
19  |   pub struct Timeout<T> {
    |   --------------------- doesn't satisfy `_: Clone`
    |
   ::: /home/addison/.cargo/registry/src/gh.neting.cc-1ecc6299db9ec823/tower-0.4.13/src/retry/mod.rs:16:1
    |
16  | / pin_project! {
17  | |     /// Configure retrying requests of "failed" responses.
18  | |     ///
19  | |     /// A [`Policy`] classifies what is a "failed" response.
...   |
25  | |     }
26  | | }
    | |_- doesn't satisfy `_: tower::Service<PulsarRequest>`
    |
note: trait bound `tower::retry::Retry<retries::FixedRetryPolicy<PulsarRetryLogic>, tower::timeout::Timeout<PulsarService<TokioExecutor>>>: tower::Service<PulsarRequest>` was not satisfied
   --> src/sinks/util/adaptive_concurrency/service.rs:50:8
    |
48  | impl<S, L, Request> Service<Request> for AdaptiveConcurrencyLimit<S, L>
    |                     ----------------     ------------------------------
49  | where
50  |     S: Service<Request>,
    |        ^^^^^^^^^^^^^^^^ unsatisfied trait bound introduced here
    = note: the following trait bounds were not satisfied:
            `RateLimit<sinks::util::adaptive_concurrency::service::AdaptiveConcurrencyLimit<tower::retry::Retry<retries::FixedRetryPolicy<PulsarRetryLogic>, tower::timeout::Timeout<PulsarService<TokioExecutor>>>, PulsarRetryLogic>>: tower::Service<PulsarRequest>`
            `sinks::util::adaptive_concurrency::service::AdaptiveConcurrencyLimit<tower::retry::Retry<retries::FixedRetryPolicy<PulsarRetryLogic>, tower::timeout::Timeout<PulsarService<TokioExecutor>>>, PulsarRetryLogic>: tower::Service<PulsarRequest>`
            `tower::timeout::Timeout<PulsarService<TokioExecutor>>: Clone`
note: the following trait must be implemented
   --> /home/addison/.cargo/registry/src/gh.neting.cc-1ecc6299db9ec823/tower-service-0.3.1/src/lib.rs:234:1
    |
234 | pub trait Service<Request> {
    | ^^^^^^^^^^^^^^^^^^^^^^^^^^

Comparing to other sinks, I don't see any others that need a clone on passing the service.

I also don't understand what I am missing in implementing the traits, as they seems like they should all be internal to the tower service.

I am going to keep poking at this but if you have any ideas, I would appreciate it!

@neuronull
Copy link
Contributor

I am going to keep poking at this but if you have any ideas, I would appreciate it!

Thanks for tackling the changes!

I'll take a look at this and get back to you.

@neuronull
Copy link
Contributor

I am going to keep poking at this but if you have any ideas, I would appreciate it!

Thanks for tackling the changes!

I'll take a look at this and get back to you.

To solve the immediate problem outlined above, I believe you just need to derive Clone for PulsarService:

#[derive(Clone)] 
pub struct PulsarService<Exe: Executor> {
// ...
}

@neuronull neuronull dismissed stale reviews from bruceg and spencergilbert March 23, 2023 22:25

changes made

@neuronull neuronull added sink: pulsar Anything `pulsar` sink related and removed domain: ci Anything related to Vector's CI environment labels Mar 23, 2023
@github-actions
Copy link

Regression Detector Results

Run ID: 16996c3d-20d8-42f5-9eb4-567abb6b14cf
Baseline: 6ccb825
Comparison: 6d4368b
Total vector CPUs: 7

Explanation

A regression test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine quickly if vector performance is changed and to what degree by a pull request.

The table below, if present, lists those experiments that have experienced a statistically significant change in mean optimization goal performance between baseline and comparison SHAs with 90.00% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±5.00% change in their mean optimization goal are discarded. An experiment is erratic if its coefficient of variation is greater than 0.1. The abbreviated table will be omitted if no interesting change is observed.

No interesting changes in experiment optimization goals with confidence ≥ 90.00% and |Δ mean %| ≥ 5.00%.

Fine details of change detection per experiment.
experiment goal Δ mean Δ mean % confidence baseline mean baseline stdev baseline stderr baseline outlier % baseline CoV comparison mean comparison stdev comparison stderr comparison outlier % comparison CoV erratic declared erratic
http_text_to_http_json ingress throughput 548.94KiB/CPU-s 2.26 100.00% 23.72MiB/CPU-s 578.54KiB/CPU-s 7.12KiB/CPU-s 0.0 0.023814 24.26MiB/CPU-s 735.67KiB/CPU-s 9.05KiB/CPU-s 0.0 0.029613 False False
datadog_agent_remap_datadog_logs ingress throughput 684.91KiB/CPU-s 2.03 100.00% 33.03MiB/CPU-s 1.09MiB/CPU-s 13.68KiB/CPU-s 0.0 0.032888 33.7MiB/CPU-s 749.77KiB/CPU-s 9.23KiB/CPU-s 0.0 0.021728 False False
datadog_agent_remap_datadog_logs_acks ingress throughput 169.04KiB/CPU-s 0.51 100.00% 32.36MiB/CPU-s 1.1MiB/CPU-s 13.85KiB/CPU-s 0.0 0.03397 32.53MiB/CPU-s 1.54MiB/CPU-s 19.43KiB/CPU-s 0.0 0.047397 False False
datadog_agent_remap_blackhole_acks ingress throughput 155.28KiB/CPU-s 0.47 100.00% 32.1MiB/CPU-s 641.78KiB/CPU-s 7.9KiB/CPU-s 0.0 0.019523 32.25MiB/CPU-s 333.58KiB/CPU-s 4.11KiB/CPU-s 0.0 0.0101 False False
http_to_http_json ingress throughput 61.19KiB/CPU-s 0.44 100.00% 13.56MiB/CPU-s 327.6KiB/CPU-s 4.03KiB/CPU-s 0.0 0.023591 13.62MiB/CPU-s 220.27KiB/CPU-s 2.71KiB/CPU-s 0.0 0.015793 False False
syslog_log2metric_humio_metrics ingress throughput 12.17KiB/CPU-s 0.20 99.37% 6.01MiB/CPU-s 230.84KiB/CPU-s 2.84KiB/CPU-s 0.0 0.037502 6.02MiB/CPU-s 278.25KiB/CPU-s 3.42KiB/CPU-s 0.0 0.045114 False False
http_to_http_acks ingress throughput 3.85KiB/CPU-s 0.07 6.28% 5.23MiB/CPU-s 2.75MiB/CPU-s 34.62KiB/CPU-s 0.0 0.525247 5.24MiB/CPU-s 2.74MiB/CPU-s 34.47KiB/CPU-s 0.0 0.522322 True False
enterprise_http_to_http ingress throughput 9.36KiB/CPU-s 0.07 96.01% 13.61MiB/CPU-s 319.22KiB/CPU-s 3.93KiB/CPU-s 0.0 0.022898 13.62MiB/CPU-s 187.62KiB/CPU-s 2.31KiB/CPU-s 0.0 0.013449 False False
splunk_hec_indexer_ack_blackhole ingress throughput 997.61B/CPU-s 0.01 17.48% 13.62MiB/CPU-s 255.7KiB/CPU-s 3.15KiB/CPU-s 0.0 0.018338 13.62MiB/CPU-s 251.4KiB/CPU-s 3.09KiB/CPU-s 0.0 0.018028 False False
splunk_hec_to_splunk_hec_logs_acks ingress throughput 547.27B/CPU-s 0.00 6.65% 13.61MiB/CPU-s 360.04KiB/CPU-s 4.43KiB/CPU-s 0.0 0.025825 13.61MiB/CPU-s 376.22KiB/CPU-s 4.63KiB/CPU-s 0.0 0.026984 False False
fluent_elasticsearch ingress throughput -1.31KiB/CPU-s -0.00 75.77% 45.41MiB/CPU-s 30.06KiB/CPU-s 374.79B/CPU-s 0.0 0.000646 45.41MiB/CPU-s 86.64KiB/CPU-s 1.05KiB/CPU-s 0.0 0.001863 False False
splunk_hec_to_splunk_hec_logs_noack ingress throughput 687.52B/CPU-s 0.00 11.31% 13.61MiB/CPU-s 273.66KiB/CPU-s 3.37KiB/CPU-s 0.0 0.019629 13.61MiB/CPU-s 269.15KiB/CPU-s 3.31KiB/CPU-s 0.0 0.019305 False False
http_to_http_noack ingress throughput -247.21B/CPU-s -0.00 3.28% 13.61MiB/CPU-s 333.9KiB/CPU-s 4.11KiB/CPU-s 0.0 0.023954 13.61MiB/CPU-s 340.29KiB/CPU-s 4.19KiB/CPU-s 0.0 0.024413 False False
otlp_grpc_to_blackhole ingress throughput -792.62B/CPU-s -0.07 63.30% 1.02MiB/CPU-s 50.98KiB/CPU-s 642.39B/CPU-s 0.0 0.048734 1.02MiB/CPU-s 47.56KiB/CPU-s 599.34B/CPU-s 0.0 0.045495 False False
syslog_loki ingress throughput -43.14KiB/CPU-s -0.51 100.00% 8.33MiB/CPU-s 255.28KiB/CPU-s 3.14KiB/CPU-s 0.0 0.029913 8.29MiB/CPU-s 313.06KiB/CPU-s 3.85KiB/CPU-s 0.0 0.03687 False False
syslog_splunk_hec_logs ingress throughput -47.65KiB/CPU-s -0.53 100.00% 8.75MiB/CPU-s 215.71KiB/CPU-s 2.65KiB/CPU-s 0.0 0.024073 8.7MiB/CPU-s 205.88KiB/CPU-s 2.53KiB/CPU-s 0.0 0.023099 False False
socket_to_socket_blackhole ingress throughput -121.47KiB/CPU-s -0.89 100.00% 13.27MiB/CPU-s 320.76KiB/CPU-s 3.95KiB/CPU-s 0.0 0.023599 13.15MiB/CPU-s 372.83KiB/CPU-s 4.59KiB/CPU-s 0.0 0.027676 False False
otlp_http_to_blackhole ingress throughput -14.14KiB/CPU-s -0.90 100.00% 1.54MiB/CPU-s 117.87KiB/CPU-s 1.45KiB/CPU-s 0.0 0.074609 1.53MiB/CPU-s 119.95KiB/CPU-s 1.48KiB/CPU-s 0.0 0.076614 False False
syslog_regex_logs2metric_ddmetrics ingress throughput -44.56KiB/CPU-s -1.24 100.00% 3.51MiB/CPU-s 342.51KiB/CPU-s 4.22KiB/CPU-s 0.0 0.095301 3.47MiB/CPU-s 395.26KiB/CPU-s 4.86KiB/CPU-s 0.0 0.111361 True True
datadog_agent_remap_blackhole ingress throughput -409.21KiB/CPU-s -1.27 100.00% 31.53MiB/CPU-s 1.15MiB/CPU-s 14.51KiB/CPU-s 0.0 0.036534 31.14MiB/CPU-s 1.02MiB/CPU-s 12.8KiB/CPU-s 0.0 0.032635 False False
syslog_humio_logs ingress throughput -169.45KiB/CPU-s -1.83 100.00% 9.06MiB/CPU-s 154.33KiB/CPU-s 1.9KiB/CPU-s 0.0 0.016626 8.9MiB/CPU-s 275.41KiB/CPU-s 3.39KiB/CPU-s 0.0 0.030222 False False
splunk_hec_route_s3 ingress throughput -235.18KiB/CPU-s -1.96 100.00% 11.7MiB/CPU-s 540.02KiB/CPU-s 6.64KiB/CPU-s 0.0 0.045072 11.47MiB/CPU-s 605.0KiB/CPU-s 7.44KiB/CPU-s 0.0 0.051506 False False
syslog_log2metric_splunk_hec_metrics ingress throughput -258.5KiB/CPU-s -2.77 100.00% 9.12MiB/CPU-s 448.03KiB/CPU-s 5.51KiB/CPU-s 0.0 0.047956 8.87MiB/CPU-s 468.36KiB/CPU-s 5.76KiB/CPU-s 0.0 0.051559 False False
file_to_blackhole egress throughput -600.94KiB/CPU-s -8.62 99.94% 6.81MiB/CPU-s 4.67MiB/CPU-s 129.55KiB/CPU-s 0.0 0.685906 6.22MiB/CPU-s 4.3MiB/CPU-s 118.47KiB/CPU-s 0.0 0.690695 True True

@github-actions
Copy link

Regression Detector Results

Run ID: e8089e9d-6368-40b6-b658-c4e6d2aa0088
Baseline: 6ccb825
Comparison: 906f6a1
Total vector CPUs: 7

Explanation

A regression test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine quickly if vector performance is changed and to what degree by a pull request.

The table below, if present, lists those experiments that have experienced a statistically significant change in mean optimization goal performance between baseline and comparison SHAs with 90.00% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±5.00% change in their mean optimization goal are discarded. An experiment is erratic if its coefficient of variation is greater than 0.1. The abbreviated table will be omitted if no interesting change is observed.

No interesting changes in experiment optimization goals with confidence ≥ 90.00% and |Δ mean %| ≥ 5.00%.

Fine details of change detection per experiment.
experiment goal Δ mean Δ mean % confidence baseline mean baseline stdev baseline stderr baseline outlier % baseline CoV comparison mean comparison stdev comparison stderr comparison outlier % comparison CoV erratic declared erratic
file_to_blackhole egress throughput 533.16KiB/CPU-s 8.22 99.62% 6.34MiB/CPU-s 4.19MiB/CPU-s 125.31KiB/CPU-s 9.036658 0.661032 6.86MiB/CPU-s 4.35MiB/CPU-s 135.12KiB/CPU-s 6.543779 0.633455 True True
splunk_hec_route_s3 ingress throughput 478.83KiB/CPU-s 4.21 100.00% 11.1MiB/CPU-s 779.77KiB/CPU-s 9.59KiB/CPU-s 0.0 0.068607 11.57MiB/CPU-s 534.27KiB/CPU-s 6.57KiB/CPU-s 0.0 0.045107 False False
syslog_splunk_hec_logs ingress throughput 137.69KiB/CPU-s 1.55 100.00% 8.7MiB/CPU-s 362.53KiB/CPU-s 4.46KiB/CPU-s 0.0 0.040688 8.84MiB/CPU-s 186.3KiB/CPU-s 2.29KiB/CPU-s 0.0 0.020591 False False
datadog_agent_remap_blackhole ingress throughput 361.76KiB/CPU-s 1.19 100.00% 29.69MiB/CPU-s 2.31MiB/CPU-s 29.07KiB/CPU-s 0.0 0.077735 30.04MiB/CPU-s 1.28MiB/CPU-s 16.09KiB/CPU-s 0.0 0.042494 False False
http_text_to_http_json ingress throughput 275.32KiB/CPU-s 1.16 100.00% 23.23MiB/CPU-s 818.23KiB/CPU-s 10.07KiB/CPU-s 0.0 0.034395 23.5MiB/CPU-s 699.54KiB/CPU-s 8.61KiB/CPU-s 0.0 0.029069 False False
syslog_regex_logs2metric_ddmetrics ingress throughput 31.42KiB/CPU-s 0.90 100.00% 3.4MiB/CPU-s 434.86KiB/CPU-s 5.35KiB/CPU-s 0.0 0.124896 3.43MiB/CPU-s 436.46KiB/CPU-s 5.37KiB/CPU-s 0.0 0.124236 True True
syslog_log2metric_humio_metrics ingress throughput 23.67KiB/CPU-s 0.39 100.00% 5.86MiB/CPU-s 300.65KiB/CPU-s 3.7KiB/CPU-s 0.0 0.050125 5.88MiB/CPU-s 360.57KiB/CPU-s 4.44KiB/CPU-s 0.0 0.059879 False False
otlp_http_to_blackhole ingress throughput 5.65KiB/CPU-s 0.36 99.62% 1.54MiB/CPU-s 115.73KiB/CPU-s 1.42KiB/CPU-s 0.0 0.073166 1.55MiB/CPU-s 108.5KiB/CPU-s 1.34KiB/CPU-s 0.0 0.068355 False False
datadog_agent_remap_datadog_logs_acks ingress throughput 78.74KiB/CPU-s 0.24 99.96% 31.7MiB/CPU-s 1.23MiB/CPU-s 15.55KiB/CPU-s 0.0 0.038937 31.78MiB/CPU-s 1.27MiB/CPU-s 15.98KiB/CPU-s 0.0 0.039933 False False
socket_to_socket_blackhole ingress throughput 29.2KiB/CPU-s 0.21 100.00% 13.29MiB/CPU-s 275.88KiB/CPU-s 3.39KiB/CPU-s 0.0 0.020267 13.32MiB/CPU-s 265.24KiB/CPU-s 3.26KiB/CPU-s 0.0 0.019443 False False
splunk_hec_to_splunk_hec_logs_acks ingress throughput 12.28KiB/CPU-s 0.09 93.13% 13.6MiB/CPU-s 397.24KiB/CPU-s 4.89KiB/CPU-s 0.0 0.028516 13.61MiB/CPU-s 378.16KiB/CPU-s 4.65KiB/CPU-s 0.0 0.027122 False False
enterprise_http_to_http ingress throughput 9.25KiB/CPU-s 0.07 97.74% 13.62MiB/CPU-s 292.16KiB/CPU-s 3.59KiB/CPU-s 0.0 0.020954 13.62MiB/CPU-s 152.42KiB/CPU-s 1.88KiB/CPU-s 0.0 0.010924 False False
http_to_http_noack ingress throughput 7.72KiB/CPU-s 0.06 83.73% 13.61MiB/CPU-s 357.36KiB/CPU-s 4.4KiB/CPU-s 0.0 0.025641 13.62MiB/CPU-s 272.87KiB/CPU-s 3.36KiB/CPU-s 0.0 0.019568 False False
splunk_hec_indexer_ack_blackhole ingress throughput 1.61KiB/CPU-s 0.01 28.10% 13.62MiB/CPU-s 259.58KiB/CPU-s 3.19KiB/CPU-s 0.0 0.018618 13.62MiB/CPU-s 253.81KiB/CPU-s 3.12KiB/CPU-s 0.0 0.018201 False False
http_to_http_json ingress throughput -234.9B/CPU-s -0.00 4.61% 13.62MiB/CPU-s 228.42KiB/CPU-s 2.81KiB/CPU-s 0.0 0.016376 13.62MiB/CPU-s 227.86KiB/CPU-s 2.8KiB/CPU-s 0.0 0.016336 False False
fluent_elasticsearch ingress throughput -10.02KiB/CPU-s -0.02 81.38% 45.41MiB/CPU-s 30.4KiB/CPU-s 378.83B/CPU-s 0.0 0.000654 45.4MiB/CPU-s 622.28KiB/CPU-s 7.57KiB/CPU-s 0.0 0.013383 False False
splunk_hec_to_splunk_hec_logs_noack ingress throughput -3.28KiB/CPU-s -0.02 54.17% 13.62MiB/CPU-s 242.45KiB/CPU-s 2.98KiB/CPU-s 0.0 0.017385 13.62MiB/CPU-s 266.06KiB/CPU-s 3.27KiB/CPU-s 0.0 0.019082 False False
http_to_http_acks ingress throughput -12.23KiB/CPU-s -0.23 19.79% 5.25MiB/CPU-s 2.73MiB/CPU-s 34.46KiB/CPU-s 0.0 0.52054 5.24MiB/CPU-s 2.74MiB/CPU-s 34.54KiB/CPU-s 0.0 0.522938 True False
datadog_agent_remap_datadog_logs ingress throughput -112.72KiB/CPU-s -0.34 100.00% 32.17MiB/CPU-s 1.42MiB/CPU-s 17.86KiB/CPU-s 0.0 0.044063 32.06MiB/CPU-s 1.55MiB/CPU-s 19.47KiB/CPU-s 0.0 0.048205 False False
syslog_log2metric_splunk_hec_metrics ingress throughput -91.42KiB/CPU-s -0.98 100.00% 9.1MiB/CPU-s 263.68KiB/CPU-s 3.24KiB/CPU-s 0.0 0.028283 9.01MiB/CPU-s 356.58KiB/CPU-s 4.39KiB/CPU-s 0.0 0.038626 False False
otlp_grpc_to_blackhole ingress throughput -10.99KiB/CPU-s -1.05 100.00% 1.02MiB/CPU-s 51.97KiB/CPU-s 654.69B/CPU-s 0.0 0.049538 1.01MiB/CPU-s 48.01KiB/CPU-s 604.99B/CPU-s 0.0 0.046248 False False
syslog_humio_logs ingress throughput -183.51KiB/CPU-s -2.03 100.00% 8.84MiB/CPU-s 266.14KiB/CPU-s 3.27KiB/CPU-s 0.0 0.029398 8.66MiB/CPU-s 265.93KiB/CPU-s 3.27KiB/CPU-s 0.0 0.029982 False False
datadog_agent_remap_blackhole_acks ingress throughput -718.47KiB/CPU-s -2.30 100.00% 30.55MiB/CPU-s 1.37MiB/CPU-s 17.28KiB/CPU-s 0.0 0.044872 29.85MiB/CPU-s 1.04MiB/CPU-s 13.07KiB/CPU-s 0.0 0.034727 False False
syslog_loki ingress throughput -244.75KiB/CPU-s -2.85 100.00% 8.39MiB/CPU-s 197.51KiB/CPU-s 2.43KiB/CPU-s 0.0 0.022976 8.16MiB/CPU-s 303.15KiB/CPU-s 3.73KiB/CPU-s 0.0 0.036298 False False

@neuronull neuronull removed the request for review from davidhuie-dd March 24, 2023 21:33
Copy link
Contributor Author

@addisonj addisonj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems reasonable, I don't have any issues! Excited to see this make it in :)

src/sinks/pulsar/service.rs Outdated Show resolved Hide resolved
src/sinks/pulsar/service.rs Outdated Show resolved Hide resolved
src/sinks/pulsar/service.rs Outdated Show resolved Hide resolved
@netlify
Copy link

netlify bot commented Apr 10, 2023

Deploy Preview for vrl-playground canceled.

Name Link
🔨 Latest commit f1a45e8
🔍 Latest deploy log https://app.netlify.com/sites/vrl-playground/deploys/6435b29c8bf3a200089a2568

@github-actions github-actions bot added the domain: ci Anything related to Vector's CI environment label Apr 10, 2023
@github-actions
Copy link

Regression Detector Results

Run ID: 0e981b33-098b-460b-a9fe-6d1234089bf8
Baseline: 887d6d7
Comparison: 12111d2
Total vector CPUs: 7

Explanation

A regression test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine quickly if vector performance is changed and to what degree by a pull request.

The table below, if present, lists those experiments that have experienced a statistically significant change in mean optimization goal performance between baseline and comparison SHAs with 90.00% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±5.00% change in their mean optimization goal are discarded. An experiment is erratic if its coefficient of variation is greater than 0.1. The abbreviated table will be omitted if no interesting change is observed.

No interesting changes in experiment optimization goals with confidence ≥ 90.00% and |Δ mean %| ≥ 5.00%.

Fine details of change detection per experiment.
experiment goal Δ mean Δ mean % confidence baseline mean baseline stdev baseline stderr baseline outlier % baseline CoV comparison mean comparison stdev comparison stderr comparison outlier % comparison CoV erratic declared erratic
file_to_blackhole egress throughput 128.83KiB/CPU-s 2.03 51.13% 6.2MiB/CPU-s 4.4MiB/CPU-s 134.9KiB/CPU-s 1.165919 0.708924 6.33MiB/CPU-s 4.32MiB/CPU-s 128.12KiB/CPU-s 0.0 0.681758 True True
syslog_regex_logs2metric_ddmetrics ingress throughput 56.09KiB/CPU-s 1.58 100.00% 3.46MiB/CPU-s 378.17KiB/CPU-s 4.88KiB/CPU-s 0.0 0.106817 3.51MiB/CPU-s 367.22KiB/CPU-s 4.74KiB/CPU-s 0.0 0.102107 True True
otlp_http_to_blackhole ingress throughput 21.9KiB/CPU-s 1.40 100.00% 1.53MiB/CPU-s 122.49KiB/CPU-s 1.58KiB/CPU-s 0.0 0.078212 1.55MiB/CPU-s 106.68KiB/CPU-s 1.38KiB/CPU-s 0.0 0.067178 False False
socket_to_socket_blackhole ingress throughput 190.22KiB/CPU-s 1.38 100.00% 13.48MiB/CPU-s 295.57KiB/CPU-s 3.81KiB/CPU-s 0.0 0.021413 13.66MiB/CPU-s 206.37KiB/CPU-s 2.66KiB/CPU-s 0.0 0.014747 False False
syslog_splunk_hec_logs ingress throughput 85.54KiB/CPU-s 0.94 100.00% 8.84MiB/CPU-s 319.87KiB/CPU-s 4.13KiB/CPU-s 0.0 0.035332 8.92MiB/CPU-s 215.73KiB/CPU-s 2.78KiB/CPU-s 0.0 0.023605 False False
syslog_log2metric_humio_metrics ingress throughput 58.66KiB/CPU-s 0.88 100.00% 6.51MiB/CPU-s 419.24KiB/CPU-s 5.41KiB/CPU-s 0.0 0.06284 6.57MiB/CPU-s 213.8KiB/CPU-s 2.76KiB/CPU-s 0.0 0.031768 False False
datadog_agent_remap_datadog_logs_acks ingress throughput 181.78KiB/CPU-s 0.55 100.00% 32.52MiB/CPU-s 1.34MiB/CPU-s 17.65KiB/CPU-s 0.0 0.041084 32.69MiB/CPU-s 1.43MiB/CPU-s 18.86KiB/CPU-s 0.0 0.043668 False False
http_to_http_acks ingress throughput 23.55KiB/CPU-s 0.44 36.11% 5.2MiB/CPU-s 2.69MiB/CPU-s 35.49KiB/CPU-s 0.0 0.516481 5.22MiB/CPU-s 2.69MiB/CPU-s 35.49KiB/CPU-s 0.0 0.514319 True False
http_to_http_json ingress throughput 35.1KiB/CPU-s 0.25 100.00% 13.57MiB/CPU-s 301.62KiB/CPU-s 3.89KiB/CPU-s 0.0 0.021711 13.6MiB/CPU-s 270.81KiB/CPU-s 3.49KiB/CPU-s 0.0 0.019444 False False
http_to_http_noack ingress throughput 10.12KiB/CPU-s 0.07 89.82% 13.61MiB/CPU-s 387.99KiB/CPU-s 5.01KiB/CPU-s 0.0 0.027845 13.62MiB/CPU-s 281.16KiB/CPU-s 3.63KiB/CPU-s 0.0 0.020163 False False
http_text_to_http_json ingress throughput 11.49KiB/CPU-s 0.04 53.36% 25.07MiB/CPU-s 957.24KiB/CPU-s 12.35KiB/CPU-s 0.0 0.03729 25.08MiB/CPU-s 759.2KiB/CPU-s 9.8KiB/CPU-s 0.0 0.029562 False False
enterprise_http_to_http ingress throughput 4.04KiB/CPU-s 0.03 74.88% 13.62MiB/CPU-s 227.15KiB/CPU-s 2.93KiB/CPU-s 0.0 0.016285 13.62MiB/CPU-s 151.02KiB/CPU-s 1.95KiB/CPU-s 0.0 0.010824 False False
fluent_elasticsearch ingress throughput 112.21B/CPU-s 0.00 15.98% 45.41MiB/CPU-s 30.07KiB/CPU-s 393.03B/CPU-s 0.0 0.000647 45.41MiB/CPU-s 30.13KiB/CPU-s 393.93B/CPU-s 0.0 0.000648 False False
splunk_hec_to_splunk_hec_logs_acks ingress throughput 60.98B/CPU-s 0.00 0.72% 13.61MiB/CPU-s 347.21KiB/CPU-s 4.48KiB/CPU-s 0.0 0.024905 13.61MiB/CPU-s 371.73KiB/CPU-s 4.8KiB/CPU-s 0.0 0.026664 False False
splunk_hec_to_splunk_hec_logs_noack ingress throughput 343.23B/CPU-s 0.00 5.63% 13.62MiB/CPU-s 261.83KiB/CPU-s 3.38KiB/CPU-s 0.0 0.018778 13.62MiB/CPU-s 258.11KiB/CPU-s 3.33KiB/CPU-s 0.0 0.01851 False False
splunk_hec_indexer_ack_blackhole ingress throughput -3.19KiB/CPU-s -0.02 50.85% 13.62MiB/CPU-s 247.29KiB/CPU-s 3.19KiB/CPU-s 0.0 0.017732 13.61MiB/CPU-s 261.1KiB/CPU-s 3.37KiB/CPU-s 0.0 0.018727 False False
datadog_agent_remap_blackhole ingress throughput -52.16KiB/CPU-s -0.16 99.05% 30.9MiB/CPU-s 1.11MiB/CPU-s 14.69KiB/CPU-s 0.0 0.035983 30.84MiB/CPU-s 1.04MiB/CPU-s 13.71KiB/CPU-s 0.0 0.033636 False False
datadog_agent_remap_blackhole_acks ingress throughput -78.0KiB/CPU-s -0.25 99.96% 30.89MiB/CPU-s 1.21MiB/CPU-s 15.97KiB/CPU-s 0.0 0.039106 30.81MiB/CPU-s 1.13MiB/CPU-s 14.87KiB/CPU-s 0.0 0.036507 False False
otlp_grpc_to_blackhole ingress throughput -2.73KiB/CPU-s -0.26 99.91% 1.03MiB/CPU-s 40.63KiB/CPU-s 537.11B/CPU-s 0.0 0.038352 1.03MiB/CPU-s 49.25KiB/CPU-s 650.87B/CPU-s 0.0 0.04661 False False
datadog_agent_remap_datadog_logs ingress throughput -98.46KiB/CPU-s -0.30 99.96% 32.17MiB/CPU-s 1.37MiB/CPU-s 18.09KiB/CPU-s 0.0 0.042559 32.07MiB/CPU-s 1.59MiB/CPU-s 20.95KiB/CPU-s 0.0 0.049438 False False
syslog_loki ingress throughput -27.64KiB/CPU-s -0.32 100.00% 8.51MiB/CPU-s 245.34KiB/CPU-s 3.16KiB/CPU-s 0.0 0.028166 8.48MiB/CPU-s 248.61KiB/CPU-s 3.21KiB/CPU-s 0.0 0.028633 False False
syslog_humio_logs ingress throughput -33.08KiB/CPU-s -0.36 100.00% 9.0MiB/CPU-s 282.87KiB/CPU-s 3.65KiB/CPU-s 0.0 0.030677 8.97MiB/CPU-s 299.44KiB/CPU-s 3.86KiB/CPU-s 0.0 0.032591 False False
splunk_hec_route_s3 ingress throughput -122.71KiB/CPU-s -1.04 100.00% 11.56MiB/CPU-s 614.65KiB/CPU-s 7.93KiB/CPU-s 0.0 0.05193 11.44MiB/CPU-s 727.24KiB/CPU-s 9.38KiB/CPU-s 0.0 0.062086 False False
syslog_log2metric_splunk_hec_metrics ingress throughput -307.72KiB/CPU-s -3.22 100.00% 9.34MiB/CPU-s 332.38KiB/CPU-s 4.29KiB/CPU-s 0.0 0.034753 9.04MiB/CPU-s 328.48KiB/CPU-s 4.24KiB/CPU-s 0.0 0.035486 False False

@neuronull neuronull enabled auto-merge (squash) April 11, 2023 19:23
@neuronull neuronull disabled auto-merge April 11, 2023 19:26
@github-actions
Copy link

Regression Detector Results

Run ID: 4a63dcbd-a9e2-4169-b89d-c5d275781b2e
Baseline: f3b5d42
Comparison: f1a45e8
Total vector CPUs: 7

Explanation

A regression test is an integrated performance test for vector in a repeatable rig, with varying configuration for vector. What follows is a statistical summary of a brief vector run for each configuration across SHAs given above. The goal of these tests are to determine quickly if vector performance is changed and to what degree by a pull request.

The table below, if present, lists those experiments that have experienced a statistically significant change in mean optimization goal performance between baseline and comparison SHAs with 90.00% confidence OR have been detected as newly erratic. Negative values mean that baseline is faster, positive comparison. Results that do not exhibit more than a ±5.00% change in their mean optimization goal are discarded. An experiment is erratic if its coefficient of variation is greater than 0.1. The abbreviated table will be omitted if no interesting change is observed.

No interesting changes in experiment optimization goals with confidence ≥ 90.00% and |Δ mean %| ≥ 5.00%.

Fine details of change detection per experiment.
experiment goal Δ mean Δ mean % confidence baseline mean baseline stdev baseline stderr baseline outlier % baseline CoV comparison mean comparison stdev comparison stderr comparison outlier % comparison CoV erratic declared erratic
file_to_blackhole egress throughput 1.04MiB/CPU-s 17.34 100.00% 6.02MiB/CPU-s 4.65MiB/CPU-s 131.09KiB/CPU-s 0.0 0.771565 7.07MiB/CPU-s 4.26MiB/CPU-s 142.55KiB/CPU-s 6.289979 0.6029 True True
datadog_agent_remap_blackhole ingress throughput 1.14MiB/CPU-s 3.82 100.00% 29.72MiB/CPU-s 1.52MiB/CPU-s 20.08KiB/CPU-s 0.0 0.05114 30.86MiB/CPU-s 1.29MiB/CPU-s 17.07KiB/CPU-s 0.0 0.041841 False False
datadog_agent_remap_blackhole_acks ingress throughput 1.05MiB/CPU-s 3.53 100.00% 29.78MiB/CPU-s 1.48MiB/CPU-s 19.53KiB/CPU-s 0.0 0.04962 30.83MiB/CPU-s 1.18MiB/CPU-s 15.55KiB/CPU-s 0.0 0.038161 False False
datadog_agent_remap_datadog_logs ingress throughput 857.7KiB/CPU-s 2.62 100.00% 32.01MiB/CPU-s 1.84MiB/CPU-s 24.32KiB/CPU-s 0.0 0.057481 32.85MiB/CPU-s 1.66MiB/CPU-s 21.97KiB/CPU-s 0.0 0.050614 False False
datadog_agent_remap_datadog_logs_acks ingress throughput 802.04KiB/CPU-s 2.45 100.00% 32.03MiB/CPU-s 1.63MiB/CPU-s 21.6KiB/CPU-s 0.0 0.051034 32.82MiB/CPU-s 1.39MiB/CPU-s 18.38KiB/CPU-s 0.0 0.042379 False False
syslog_regex_logs2metric_ddmetrics ingress throughput 87.84KiB/CPU-s 2.42 100.00% 3.54MiB/CPU-s 506.7KiB/CPU-s 6.54KiB/CPU-s 0.0 0.139676 3.63MiB/CPU-s 501.84KiB/CPU-s 6.48KiB/CPU-s 0.0 0.135066 True True
socket_to_socket_blackhole ingress throughput 286.9KiB/CPU-s 2.13 100.00% 13.14MiB/CPU-s 713.44KiB/CPU-s 9.21KiB/CPU-s 0.0 0.053037 13.42MiB/CPU-s 419.7KiB/CPU-s 5.42KiB/CPU-s 0.0 0.030549 False False
syslog_log2metric_splunk_hec_metrics ingress throughput 157.27KiB/CPU-s 1.68 100.00% 9.13MiB/CPU-s 435.47KiB/CPU-s 5.62KiB/CPU-s 0.0 0.046594 9.28MiB/CPU-s 351.01KiB/CPU-s 4.53KiB/CPU-s 0.0 0.036935 False False
syslog_humio_logs ingress throughput 103.9KiB/CPU-s 1.17 100.00% 8.67MiB/CPU-s 477.69KiB/CPU-s 6.16KiB/CPU-s 0.0 0.053816 8.77MiB/CPU-s 295.77KiB/CPU-s 3.82KiB/CPU-s 0.0 0.032936 False False
http_text_to_http_json ingress throughput 216.22KiB/CPU-s 0.84 100.00% 25.27MiB/CPU-s 922.77KiB/CPU-s 11.91KiB/CPU-s 0.0 0.035655 25.48MiB/CPU-s 653.73KiB/CPU-s 8.44KiB/CPU-s 0.0 0.02505 False False
syslog_splunk_hec_logs ingress throughput 64.11KiB/CPU-s 0.72 100.00% 8.76MiB/CPU-s 329.06KiB/CPU-s 4.24KiB/CPU-s 0.0 0.036699 8.82MiB/CPU-s 308.9KiB/CPU-s 3.98KiB/CPU-s 0.0 0.034205 False False
otlp_grpc_to_blackhole ingress throughput 4.99KiB/CPU-s 0.48 100.00% 1.01MiB/CPU-s 66.64KiB/CPU-s 880.65B/CPU-s 0.0 0.064668 1.01MiB/CPU-s 55.29KiB/CPU-s 730.73B/CPU-s 0.0 0.053392 False False
splunk_hec_route_s3 ingress throughput 52.75KiB/CPU-s 0.45 100.00% 11.55MiB/CPU-s 584.74KiB/CPU-s 7.55KiB/CPU-s 0.0 0.049426 11.6MiB/CPU-s 672.76KiB/CPU-s 8.68KiB/CPU-s 0.0 0.056613 False False
enterprise_http_to_http ingress throughput 10.25KiB/CPU-s 0.07 98.13% 13.61MiB/CPU-s 302.55KiB/CPU-s 3.9KiB/CPU-s 0.0 0.0217 13.62MiB/CPU-s 149.79KiB/CPU-s 1.93KiB/CPU-s 0.0 0.010736 False False
http_to_http_noack ingress throughput 6.27KiB/CPU-s 0.04 71.43% 13.61MiB/CPU-s 355.76KiB/CPU-s 4.59KiB/CPU-s 0.0 0.025525 13.62MiB/CPU-s 283.62KiB/CPU-s 3.66KiB/CPU-s 0.0 0.02034 False False
splunk_hec_indexer_ack_blackhole ingress throughput 1.73KiB/CPU-s 0.01 29.59% 13.62MiB/CPU-s 253.52KiB/CPU-s 3.27KiB/CPU-s 0.0 0.018181 13.62MiB/CPU-s 245.43KiB/CPU-s 3.17KiB/CPU-s 0.0 0.017599 False False
splunk_hec_to_splunk_hec_logs_acks ingress throughput 429.02B/CPU-s 0.00 5.34% 13.61MiB/CPU-s 336.76KiB/CPU-s 4.34KiB/CPU-s 0.0 0.024156 13.61MiB/CPU-s 348.51KiB/CPU-s 4.5KiB/CPU-s 0.0 0.024998 False False
fluent_elasticsearch ingress throughput -295.85B/CPU-s -0.00 40.58% 45.41MiB/CPU-s 29.69KiB/CPU-s 387.94B/CPU-s 0.0 0.000638 45.41MiB/CPU-s 30.4KiB/CPU-s 397.25B/CPU-s 0.0 0.000654 False False
splunk_hec_to_splunk_hec_logs_noack ingress throughput -4.52KiB/CPU-s -0.03 65.65% 13.62MiB/CPU-s 245.15KiB/CPU-s 3.16KiB/CPU-s 0.0 0.017579 13.61MiB/CPU-s 277.15KiB/CPU-s 3.58KiB/CPU-s 0.0 0.01988 False False
otlp_http_to_blackhole ingress throughput -2.51KiB/CPU-s -0.16 74.57% 1.54MiB/CPU-s 123.05KiB/CPU-s 1.59KiB/CPU-s 0.0 0.078249 1.53MiB/CPU-s 117.91KiB/CPU-s 1.52KiB/CPU-s 0.0 0.075104 False False
syslog_loki ingress throughput -62.2KiB/CPU-s -0.71 100.00% 8.55MiB/CPU-s 288.77KiB/CPU-s 3.72KiB/CPU-s 0.0 0.032985 8.49MiB/CPU-s 396.93KiB/CPU-s 5.12KiB/CPU-s 0.0 0.045664 False False
http_to_http_json ingress throughput -102.35KiB/CPU-s -0.74 100.00% 13.55MiB/CPU-s 322.99KiB/CPU-s 4.17KiB/CPU-s 0.0 0.023271 13.45MiB/CPU-s 419.88KiB/CPU-s 5.42KiB/CPU-s 0.0 0.030477 False False
http_to_http_acks ingress throughput -65.46KiB/CPU-s -1.22 80.05% 5.26MiB/CPU-s 2.73MiB/CPU-s 36.01KiB/CPU-s 0.0 0.518248 5.19MiB/CPU-s 2.73MiB/CPU-s 36.14KiB/CPU-s 0.0 0.526404 True False
syslog_log2metric_humio_metrics ingress throughput -221.76KiB/CPU-s -3.30 100.00% 6.57MiB/CPU-s 298.26KiB/CPU-s 3.85KiB/CPU-s 0.0 0.044325 6.35MiB/CPU-s 399.41KiB/CPU-s 5.15KiB/CPU-s 0.0 0.061381 False False

@neuronull
Copy link
Contributor

At last, merging this one in.
Thanks again for the contribution @addisonj 🚀 !
One day we will have all sinks be stream based and you've helped us get that much closer.

@neuronull neuronull merged commit 1e97a2f into vectordotdev:master Apr 11, 2023
@addisonj
Copy link
Contributor Author

Woohoo! thanks for getting this over the line.

Will try and do some testing of this as we have some pretty high scale use cases

aholmberg pushed a commit to aholmberg/vector that referenced this pull request Feb 14, 2024
# [1.13.0](answerbook/vector@v1.12.1...v1.13.0) (2023-09-13)

### Bug Fixes

* **appsignal sink**: Add TLS config option (vectordotdev#17122) [198068c](answerbook/vector@198068c) - GitHub
* **buffers**: correctly handle partial writes in reader seek during initialization (vectordotdev#17099) [a791595](answerbook/vector@a791595) - GitHub
* **config**: recurse through schema refs when determining eligibility for unevaluated properties (vectordotdev#17150) [71d1bf6](answerbook/vector@71d1bf6) - GitHub
* **docker_logs source**: Support tcp schema [e1c0c02](answerbook/vector@e1c0c02) - GitHub
* **elasticsearch sink**:  Elasticsearch sink with api_version set to "auto" does not recognize the API version of ES6 as V6 (vectordotdev#17226) (vectordotdev#17227) [9b6ef24](answerbook/vector@9b6ef24) - GitHub
* **gcp_stackdriver_metrics sink**: Call function to regenerate auth token (vectordotdev#17297) [bf7904b](answerbook/vector@bf7904b) - GitHub
* **influxdb_logs**: encode influx line when no tags present (vectordotdev#17029) [c3aa14f](answerbook/vector@c3aa14f) - GitHub
* **reduce transform**: Revert flushing on interval change to `expire_metrics_ms` (vectordotdev#17084) [e86b155](answerbook/vector@e86b155) - GitHub
* **releasing**: Fix globbing of release artifacts for GitHub (vectordotdev#17114) [7fe089c](answerbook/vector@7fe089c) - GitHub
* **schemas**: Dont panic with non object field kinds (vectordotdev#17140) [1e43208](answerbook/vector@1e43208) - GitHub

### Chores

* (syslog source): add source_ip to some syslog tests (vectordotdev#17235) [29c34c0](answerbook/vector@29c34c0) - GitHub
* add note to DEVELOPING.md re panics (vectordotdev#17277) [03e905e](answerbook/vector@03e905e) - GitHub
* Add UX note about encoding of log_schema keys (vectordotdev#17266) [dc6e54c](answerbook/vector@dc6e54c) - GitHub
* **administration**: add `appsignal` to codeowners (vectordotdev#17127) [7b15d19](answerbook/vector@7b15d19) - GitHub
* **buffer**: tidy up some of the module level docs for `disk_v2` (vectordotdev#17093) [edaa612](answerbook/vector@edaa612) - GitHub
* **ci**: bump docker/metadata-action from 4.3.0 to 4.4.0 (vectordotdev#17170) [854d71e](answerbook/vector@854d71e) - GitHub
* **ci**: Disable `appsignal` integration test until CA issues are resolved (vectordotdev#17109) [f3b5d42](answerbook/vector@f3b5d42) - GitHub
* **ci**: Disable scheduled runs of Baseline Timings workflow (vectordotdev#17281) [4335b0a](answerbook/vector@4335b0a) - GitHub
* **ci**: Fix event assertions for `aws_ec2_metadata` transform (vectordotdev#17413) [da36fb6](answerbook/vector@da36fb6) - GitHub
* **ci**: Increase timeout for integration tests (vectordotdev#17326) [e1f125a](answerbook/vector@e1f125a) - GitHub
* **ci**: Increase timeout for integration tests to 30m (vectordotdev#17350) [5d3f619](answerbook/vector@5d3f619) - GitHub
* **ci**: re-enable `appsignal` integration test (vectordotdev#17111) [48fc574](answerbook/vector@48fc574) - GitHub
* **ci**: Remove ci-sweep tasks (vectordotdev#17415) [5c33f99](answerbook/vector@5c33f99) - GitHub
* **ci**: remove unnecessary dep install (vectordotdev#17128) [f56d1ef](answerbook/vector@f56d1ef) - GitHub
* **ci**: Try to fix apt retries (vectordotdev#17393) [6b3db04](answerbook/vector@6b3db04) - GitHub
* **ci**: update unsupported ubuntu version runners (vectordotdev#17113) [e7c4815](answerbook/vector@e7c4815) - GitHub
* **ci**: use python v3.8 in ubuntu 20.04 runner (vectordotdev#17116) [7a40c81](answerbook/vector@7a40c81) - GitHub
* **config**: begin laying out primitives for programmatically querying schema (vectordotdev#17130) [aad8115](answerbook/vector@aad8115) - GitHub
* **config**: emit human-friendly version of enum variant/property names in schema (vectordotdev#17171) [3b38ba8](answerbook/vector@3b38ba8) - GitHub
* **config**: improve config schema output with more precise `unevaluatedProperties` + schema ref flattening (vectordotdev#17026) [2d72f82](answerbook/vector@2d72f82) - GitHub
* **deps**: Add 3rd party license file and CI checks (vectordotdev#17344) [7350e1a](answerbook/vector@7350e1a) - GitHub
* **deps**: bump anyhow from 1.0.70 to 1.0.71 (vectordotdev#17300) [6a5af3b](answerbook/vector@6a5af3b) - GitHub
* **deps**: bump assert_cmd from 2.0.10 to 2.0.11 (vectordotdev#17290) [c4784fd](answerbook/vector@c4784fd) - GitHub
* **deps**: bump async-compression from 0.3.15 to 0.4.0 (vectordotdev#17365) [b9aac47](answerbook/vector@b9aac47) - GitHub
* **deps**: bump async-graphql from 5.0.7 to 5.0.8 (vectordotdev#17357) [05a4f17](answerbook/vector@05a4f17) - GitHub
* **deps**: bump async-graphql-warp from 5.0.7 to 5.0.8 (vectordotdev#17367) [693584e](answerbook/vector@693584e) - GitHub
* **deps**: bump async-stream from 0.3.4 to 0.3.5 (vectordotdev#17076) [c29c817](answerbook/vector@c29c817) - GitHub
* **deps**: bump aws-sigv4 from 0.55.0 to 0.55.1 (vectordotdev#17138) [dbb3f25](answerbook/vector@dbb3f25) - GitHub
* **deps**: bump axum from 0.6.12 to 0.6.18 (vectordotdev#17257) [41ac76e](answerbook/vector@41ac76e) - GitHub
* **deps**: bump cached from 0.42.0 to 0.43.0 (vectordotdev#17118) [f90b3b3](answerbook/vector@f90b3b3) - GitHub
* **deps**: bump chrono-tz from 0.8.1 to 0.8.2 (vectordotdev#17088) [623b838](answerbook/vector@623b838) - GitHub
* **deps**: bump clap_complete from 4.2.0 to 4.2.1 (vectordotdev#17229) [d286d16](answerbook/vector@d286d16) - GitHub
* **deps**: bump clap_complete from 4.2.1 to 4.2.2 (vectordotdev#17359) [565668e](answerbook/vector@565668e) - GitHub
* **deps**: bump clap_complete from 4.2.2 to 4.2.3 (vectordotdev#17383) [111cd07](answerbook/vector@111cd07) - GitHub
* **deps**: bump console-subscriber from 0.1.8 to 0.1.9 (vectordotdev#17358) [97b862c](answerbook/vector@97b862c) - GitHub
* **deps**: bump directories from 5.0.0 to 5.0.1 (vectordotdev#17271) [be69f5f](answerbook/vector@be69f5f) - GitHub
* **deps**: bump dunce from 1.0.3 to 1.0.4 (vectordotdev#17244) [cfc387d](answerbook/vector@cfc387d) - GitHub
* **deps**: bump enumflags2 from 0.7.5 to 0.7.6 (vectordotdev#17079) [cbc17be](answerbook/vector@cbc17be) - GitHub
* **deps**: bump enumflags2 from 0.7.6 to 0.7.7 (vectordotdev#17206) [c80c5eb](answerbook/vector@c80c5eb) - GitHub
* **deps**: bump flate2 from 1.0.25 to 1.0.26 (vectordotdev#17320) [ef13370](answerbook/vector@ef13370) - GitHub
* **deps**: bump getrandom from 0.2.8 to 0.2.9 (vectordotdev#17101) [d53240b](answerbook/vector@d53240b) - GitHub
* **deps**: bump h2 from 0.3.18 to 0.3.19 (vectordotdev#17388) [6088abd](answerbook/vector@6088abd) - GitHub
* **deps**: bump hashlink from 0.8.1 to 0.8.2 (vectordotdev#17419) [01b3cd7](answerbook/vector@01b3cd7) - GitHub
* **deps**: bump hyper from 0.14.25 to 0.14.26 (vectordotdev#17347) [c43dcfd](answerbook/vector@c43dcfd) - GitHub
* **deps**: bump inventory from 0.3.5 to 0.3.6 (vectordotdev#17401) [5b5ad16](answerbook/vector@5b5ad16) - GitHub
* **deps**: bump libc from 0.2.140 to 0.2.141 (vectordotdev#17104) [dd9608a](answerbook/vector@dd9608a) - GitHub
* **deps**: bump libc from 0.2.141 to 0.2.142 (vectordotdev#17273) [bc618a2](answerbook/vector@bc618a2) - GitHub
* **deps**: bump libc from 0.2.142 to 0.2.143 (vectordotdev#17338) [6afe206](answerbook/vector@6afe206) - GitHub
* **deps**: bump libc from 0.2.143 to 0.2.144 (vectordotdev#17346) [99b8dc1](answerbook/vector@99b8dc1) - GitHub
* **deps**: bump memmap2 from 0.5.10 to 0.6.0 (vectordotdev#17355) [dae0c6a](answerbook/vector@dae0c6a) - GitHub
* **deps**: bump memmap2 from 0.6.0 to 0.6.1 (vectordotdev#17364) [58ba741](answerbook/vector@58ba741) - GitHub
* **deps**: bump metrics, metrics-tracing-context, metrics-util (vectordotdev#17336) [9a723e3](answerbook/vector@9a723e3) - GitHub
* **deps**: bump mlua from 0.8.8 to 0.8.9 (vectordotdev#17423) [57f8bd4](answerbook/vector@57f8bd4) - GitHub
* **deps**: bump mock_instant from 0.2.1 to 0.3.0 (vectordotdev#17210) [40c9afc](answerbook/vector@40c9afc) - GitHub
* **deps**: bump mongodb from 2.4.0 to 2.5.0 (vectordotdev#17337) [64f4f69](answerbook/vector@64f4f69) - GitHub
* **deps**: bump nkeys from 0.2.0 to 0.3.0 (vectordotdev#17421) [3320eda](answerbook/vector@3320eda) - GitHub
* **deps**: bump notify from 5.1.0 to 6.0.0 (vectordotdev#17422) [58603b9](answerbook/vector@58603b9) - GitHub
* **deps**: bump num_enum from 0.5.11 to 0.6.0 (vectordotdev#17106) [42f298b](answerbook/vector@42f298b) - GitHub
* **deps**: bump num_enum from 0.6.0 to 0.6.1 (vectordotdev#17272) [f696e7b](answerbook/vector@f696e7b) - GitHub
* **deps**: bump opendal from 0.30.5 to 0.31.0 (vectordotdev#17119) [8762563](answerbook/vector@8762563) - GitHub
* **deps**: bump opendal from 0.31.0 to 0.33.2 (vectordotdev#17286) [3d41931](answerbook/vector@3d41931) - GitHub
* **deps**: bump opendal from 0.33.2 to 0.34.0 (vectordotdev#17354) [ae602da](answerbook/vector@ae602da) - GitHub
* **deps**: bump openssl from 0.10.48 to 0.10.50 (vectordotdev#17087) [9a56ed8](answerbook/vector@9a56ed8) - GitHub
* **deps**: bump openssl from 0.10.50 to 0.10.52 (vectordotdev#17299) [0ecceb3](answerbook/vector@0ecceb3) - GitHub
* **deps**: bump pin-project from 1.0.12 to 1.1.0 (vectordotdev#17385) [e8d3002](answerbook/vector@e8d3002) - GitHub
* **deps**: bump prettydiff from 0.6.2 to 0.6.4 (vectordotdev#17089) [e090610](answerbook/vector@e090610) - GitHub
* **deps**: bump prettydiff from 0.6.2 to 0.6.4 (vectordotdev#17315) [a1ec68d](answerbook/vector@a1ec68d) - GitHub
* **deps**: bump proc-macro2 from 1.0.55 to 1.0.56 (vectordotdev#17103) [6f74523](answerbook/vector@6f74523) - GitHub
* **deps**: bump proc-macro2 from 1.0.56 to 1.0.57 (vectordotdev#17400) [a6e1ae7](answerbook/vector@a6e1ae7) - GitHub
* **deps**: bump prost-build from 0.11.8 to 0.11.9 (vectordotdev#17260) [a88aba4](answerbook/vector@a88aba4) - GitHub
* **deps**: bump quote from 1.0.26 to 1.0.27 (vectordotdev#17348) [f81ff18](answerbook/vector@f81ff18) - GitHub
* **deps**: bump rdkafka from 0.29.0 to 0.30.0 (vectordotdev#17387) [9703188](answerbook/vector@9703188) - GitHub
* **deps**: bump regex from 1.7.3 to 1.8.1 (vectordotdev#17222) [410aa3c](answerbook/vector@410aa3c) - GitHub
* **deps**: bump reqwest from 0.11.16 to 0.11.17 (vectordotdev#17316) [09176ec](answerbook/vector@09176ec) - GitHub
* **deps**: bump security-framework from 2.8.2 to 2.9.0 (vectordotdev#17386) [1287168](answerbook/vector@1287168) - GitHub
* **deps**: bump serde from 1.0.159 to 1.0.160 (vectordotdev#17270) [036ad4a](answerbook/vector@036ad4a) - GitHub
* **deps**: bump serde from 1.0.160 to 1.0.162 (vectordotdev#17317) [79e97a2](answerbook/vector@79e97a2) - GitHub
* **deps**: bump serde from 1.0.162 to 1.0.163 (vectordotdev#17366) [9852c17](answerbook/vector@9852c17) - GitHub
* **deps**: bump serde_json from 1.0.95 to 1.0.96 (vectordotdev#17258) [7570bb3](answerbook/vector@7570bb3) - GitHub
* **deps**: bump serde_with from 2.3.1 to 2.3.2 (vectordotdev#17090) [adbf4d5](answerbook/vector@adbf4d5) - GitHub
* **deps**: bump serde_yaml from 0.9.19 to 0.9.21 (vectordotdev#17120) [d6f2625](answerbook/vector@d6f2625) - GitHub
* **deps**: bump socket2 from 0.4.7 to 0.5.2 (vectordotdev#17121) [db39d83](answerbook/vector@db39d83) - GitHub
* **deps**: bump socket2 from 0.5.2 to 0.5.3 (vectordotdev#17384) [ac51b8a](answerbook/vector@ac51b8a) - GitHub
* **deps**: bump syslog from 6.0.1 to 6.1.0 (vectordotdev#17301) [61e6154](answerbook/vector@61e6154) - GitHub
* **deps**: bump tokio from 1.27.0 to 1.28.0 (vectordotdev#17231) [8067f84](answerbook/vector@8067f84) - GitHub
* **deps**: bump tokio from 1.28.0 to 1.28.1 (vectordotdev#17368) [ae6a51b](answerbook/vector@ae6a51b) - GitHub
* **deps**: bump tokio-stream from 0.1.12 to 0.1.14 (vectordotdev#17339) [80c8247](answerbook/vector@80c8247) - GitHub
* **deps**: bump tokio-tungstenite from 0.18.0 to 0.19.0 (vectordotdev#17404) [ae1dd6e](answerbook/vector@ae1dd6e) - GitHub
* **deps**: bump tonic from 0.8.3 to 0.9.1 (vectordotdev#17077) [eafba69](answerbook/vector@eafba69) - GitHub
* **deps**: bump tonic from 0.9.1 to 0.9.2 (vectordotdev#17221) [aa9cbd0](answerbook/vector@aa9cbd0) - GitHub
* **deps**: bump tonic-build from 0.8.4 to 0.9.2 (vectordotdev#17274) [e0a07c6](answerbook/vector@e0a07c6) - GitHub
* **deps**: bump tracing-subscriber from 0.3.16 to 0.3.17 (vectordotdev#17268) [1406c08](answerbook/vector@1406c08) - GitHub
* **deps**: bump typetag from 0.2.7 to 0.2.8 (vectordotdev#17302) [c8e0e5f](answerbook/vector@c8e0e5f) - GitHub
* **deps**: bump uuid from 1.3.0 to 1.3.1 (vectordotdev#17091) [9cc2f1d](answerbook/vector@9cc2f1d) - GitHub
* **deps**: bump uuid from 1.3.0 to 1.3.2 (vectordotdev#17256) [bc6f7fd](answerbook/vector@bc6f7fd) - GitHub
* **deps**: bump uuid from 1.3.2 to 1.3.3 (vectordotdev#17403) [3a3fe63](answerbook/vector@3a3fe63) - GitHub
* **deps**: bump warp from 0.3.4 to 0.3.5 (vectordotdev#17288) [d8c1f12](answerbook/vector@d8c1f12) - GitHub
* **deps**: bump wasm-bindgen from 0.2.84 to 0.2.85 (vectordotdev#17356) [ea24b4d](answerbook/vector@ea24b4d) - GitHub
* **deps**: bump wasm-bindgen from 0.2.85 to 0.2.86 (vectordotdev#17402) [0518176](answerbook/vector@0518176) - GitHub
* **deps**: bump wiremock from 0.5.17 to 0.5.18 (vectordotdev#17092) [51312aa](answerbook/vector@51312aa) - GitHub
* **deps**: Fix up missing license (vectordotdev#17379) [a2b8903](answerbook/vector@a2b8903) - GitHub
* **deps**: Reset dependencies bumped by a61dea1 (vectordotdev#17100) [887d6d7](answerbook/vector@887d6d7) - GitHub
* **deps**: true up cargo.lock (vectordotdev#17149) [10fce65](answerbook/vector@10fce65) - GitHub
* **deps**: Update h2 (vectordotdev#17189) [a2882f3](answerbook/vector@a2882f3) - GitHub
* **deps**: Upgrade cue to 0.5.0 (vectordotdev#17204) [d396320](answerbook/vector@d396320) - GitHub
* **deps**: Upgrade Debian to bullseye for distroless image (vectordotdev#17160) [c304a8c](answerbook/vector@c304a8c) - GitHub
* **deps**: Upgrade rust to 1.69.0 (vectordotdev#17194) [ef15696](answerbook/vector@ef15696) - GitHub
* **dev**: Add note about generating licenses to CONTRIBUTING.md (vectordotdev#17410) [539f379](answerbook/vector@539f379) - GitHub
* **dev**: ignore `.helix` dir (vectordotdev#17203) [32a935b](answerbook/vector@32a935b) - GitHub
* **dev**: Install the correct `mold` based on CPU architecture (vectordotdev#17248) [4b80c71](answerbook/vector@4b80c71) - GitHub
* **dev**: remove editors from gitignore (vectordotdev#17267) [61c0d76](answerbook/vector@61c0d76) - GitHub
* **docs**: Add Enterprise link and update Support link (vectordotdev#17408) [5184d50](answerbook/vector@5184d50) - GitHub
* **docs**: Add missing 0.28.2 version [38607cd](answerbook/vector@38607cd) - Jesse Szwedko
* **docs**: Clarify `key_field` for `sample` and `throttle` transforms (vectordotdev#17372) [d1e5588](answerbook/vector@d1e5588) - GitHub
* **docs**: Document event type conditions (vectordotdev#17311) [a9c8dc8](answerbook/vector@a9c8dc8) - GitHub
* **docs**: make doc style edits (vectordotdev#17155) [65a8856](answerbook/vector@65a8856) - GitHub
* **docs**: Remove trailing, unmatched quote (vectordotdev#17163) [3c92556](answerbook/vector@3c92556) - GitHub
* **docs**: Remove unneeded `yaml` dependency from website (vectordotdev#17215) [752d424](answerbook/vector@752d424) - GitHub
* **docs**: Update component statuses 2023Q2 (vectordotdev#17362) [22cda94](answerbook/vector@22cda94) - GitHub
* **docs**: update the `v0.28.0` upgrade guide with note about `datadog_logs` sink `hostname` key (vectordotdev#17156) [c169131](answerbook/vector@c169131) - GitHub
* **external docs**: correctly mark some sinks as stateful (vectordotdev#17085) [64d560d](answerbook/vector@64d560d) - GitHub
* **loki sink**: warn on label expansions and collisions (vectordotdev#17052) [f06692b](answerbook/vector@f06692b) - GitHub
* **pulsar**: pulsar-rs bump to v5.1.1 (vectordotdev#17159) [68b54a9](answerbook/vector@68b54a9) - GitHub
* Re-add transform definitions (vectordotdev#17152) [9031d0f](answerbook/vector@9031d0f) - GitHub
* Regen docs for sample and throttle (vectordotdev#17390) [6c57ca0](answerbook/vector@6c57ca0) - GitHub
* **releasing**: Add known issues fixed by 0.29.1 (vectordotdev#17218) [40d543a](answerbook/vector@40d543a) - GitHub
* **releasing**: Bump Vector version to 0.30.0 (vectordotdev#17134) [3834612](answerbook/vector@3834612) - GitHub
* **releasing**: Fix homebrew release script (vectordotdev#17131) [cfbf233](answerbook/vector@cfbf233) - Jesse Szwedko
* **releasing**: Fix release channels (vectordotdev#17133) [58b44e8](answerbook/vector@58b44e8) - Jesse Szwedko
* **releasing**: Prepare v0.28.2 release [a61dea1](answerbook/vector@a61dea1) - Jesse Szwedko
* **releasing**: Prepare v0.29.0 release [4bf6805](answerbook/vector@4bf6805) - Jesse Szwedko
* **releasing**: Prepare v0.30.0 release [38c3f0b](answerbook/vector@38c3f0b) - Jesse Szwedko
* **releasing**: Regenerate Kubernetes manifests for 0.21.2 (vectordotdev#17108) [fd13d64](answerbook/vector@fd13d64) - GitHub
* **releasing**: Regenerate manifests for 0.21.1 chart (vectordotdev#17187) [1f0de6b](answerbook/vector@1f0de6b) - GitHub
* **releasing**: Regenerate manifests for 0.22.0 chart (vectordotdev#17135) [e7ea0a8](answerbook/vector@e7ea0a8) - GitHub
* **releasing**: update patch release template with extra step details [27c3526](answerbook/vector@27c3526) - GitHub
* Remove skaffold from project (vectordotdev#17145) [d245927](answerbook/vector@d245927) - GitHub
* remove transform type coercion (vectordotdev#17411) [b6c7e0a](answerbook/vector@b6c7e0a) - GitHub
* Revert transform definitions (vectordotdev#17146) [05a3f44](answerbook/vector@05a3f44) - GitHub
* **socket source**: Remove deprecated `max_length` setting from `tcp` and `unix` modes. (vectordotdev#17162) [9ecfc8c](answerbook/vector@9ecfc8c) - GitHub
* **syslog source**: remove the remove of source_ip (vectordotdev#17184) [5dff0ed](answerbook/vector@5dff0ed) - GitHub
* **topology**: Transform outputs hash table of OutputId -> Definition (vectordotdev#17059) [1bdb24d](answerbook/vector@1bdb24d) - GitHub
* Upgrade `VRL` to `0.3.0` (vectordotdev#17325) [4911d36](answerbook/vector@4911d36) - GitHub

### Features

* adds 'metric_name' field to internal logs for the tag_cardinality_limit component (vectordotdev#17295) [4317340](answerbook/vector@4317340) - GitHub
* **codecs**: Add full codec support to AWS S3 source/sink (vectordotdev#17098) [d648c86](answerbook/vector@d648c86) - GitHub
* **kubernetes_logs**: use kube-apiserver cache for list requests (vectordotdev#17095) [e46fae7](answerbook/vector@e46fae7) - GitHub
* Merge with upstream v0.30.0 [6b93177](answerbook/vector@6b93177) - GitHub [LOG-17997](https://logdna.atlassian.net/browse/LOG-17997)
* **new sink**: Initial `datadog_events` sink (vectordotdev#7678) [fef3810](answerbook/vector@fef3810) - Jesse Szwedko
* Update VRL library [6ace1e6](answerbook/vector@6ace1e6) - Jorge Bay

### Miscellaneous

* Merge branch 'master' [d4b35bb](answerbook/vector@d4b35bb) - Jorge Bay
* Merge tag 'v0.30.0' into update-upstream [ee2f300](answerbook/vector@ee2f300) - Jorge Bay
* Merge commit '9031d0faa2811187874364e1b5a3305c9ed0c0da' into update-upstream [c12faae](answerbook/vector@c12faae) - Jorge Bay
* Improve tokio::select behavior for kafka source and finalizers (vectordotdev#17380) [d4df21c](answerbook/vector@d4df21c) - GitHub
* Prepare v0.29.1 release [21beed7](answerbook/vector@21beed7) - Kyle Criddle
* Add a quickfix to handle special capitalization cases (vectordotdev#17141) [ba63e21](answerbook/vector@ba63e21) - GitHub
* Adjust doc comment locations (vectordotdev#17154) [730c938](answerbook/vector@730c938) - GitHub
* **amqp sink**: Support AMQ Properties (content-type) in AMQP sink (vectordotdev#17174) [c10d30b](answerbook/vector@c10d30b) - GitHub
* **aws provider**: Let `region` be configured for default authentication (vectordotdev#17414) [c81ad30](answerbook/vector@c81ad30) - GitHub
* **core**: add unit test (ignored) for support for encoding special chars in `ProxyConfig` (vectordotdev#17148) [5247972](answerbook/vector@5247972) - GitHub
* **loki sink**: Fix formatting in labels example (vectordotdev#17396) [f3734e8](answerbook/vector@f3734e8) - GitHub
* **observability**: Log underlying error for unhandled HTTP errors (vectordotdev#17327) [bf8376c](answerbook/vector@bf8376c) - GitHub
* **observability**: Update internal log rate limiting messages (vectordotdev#17394) [1951535](answerbook/vector@1951535) - GitHub
* **pulsar sink**: Refactor to use StreamSink (vectordotdev#14345) [1e97a2f](answerbook/vector@1e97a2f) - GitHub
* **topology**: Add source id to metadata (vectordotdev#17369) [c683999](answerbook/vector@c683999) - GitHub
* **vdev**: Load compose files and inject network block (vectordotdev#17025) [5d88655](answerbook/vector@5d88655) - GitHub
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ci-condition: integration tests enable Run integration tests on this PR domain: ci Anything related to Vector's CI environment domain: external docs Anything related to Vector's external, public documentation domain: sinks Anything related to the Vector's sinks sink: pulsar Anything `pulsar` sink related
Projects
None yet
Development

Successfully merging this pull request may close these issues.

9 participants