Certificate verify failing when using a proxy for the vector sink

[joemiller]

A note for the community

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Problem

Attempting to use a CONNECT proxy with the vector sink results in certificate verification error like:

error trying to connect: error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:ssl/statem/statem_clnt.c:1921

Configuration

The relevant part of the config is:

    proxy:
      enabled: true
      https: https://proxy.dom.tld:8443

    sinks:
      vector_aggregator:
        type: vector
        inputs:
          - add_env_vars
        address: vector-aggregator.vector.svc.cluster.local:6000
        tls:
          enabled: true
          verify_certificate: false
          verify_hostname: false
          ca_file: /tls/ca.crt
          key_file: /tls/tls.key
          crt_file: /tls/tls.crt

Version

0.30.0

Additional Context

I am able to successfully connect through the proxy using curl.

I believe the proxy side (envoy) is working as I see successful proxy messages in the envoy logs.

Interestingly, when I setup a dummy http sink that targets the same URL I do not receive tls verification errors. This leads me to suspect the http sink's tls-config is being used by the global proxy config, but the same is not the case for the vector sink.

References

These issues seem similar, if I understand them correctly. Is this a case of needing to port over a similar change to the vector sink? I am weak with rust and not able to read the code well enough to determine a path forward at this time.

#13683
#13759

[joemiller]

Attempt to address this issue here: #17629

[neuronull]

Thanks @joemiller , we'll get this looked at. Appreciate the added effort to provide a solution!