Clang msan support

[htuch]

Clang ASAN/TSAN support was introduced under #443 and #904.

MSAN isn't going to be easy, and questionably worth doing in our current CI environment. You basically need to rebuild lib[std]c++ and all external deps, see https://github.com/google/sanitizers/wiki/MemorySanitizerLibcxxHowTo. I tried it without this and I see false positives such as:

Uninitialized bytes in __interceptor_strlen at offset 18 inside [0x703000000168, 19)
==7==WARNING: MemorySanitizer: use-of-uninitialized-value
    #0 0x7fe5d4e5d5ce  (/google/src/files/151379707/depot/google3/third_party/crosstool/v18/llvm_unstable/toolchain/x86_64-grtev4-linux-gnu/lib64/libstdc++.so.6+0xc75ce)
    #1 0x44f11cb  (/usr/local/google/home/htuch/.cache/bazel/_bazel_htuch/89676793239ac96d94294e6c7a44597f/execroot/envoy/bazel-out/local-dbg/bin/test/config_test/example_configs_test+0x44f11cb)
    #2 0x4501720  (/usr/local/google/home/htuch/.cache/bazel/_bazel_htuch/89676793239ac96d94294e6c7a44597f/execroot/envoy/bazel-out/local-dbg/bin/test/config_test/example_configs_test+0x4501720)
    #3 0x44e7048  (/usr/local/google/home/htuch/.cache/bazel/_bazel_htuch/89676793239ac96d94294e6c7a44597f/execroot/envoy/bazel-out/local-dbg/bin/test/config_test/example_configs_test+0x44e7048)
    #4 0x427c0a  (/usr/local/google/home/htuch/.cache/bazel/_bazel_htuch/89676793239ac96d94294e6c7a44597f/execroot/envoy/bazel-out/local-dbg/bin/test/config_test/example_configs_test+0x427c0a)
    #5 0x427e91  (/usr/local/google/home/htuch/.cache/bazel/_bazel_htuch/89676793239ac96d94294e6c7a44597f/execroot/envoy/bazel-out/local-dbg/bin/test/config_test/example_configs_test+0x427e91)
    #6 0x452d97d  (/usr/local/google/home/htuch/.cache/bazel/_bazel_htuch/89676793239ac96d94294e6c7a44597f/execroot/envoy/bazel-out/local-dbg/bin/test/config_test/example_configs_test+0x452d97d)
    #7 0x7fe5d3db4cfe  (/usr/grte/v4/lib64/libc.so.6+0x38cfe)
    #8 0x639438  (/usr/local/google/home/htuch/.cache/bazel/_bazel_htuch/89676793239ac96d94294e6c7a44597f/execroot/envoy/bazel-out/local-dbg/bin/test/config_test/example_configs_test+0x639438)

  Uninitialized value was created by a heap allocation
    #0 0x6a86e9  (/usr/local/google/home/htuch/.cache/bazel/_bazel_htuch/89676793239ac96d94294e6c7a44597f/execroot/envoy/bazel-out/local-dbg/bin/test/config_test/example_configs_test+0x6a86e9)
    #1 0x7fe5d4e5d668  (/google/src/files/151379707/depot/google3/third_party/crosstool/v18/llvm_unstable/toolchain/x86_64-grtev4-linux-gnu/lib64/libstdc++.so.6+0xc7668)
    #2 0x427c0a  (/usr/local/google/home/htuch/.cache/bazel/_bazel_htuch/89676793239ac96d94294e6c7a44597f/execroot/envoy/bazel-out/local-dbg/bin/test/config_test/example_configs_test+0x427c0a)
    #3 0x427e91  (/usr/local/google/home/htuch/.cache/bazel/_bazel_htuch/89676793239ac96d94294e6c7a44597f/execroot/envoy/bazel-out/local-dbg/bin/test/config_test/example_configs_test+0x427e91)
    #4 0x452d97d  (/usr/local/google/home/htuch/.cache/bazel/_bazel_htuch/89676793239ac96d94294e6c7a44597f/execroot/envoy/bazel-out/local-dbg/bin/test/config_test/example_configs_test+0x452d97d)
    #5 0x7fe5d3db4cfe  (/usr/grte/v4/lib64/libc.so.6+0x38cfe)
    #6 0x639438  (/usr/local/google/home/htuch/.cache/bazel/_bazel_htuch/89676793239ac96d94294e6c7a44597f/execroot/envoy/bazel-out/local-dbg/bin/test/config_test/example_configs_test+0x639438)

SUMMARY: MemorySanitizer: use-of-uninitialized-value (/google/src/files/151379707/depot/google3/third_party/crosstool/v18/llvm_unstable/toolchain/x86_64-grtev4-linux-gnu/lib64/libstdc++.so.6+0xc75ce) 

We should consider adding this when we have better CI resourcing for dynamically rebuilding all libraries (so we don't need to bloat the Docker CI image).

[dnoe]

We should consider doing this again now that we are under CNCF and have access to more CI resources. MSAN has found a few bugs internally that would have been caught by CI before merging.

[keith]

It seems like a lot has changed since this issue originated, I see some configuration for msan on CI and a few bazel configs for it, has anyone tried this recently? I'm curious about how many false positives we would find

[keith]

It looks like at this point libc++ is setup correctly for this at least envoyproxy/envoy-build-tools@f741613

[github-actions]

This issue has been automatically marked as stale because it has not had activity in the last 30 days. It will be closed in the next 7 days unless it is tagged "help wanted" or "no stalebot" or other activity occurs. Thank you for your contributions.

[keith]

Given this test PR #20680 I think we could probably enable this on CI, starting with an opt out list of places that need to be investigated, but it might be too many violations to be ready for prime time.

[lizan]

#21348