rate limit panic

[msarajam]

the Envoy is panicing when any client is trying to connect to it , it is happening after adding rate_limit to the yaml file :

this is the part that i added :

filter_chains:
  filters:
  - name: envoy.ratelimit
    config:
      domain: rate_limit_test
      stat_prefix: "/rate_limit"
      timeout: 2s
      failure_mode_deny: true
      descriptors: 
      - entries:
        - key: "path"
          value: "/not_allow"

after this change , envoy started with no problem , but when i tried to call "localhost/not_allow" , it paniced :

[2019-03-21 19:04:22.477][16][critical][assert] [source/common/grpc/async_client_manager_impl.cc:89] panic: not reached
[2019-03-21 19:04:22.477][16][critical][backtrace] [bazel-out/k8-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:125] Caught Aborted, suspect faulting address 0x1
[2019-03-21 19:04:22.477][16][critical][backtrace] [bazel-out/k8-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:94] Backtrace thr<16> obj</lib/x86_64-linux-gnu/libc.so.6> (If unsymbolized, use tools/stack_decode.py):
[2019-03-21 19:04:22.478][16][critical][backtrace] [bazel-out/k8-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:117] thr<16> #0 0x7fbf1d8a7428 (unknown)
[2019-03-21 19:04:22.478][16][critical][backtrace] [bazel-out/k8-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:117] thr<16> #1 0x7fbf1d8a9029 (unknown)
[2019-03-21 19:04:22.478][16][critical][backtrace] [bazel-out/k8-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:104] thr<16> obj
[2019-03-21 19:04:22.478][16][critical][backtrace] [bazel-out/k8-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:117] thr<16> #2 0x83d686 (unknown)
[2019-03-21 19:04:22.478][16][critical][backtrace] [bazel-out/k8-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:117] thr<16> #3 0x7b0823 (unknown)
[2019-03-21 19:04:22.478][16][critical][backtrace] [bazel-out/k8-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:117] thr<16> #4 0x60a98c (unknown)
[2019-03-21 19:04:22.478][16][critical][backtrace] [bazel-out/k8-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:117] thr<16> #5 0x7adcef (unknown)
[2019-03-21 19:04:22.478][16][critical][backtrace] [bazel-out/k8-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:117] thr<16> #6 0x7d2d39 (unknown)
[2019-03-21 19:04:22.478][16][critical][backtrace] [bazel-out/k8-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:117] thr<16> #7 0x7d301b (unknown)
[2019-03-21 19:04:22.478][16][critical][backtrace] [bazel-out/k8-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:117] thr<16> #8 0x7d220e (unknown)
[2019-03-21 19:04:22.478][16][critical][backtrace] [bazel-out/k8-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:117] thr<16> #9 0x7e16b0 (unknown)
[2019-03-21 19:04:22.478][16][critical][backtrace] [bazel-out/k8-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:117] thr<16> #10 0xb363c7 (unknown)
[2019-03-21 19:04:22.478][16][critical][backtrace] [bazel-out/k8-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:117] thr<16> #11 0xb28c14 (unknown)
[2019-03-21 19:04:22.478][16][critical][backtrace] [bazel-out/k8-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:117] thr<16> #12 0xb2935e (unknown)
[2019-03-21 19:04:22.478][16][critical][backtrace] [bazel-out/k8-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:117] thr<16> #13 0xb2c0e7 (unknown)
[2019-03-21 19:04:22.479][16][critical][backtrace] [bazel-out/k8-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:117] thr<16> #14 0x7d4662 (unknown)
[2019-03-21 19:04:22.479][16][critical][backtrace] [bazel-out/k8-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:117] thr<16> #15 0x7d0968 (unknown)
[2019-03-21 19:04:22.479][16][critical][backtrace] [bazel-out/k8-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:117] thr<16> #16 0xdcabc4 (unknown)
[2019-03-21 19:04:22.479][16][critical][backtrace] [bazel-out/k8-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:104] thr<16> obj</lib/x86_64-linux-gnu/libpthread.so.0>
[2019-03-21 19:04:22.479][16][critical][backtrace] [bazel-out/k8-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<16> #17 0x7fbf1df4c6b9 start_thread
[2019-03-21 19:04:22.479][16][critical][backtrace] [bazel-out/k8-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:104] thr<16> obj</lib/x86_64-linux-gnu/libc.so.6>
[2019-03-21 19:04:22.479][16][critical][backtrace] [bazel-out/k8-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:117] thr<16> #18 0x7fbf1d97941c (unknown)
[2019-03-21 19:04:22.479][16][critical][backtrace] [bazel-out/k8-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:121] end backtrace thread 16
[2019-03-21 19:04:22.479][16][critical][backtrace] [bazel-out/k8-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:125] Caught Segmentation fault, suspect faulting address 0x0
[2019-03-21 19:04:22.479][16][critical][backtrace] [bazel-out/k8-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:94] Backtrace thr<16> obj</lib/x86_64-linux-gnu/libc.so.6> (If unsymbolized, use tools/stack_decode.py):
[2019-03-21 19:04:22.479][16][critical][backtrace] [bazel-out/k8-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:117] thr<16> #0 0x7fbf1d8a9196 (unknown)
[2019-03-21 19:04:22.479][16][critical][backtrace] [bazel-out/k8-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:104] thr<16> obj
[2019-03-21 19:04:22.479][16][critical][backtrace] [bazel-out/k8-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:117] thr<16> #1 0x83d686 (unknown)
[2019-03-21 19:04:22.479][16][critical][backtrace] [bazel-out/k8-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:117] thr<16> #2 0x7b0823 (unknown)
[2019-03-21 19:04:22.479][16][critical][backtrace] [bazel-out/k8-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:117] thr<16> #3 0x60a98c (unknown)
[2019-03-21 19:04:22.479][16][critical][backtrace] [bazel-out/k8-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:117] thr<16> #4 0x7adcef (unknown)
[2019-03-21 19:04:22.479][16][critical][backtrace] [bazel-out/k8-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:117] thr<16> #5 0x7d2d39 (unknown)
[2019-03-21 19:04:22.479][16][critical][backtrace] [bazel-out/k8-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:117] thr<16> #6 0x7d301b (unknown)
[2019-03-21 19:04:22.479][16][critical][backtrace] [bazel-out/k8-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:117] thr<16> #7 0x7d220e (unknown)
[2019-03-21 19:04:22.479][16][critical][backtrace] [bazel-out/k8-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:117] thr<16> #8 0x7e16b0 (unknown)
[2019-03-21 19:04:22.479][16][critical][backtrace] [bazel-out/k8-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:117] thr<16> #9 0xb363c7 (unknown)
[2019-03-21 19:04:22.479][16][critical][backtrace] [bazel-out/k8-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:117] thr<16> #10 0xb28c14 (unknown)
[2019-03-21 19:04:22.479][16][critical][backtrace] [bazel-out/k8-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:117] thr<16> #11 0xb2935e (unknown)
[2019-03-21 19:04:22.480][16][critical][backtrace] [bazel-out/k8-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:117] thr<16> #12 0xb2c0e7 (unknown)
[2019-03-21 19:04:22.480][16][critical][backtrace] [bazel-out/k8-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:117] thr<16> #13 0x7d4662 (unknown)
[2019-03-21 19:04:22.480][16][critical][backtrace] [bazel-out/k8-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:117] thr<16> #14 0x7d0968 (unknown)
[2019-03-21 19:04:22.480][16][critical][backtrace] [bazel-out/k8-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:117] thr<16> #15 0xdcabc4 (unknown)
[2019-03-21 19:04:22.480][16][critical][backtrace] [bazel-out/k8-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:104] thr<16> obj</lib/x86_64-linux-gnu/libpthread.so.0>
[2019-03-21 19:04:22.480][16][critical][backtrace] [bazel-out/k8-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:114] thr<16> #16 0x7fbf1df4c6b9 start_thread
[2019-03-21 19:04:22.480][16][critical][backtrace] [bazel-out/k8-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:104] thr<16> obj</lib/x86_64-linux-gnu/libc.so.6>
[2019-03-21 19:04:22.480][16][critical][backtrace] [bazel-out/k8-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:117] thr<16> #17 0x7fbf1d97941c (unknown)
[2019-03-21 19:04:22.480][16][critical][backtrace] [bazel-out/k8-opt/bin/source/server/_virtual_includes/backtrace_lib/server/backtrace.h:121] end backtrace thread 16

[mattklein123]

I think this is because you have not setup an upstream rate limit server, and you are probably using a build that is an intermediate state with no required annotations. Can you try current master and I think you will get an error?

[msarajam]

I'lll try that , thanks

[stale]

This issue has been automatically marked as stale because it has not had activity in the last 30 days. It will be closed in the next 7 days unless it is tagged "help wanted" or other activity occurs. Thank you for your contributions.

[stale]

This issue has been automatically closed because it has not had activity in the last 37 days. If this issue is still valid, please ping a maintainer and ask them to label it as "help wanted". Thank you for your contributions.