Kong - multi process with Prometheus

[xluffy]

Kong Gateway is built on top of OpenResty/Nginx, which is a multi-process single-threaded architecture. To collect and aggregate metrics from different processes, we implement the Prometheus plugin with shared memory.

Nginx handles requests in a non-blocking way and is normally very efficient. However, every read and write operation to the shared memory requires a mutex lock to lock the critical section and will block all worker processes from processing requests. When the plugin is used to monitor metrics with high cardinality, it can affect Kong Gateway performance significantly, especially increasing the long tail latencies

https://konghq.com/blog/engineering/how-to-use-prometheus-to-monitor-kong-gateway

Nice discussion here, must read: Kong/kong#8844

[xluffy]

prometheus/client_ruby#9 same issue

https://github.com/prometheus/client_ruby/pull/95/files solution

[xluffy]

https://gitlab.com/gitlab-org/ruby/gems/prometheus-client-mmap

This Prometheus library is fork of Prometheus Ruby Client that uses mmap'ed files to share metrics from multiple processes. This allows efficient metrics processing for Ruby web apps running in multiprocess setups like Unicorn.