feat: export rcmgr metrics to prometheus #8785
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
vyzo
approved these changes
Mar 11, 2022
marten-seemann
force-pushed
the
update-libp2p-v018
branch
from
5a98324 to
d23a188
Compare
marten-seemann
force-pushed
the
rcmgr-metrics
branch
2 times, most recently
from
f9ff9d1 to
18b9d31
Compare
16 tasks
20 tasks
|
@marten-seemann : just checking my understanding, but it looks like one sharness test is not passing: https://app.circleci.com/pipelines/github/ipfs/go-ipfs/6284/workflows/1dc038d2-dd08-442e-93e5-30512e10193d/jobs/68339 I assume:
|
69 tasks
|
Yes, that test checks all exported metrics against a list, and fails if any is missing / not expected. I was planning to fix this later. Note: once libp2p gets a coherent metrics story (see libp2p/go-libp2p#1356), this test should probably be modified to exclude libp2p metrics. I'll leave that decision to the IPFS stewards though. |
marten-seemann
force-pushed
the
update-libp2p-v018
branch
from
33fbdbf to
26b8c42
Compare
6 tasks
|
@marten-seemann : please comment/ping when the test is updated/passing. Also, I assume we need to update so that this PR only shows the incremental diff on top of #8680 ? I'll make sure we then get reviewer eyes to land this. |
lidel
force-pushed
the
update-libp2p-v018
branch
from
26b8c42 to
859e648
Compare
lidel
changed the title
lidel
approved these changes
Apr 5, 2022
There was a problem hiding this comment.
LGTM, thank you!
- Rebased this on top of feat: opt-in Swarm.ResourceMgr (go-libp2p v0.18) #8680
- Refactored a bit to ensure these metrics are created and exposed only when
Swarm.ResourceMgr.Enabled
guseggert
pushed a commit
that referenced
this pull request
Apr 8, 2022
* update go-libp2p to v0.18.0
* initialize the resource manager
* add resource manager stats/limit commands
* load limit file when building resource manager
* log absent limit file
* write rcmgr to file when IPFS_DEBUG_RCMGR is set
* fix: mark swarm limit|stats as experimental
* feat(cfg): opt-in Swarm.ResourceMgr
This ensures we can safely test the resource manager without impacting
default behavior.
- Resource manager is disabled by default
- Default for Swarm.ResourceMgr.Enabled is false for now
- Swarm.ResourceMgr.Limits allows user to tweak limits per specific
scope in a way that is persisted across restarts
- 'ipfs swarm limit system' outputs human-readable json
- 'ipfs swarm limit system new-limits.json' sets new runtime limits
(but does not change Swarm.ResourceMgr.Limits in the config)
Conventions to make libp2p devs life easier:
- 'IPFS_RCMGR=1 ipfs daemon' overrides the config and enables resource manager
- 'limit.json' overrides implicit defaults from libp2p (if present)
* docs(config): small tweaks
* fix: skip libp2p.ResourceManager if disabled
This ensures 'ipfs swarm limit|stats' work only when enabled.
* fix: use NullResourceManager when disabled
This reverts commit b19f7c9.
after clarification feedback from
#8680 (comment)
* style: rename IPFS_RCMGR to LIBP2P_RCMGR
preexisting libp2p toggles use LIBP2P_ prefix
* test: Swarm.ResourceMgr
* fix: location of opt-in limit.json and rcmgr.json.gz
Places these files inside of IPFS_PATH
* Update docs/config.md
* feat: expose rcmgr metrics when enabled (#8785)
* add metrics for the resource manager
* export protocol and service name in Prometheus metrics
* fix: expose rcmgr metrics only when enabled
Co-authored-by: Marcin Rataj <lidel@lidel.org>
* refactor: rcmgr_metrics.go
* refactor: rcmgr_defaults.go
This file defines implicit limit defaults used when Swarm.ResourceMgr.Enabled
We keep vendored copy to ensure go-ipfs is not impacted when go-libp2p
decides to change defaults in any of the future releases.
* refactor: adjustedDefaultLimits
Cleans up the way we initialize defaults and adds a fix for case
when connection manager runs with high limits.
It also hides `Swarm.ResourceMgr.Limits` until we have a better
understanding what syntax makes sense.
* chore: cleanup after a review
* fix: restore go-ipld-prime v0.14.2
* fix: restore go-ds-flatfs v0.5.1
Co-authored-by: Lucas Molas <schomatis@gmail.com>
Co-authored-by: Marcin Rataj <lidel@lidel.org>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Part of #8761
Adds basic metrics under
libp2p_rcmgr_*Demo sample