feat(p2p): add native libp2p debug metrics #1795

derrandz · 2023-02-21T15:23:04Z

Overview

This PR introduces libp2p metrics. The metrics that are introduced in this PR are plentiful, and it's sufficient to list the libp2p components they cover:

Resource Manager
AutoNat
Identify
EventBus
Swarm

(check the following link)

Usage

Run your node with flag:

$ celestia light start ... --metrics --metrics.endpoint ... --p2p.metrics

Results

Dashboards:

Checklist

New and updated code has appropriate documentation
New and updated code has new and/or updated testing
Required CI checks are passing
Visual proof for any user facing features like CLI or documentation updates
Linked issues closed with keywords

Wondertan

Initial quick look

nodebuilder/settings.go

nodebuilder/p2p/module.go

codecov-commenter · 2023-03-17T18:55:43Z

Codecov Report

Merging #1795 (c9471d3) into main (f18b1ab) will decrease coverage by 0.33%.
The diff coverage is 14.96%.

@@            Coverage Diff             @@
##             main    #1795      +/-   ##
==========================================
- Coverage   54.95%   54.63%   -0.33%     
==========================================
  Files         216      216              
  Lines       13925    14009      +84     
==========================================
+ Hits         7653     7654       +1     
- Misses       5460     5537      +77     
- Partials      812      818       +6

Impacted Files	Coverage Δ
cmd/celestia/main.go	`47.05% <ø> (ø)`
nodebuilder/node/admin.go	`26.66% <0.00%> (ø)`
nodebuilder/settings.go	`75.00% <0.00%> (-7.36%)`	⬇️
cmd/celestia/rpc.go	`7.03% <7.03%> (ø)`
nodebuilder/p2p/metrics.go	`11.42% <11.42%> (ø)`
share/eds/eds.go	`67.19% <40.00%> (-1.87%)`	⬇️
cmd/flags_misc.go	`48.40% <46.66%> (-0.19%)`	⬇️
api/rpc/client/client.go	`72.72% <100.00%> (-9.63%)`	⬇️
core/testing.go	`91.86% <100.00%> (+0.09%)`	⬆️
nodebuilder/p2p/config.go	`81.08% <100.00%> (+0.52%)`	⬆️
... and 2 more

... and 6 files with indirect coverage changes

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

…metrics/p2p

walldiss

Could you please elaborate what is ERROR level metrics and how it would be used? It feel counterintuitive that ERROR level is below DEBUG, when in logging it is the highest.

cmd/flags_misc.go

nodebuilder/p2p/metrics.go

cmd/flags_misc.go

Wondertan

LGTM. Thank you

cmd/flags_misc.go

nodebuilder/p2p/config.go

nodebuilder/p2p/metrics.go

nodebuilder/settings.go

Wondertan

There is a new example for metrics usage in libp2p libp2p/go-libp2p#2232
Let's double-check it to ensure we didn't miss anything.

cmd/flags_misc.go

nodebuilder/p2p/metrics.go

derrandz · 2023-04-03T16:39:50Z

@Wondertan I've checked on libp2p/go-libp2p#2232 and nothing seems to be new. We should be good.

renaynay

small nits, should be quick 🙏🏻

cmd/flags_misc.go

nodebuilder/p2p/metrics.go

addressed

feat: add p2p bandwidth metrics using opentelemetry primitives

be0a3fa

derrandz requested review from renaynay, Wondertan, vgonkivs and distractedm1nd as code owners February 21, 2023 15:23

derrandz mentioned this pull request Feb 21, 2023

feat(telemetry): add blackbox instrumentation to the header module + share module + p2p bandwidth metrics #1376

Closed

8 tasks

derrandz marked this pull request as draft February 21, 2023 16:13

derrandz changed the title ~~feat: add p2p bandwidth metrics using opentelemetry primitives~~ feat(p2p): add p2p bandwidth metrics using opentelemetry primitives + native libp2p debug metrics Feb 27, 2023

derrandz marked this pull request as ready for review February 27, 2023 23:30

derrandz requested a review from walldiss as a code owner February 27, 2023 23:30

Wondertan reviewed Mar 6, 2023

View reviewed changes

nodebuilder/settings.go Outdated Show resolved Hide resolved

nodebuilder/p2p/module.go Outdated Show resolved Hide resolved

nodebuilder/p2p/module.go Outdated Show resolved Hide resolved

derrandz added 5 commits March 7, 2023 16:57

feat: add p2p bandwidth metrics using opentelemetry primitives

7032d3f

feat: enable libp2p prometheus metrics as debug level metrics

a30d58a

fix: use fx.Decorate to replace resource manager

9887aab

Merge branch 'main' into metrics/p2p

6aea987

chore: make linter happy;

e1653e0

derrandz force-pushed the metrics/p2p branch from 48d0c5c to e1653e0 Compare March 7, 2023 16:06

Merge branch 'main' into metrics/p2p

d3f630c

derrandz requested a review from Wondertan March 17, 2023 15:07

derrandz added 3 commits March 17, 2023 16:23

pr(review): address comments

d5fe7f4

Merge branch 'main' into metrics/p2p

f77d1ba

Merge branch 'main' into metrics/p2p

307deba

derrandz added 2 commits March 30, 2023 01:19

Merge branch 'main' into metrics/p2p

22fea6d

Merge branch 'metrics/p2p' of github.com:derrandz/celestia-node into …

5ea9db5

…metrics/p2p

walldiss previously requested changes Mar 30, 2023

View reviewed changes

feat: use libp2p metrics only

dd91d7a

derrandz requested a review from walldiss March 30, 2023 15:33

derrandz changed the title ~~feat(p2p): add p2p bandwidth metrics using opentelemetry primitives + native libp2p debug metrics~~ feat(p2p): add native libp2p debug metrics Mar 30, 2023

Merge branch 'main' into metrics/p2p

5f8e6b2

derrandz force-pushed the metrics/p2p branch from e5ae7f3 to 6e209bc Compare March 30, 2023 15:48

derrandz added the kind:feat Attached to feature PRs label Mar 30, 2023

distractedm1nd previously requested changes Mar 30, 2023

View reviewed changes

cmd/flags_misc.go Outdated Show resolved Hide resolved

pr: address commentsg

45a8860

derrandz requested a review from distractedm1nd March 30, 2023 15:59

Wondertan previously approved these changes Mar 30, 2023

View reviewed changes

Merge branch 'main' into metrics/p2p

b0ce07c

renaynay reviewed Mar 31, 2023

View reviewed changes

pr: address rene's comments

29288d4

derrandz dismissed Wondertan’s stale review via 29288d4 March 31, 2023 22:54

Wondertan reviewed Apr 3, 2023

View reviewed changes

cmd/flags_misc.go Outdated Show resolved Hide resolved

cmd/flags_misc.go Outdated Show resolved Hide resolved

nodebuilder/p2p/metrics.go Show resolved Hide resolved

nodebuilder/p2p/metrics.go Outdated Show resolved Hide resolved

pr: address hlib's comments

66d3910

derrandz requested review from renaynay and Wondertan April 3, 2023 16:37

Merge branch 'main' into metrics/p2p

cf2689d

Wondertan previously approved these changes Apr 3, 2023

View reviewed changes

quickfix: do not enable p2p metrics when metrics are off

c9471d3

derrandz dismissed Wondertan’s stale review via c9471d3 April 3, 2023 17:40

renaynay reviewed Apr 3, 2023

View reviewed changes

cmd/flags_misc.go Show resolved Hide resolved

nodebuilder/p2p/metrics.go Outdated Show resolved Hide resolved

nodebuilder/p2p/metrics.go Outdated Show resolved Hide resolved

pr: address small nits

c9eecdc

renaynay approved these changes Apr 3, 2023

View reviewed changes

derrandz enabled auto-merge (squash) April 3, 2023 17:54

derrandz disabled auto-merge April 3, 2023 17:54

derrandz enabled auto-merge (squash) April 3, 2023 17:55

Wondertan approved these changes Apr 3, 2023

View reviewed changes

derrandz merged commit f5f5efb into celestiaorg:main Apr 3, 2023

renaynay added the kind:break! Attached to breaking PRs label Apr 5, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(p2p): add native libp2p debug metrics #1795

feat(p2p): add native libp2p debug metrics #1795

derrandz commented Feb 21, 2023 •

edited

Loading

Wondertan left a comment

codecov-commenter commented Mar 17, 2023 •

edited

Loading

walldiss left a comment

Wondertan left a comment

Wondertan left a comment

derrandz commented Apr 3, 2023 •

edited

Loading

renaynay left a comment

feat(p2p): add native libp2p debug metrics #1795

feat(p2p): add native libp2p debug metrics #1795

Conversation

derrandz commented Feb 21, 2023 • edited Loading

Overview

Usage

Results

Checklist

Wondertan left a comment

Choose a reason for hiding this comment

codecov-commenter commented Mar 17, 2023 • edited Loading

Codecov Report

walldiss left a comment

Choose a reason for hiding this comment

Wondertan left a comment

Choose a reason for hiding this comment

Wondertan left a comment

Choose a reason for hiding this comment

derrandz commented Apr 3, 2023 • edited Loading

renaynay left a comment

Choose a reason for hiding this comment

derrandz commented Feb 21, 2023 •

edited

Loading

codecov-commenter commented Mar 17, 2023 •

edited

Loading

derrandz commented Apr 3, 2023 •

edited

Loading