Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add gnostats, a node cluster data visualizer #2666

Draft
wants to merge 6 commits into
base: master
Choose a base branch
from

Conversation

zivkovicmilos
Copy link
Member

Description

This PR introduces gnostats, a new tool for quickly gathering node cluster data, and visualizing it on a dashboard.

TODO

Contributors' checklist...
  • Added new tests, or not needed, or not feasible
  • Provided an example (e.g. screenshot) to aid review or the PR is self-explanatory
  • Updated the official documentation or not needed
  • No breaking changes were made, or a BREAKING CHANGE: xxx message was included in the description
  • Added references to related issues and PRs
  • Provided any useful hints for running manual tests
  • Added new benchmarks to generated graphs, if any. More info here.

@zivkovicmilos zivkovicmilos added the 📦 ⛰️ gno.land Issues or PRs gno.land package related label Aug 7, 2024
@zivkovicmilos zivkovicmilos self-assigned this Aug 7, 2024
Copy link

codecov bot commented Aug 7, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 60.15%. Comparing base (5b768db) to head (e239985).
Report is 21 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #2666      +/-   ##
==========================================
- Coverage   60.16%   60.15%   -0.01%     
==========================================
  Files         561      562       +1     
  Lines       74999    75455     +456     
==========================================
+ Hits        45122    45393     +271     
- Misses      26502    26663     +161     
- Partials     3375     3399      +24     
Flag Coverage Δ
contribs/gnodev 61.40% <ø> (ø)
contribs/gnofaucet 14.46% <ø> (-0.86%) ⬇️
gno.land 64.57% <ø> (-0.18%) ⬇️
gnovm 64.33% <ø> (+0.19%) ⬆️
misc/genstd 80.54% <ø> (ø)
misc/logos 19.88% <ø> (-0.36%) ⬇️
tm2 62.04% <ø> (-0.02%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@aeddi
Copy link
Contributor

aeddi commented Aug 13, 2024

Following up here on our discussion on Slack, I just thought of something: should we implement an authentication mechanism for any node that push stat to the hub using its ID keypair?

For example, through a handshake that would prove that the node's identity is not spoofed. I see two possible simple implementations to set this up:

  • Either an additional method on the hub service that would return a value (like a nonce or maybe something more complex) to be signed with the node's ID keypair, added as a parameter during the call to the register method and verified by the hub using the node's ID public key.

  • Or by using the node's ID keypair during the TLS handshake used to encrypt the gRPC connection (similar to what libp2p does, which uses the peerID when establishing an encrypted connection via TLS or Secio).

What do you think?

@zivkovicmilos
Copy link
Member Author

Following up here on our discussion on Slack, I just thought of something: should we implement an authentication mechanism for any node that push stat to the hub using its identity keypair?

For example, through a handshake that would prove that the node's identity is not spoofed. I see two possible simple implementations to set this up:

  • Either an additional method on the hub service that would return a value (like a nonce or maybe something more complex) to be signed with the node's public key and added as a parameter during the call to the register method.
  • Or by using the node's keypair during the TLS handshake used to encrypt the gRPC connection (similar to what libp2p does, which uses the peerID when establishing an encrypted connection via TLS or Secio).

What do you think?

I've thought about this authentication 👀

The ultimate case we're gonna have is you as the operator can start the gnostats server with "predefined" nodes you expect to have. This method is not super dynamic, as the server would need to be aware in advance of what nodes to expect.
I'm also fine with having a token system, a secret shared between the gnostats hub and client (env), this would make it more dynamic. What do you think about this approach?

I like the solution with nonce signing (I've found these systems to work in the past fine), but it would require the gnostats client to have some kind of keypair to operate with, which is not the case now -- it shouldn't reuse the node's keys in any way, it would need to generate a new one

@aeddi
Copy link
Contributor

aeddi commented Aug 13, 2024

I'm not sure yet on how certain things work on Gno. For example, I don't know if the moniker is completely arbitrary or if it is registered/reserved somewhere.

And does the address in the NodeData message of the gnostats node (here) correspond in any way to the keypair used by the Gno node (for example, a hash of its public key or something like that)?

If so, then personally, I would find it better to prove that the gnostats node is indeed in possession of the associated private key by, for example, signing a random value provided by the hub. And if it is impossible to sign a value with the Gno keypair from the gnostats binary, would it be possible for Gno to generate a subkey for this purpose beforehand? And for the hub to be able to verify the association between the subkey and the master key to authenticate the node?

On the other hand, if the address in the NodeData message is just an UUID or a network address, then indeed, there is no cryptographic identity to prove, so I think that a shared secret would be enough.

## Description

This PR introduces the `gnostats` server implementation, for data
streaming.

<details><summary>Contributors' checklist...</summary>

- [x] Added new tests, or not needed, or not feasible
- [x] Provided an example (e.g. screenshot) to aid review or the PR is
self-explanatory
- [x] Updated the official documentation or not needed
- [x] No breaking changes were made, or a `BREAKING CHANGE: xxx` message
was included in the description
- [x] Added references to related issues and PRs
- [ ] Provided any useful hints for running manual tests
- [ ] Added new benchmarks to [generated
graphs](https://gnoland.github.io/benchmarks), if any. More info
[here](https://github.com/gnolang/gno/blob/master/.benchmarks/README.md).
</details>
@aeddi aeddi mentioned this pull request Aug 16, 2024
7 tasks
<!-- please provide a detailed description of the changes made in this
pull request. -->

This PR introduces the gnostats agent that acts as a bridge between the
Gno node RPC endpoint and gnostats Hub.

Part of #2666.

<details><summary>Contributors' checklist...</summary>

- [x] Added new tests, or not needed, or not feasible
- [x] Provided an example (e.g. screenshot) to aid review or the PR is
self-explanatory
- [x] Updated the official documentation or not needed
- [x] No breaking changes were made, or a `BREAKING CHANGE: xxx` message
was included in the description
- [x] Added references to related issues and PRs
- [ ] Provided any useful hints for running manual tests
- [ ] Added new benchmarks to [generated
graphs](https://gnoland.github.io/benchmarks), if any. More info
[here](https://github.com/gnolang/gno/blob/master/.benchmarks/README.md).
</details>
@moul
Copy link
Member

moul commented Sep 21, 2024

gnolang/gnostats-ui#1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
📦 ⛰️ gno.land Issues or PRs gno.land package related
Projects
Status: No status
Status: In Progress
Development

Successfully merging this pull request may close these issues.

3 participants