Track and measure number of Brave browser IPFS nodes #24
Comments
This could be somewhat tricky. Collecting peer records from the DHT like that will be painful once the hydra nodes are spun down. This process seems to be starting soon https://discuss.ipfs.tech/t/dht-hydra-peers-dialling-down-non-bridging-functionality-on-2022-12-01/15567 and (given that the linked proposal is a protocol violation) hopefully the hydras will be fully dead soon. In a hydra-less world some options include:
IIRC there are already some metrics collected for number of total peerIDs seen by infra nodes. This seems like a) a good time to audit those metrics for accuracy b) like it'd be easy enough to either export more information than just peerIDs or to take the peerIDs and feed them into |
|
@aschmahmann - collecting peer records has been done long before the hydras exist - and AFAIK the hydras actually don't collect this data at all anyway. This already exists. |
@momack2 the existing data collection is what I meant by the above. IIRC it comes from aggregates of PL nodes https://github.com/ipfs/kubo/blob/master/plugin/plugins/peerlog/peerlog.go, although I don't recall which nodes participate in the logging. I recall seeing results in grafana and kibana although I'm not sure if they're the same. I was mostly flagging that the proposal of scraping the DHT will not really work, and without leveraging the hydras you'd need a different approach, like the one we already have.
Yep, what we already have is likely fine (scraping inbound peerIDs and user agents). Although when tracking and reporting these numbers we should be clear how we're collecting them. |
|
Yep! just want to clarify that the hydras don't participate in this, and it will not change with the hydras being turned down. 👍 |
|
After a brief discussion with @dennis-tra he brought up the idea of the honeypot deployed for the NAT Hole Punching study being able to help with this. Indeed, the honeypot is acting as a DHT server and also makes itself known to others so that it can attract more clients: https://github.com/dennis-tra/punchr#honeypot. It's also running for a few months now, so information might already be there in the logs? I've monitored the "Remote Peer Agent Version" reported at: https://punchr.dtrautwein.eu/grafana/d/F8qg0DP7k/punchr-performance?orgId=1 for a while, but haven't seen any @dennis-tra is the honeypot, or any of our other tools able to get this information? |
|
Yeah, I was thinking that we could use data from the punchr honeypot but I just had a brief look at the data and was searching for the
However, I just fired up the brave built-in kubo node and requested its AgentVersion + supported protocols. This is the output: You can see that the reported agent is |
|
Interesting! Thanks for digging in. So, good news and bad news:
@dennis-tra if we had an identifier (e.g., if they indeed advertised as @lidel @autonome can you double-check with Brave what agent version they're advertising? |
|
I think there must be a way to extrapolate the number incoming connections to the whole network - perhaps based on the in-degree of the honeypot, which we could get from the crawls. But this requires a bit of thinking 🤔 |
|
Lidel flagged there was a bug on this, which will be fixed soon, so you
should get real metrics shortly!
…On Thu, Dec 8, 2022 at 6:46 AM Dennis Trautwein ***@***.***> wrote:
I think there must be a way to extrapolate the number incoming connections
to the whole network - perhaps based on the in-degree of the honeypot,
which we could get from the crawls.
But this requires a bit of thinking 🤔
—
Reply to this email directly, view it on GitHub
<#24 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAEXAFYJGAMLAIO6KYN7KQLWMHKATANCNFSM6AAAAAASNX6Y4U>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
|
Where Brave implemented this: brave/brave-browser#18505 |
|
The suffix was not applied to the user agent exposed via libp2p identify protocol 🙈 We need to wait for Brave to update to Kubo 0.18 to see this. Good news is that they have reliable automatic update mechanism, similar to IPFS Desktop, so no other action is required on their end. |
|
Quick update from our side. With the dashboard from our collaborators, we see the following numbers for the last 7 days:
This shows an estimated number of around ~300 brave nodes for the last 7 days. This looks way too low to us. Our honeypot component (which arguably only covers a tiny bit of the keyspace) shows the following numbers for the last 7 days:
This shows the number of unique PeerIDs that have connected to the honeypot with a |
|
Just to add that the thinking here is to extrapolate from the portion of the key space that the honeypot is covering to the entire network. Of course, this assumes that there is a uniform distribution of requests across the key space, which isn't very accurate, but gives us a ballpark. The infrastructure we want to build (that Dennis mentions above) is to have more nodes and cover more of the key space. From that we should get a much more accurate view of the number of |
|
I know we don't have visibility/extrapolation for full network yet, |
Brave browser ships a feature which downloads and runs Kubo.
We want to measure the number of Brave IPFS nodes on the public network.
@lidel said they announce themselves as
kubo/0.16.0/braveand that we could find them by:ipfs id QmNnooDu7bfjPFoTZYxMNLWUQJyrVwtbZg5gBMjTezGAJN | jq .AgentVersionThe text was updated successfully, but these errors were encountered: