Implement statistics to support Connection Management

[hacdias]

In order to implement stats/bw on js-ipfs, we need to have some way to measure the bandwidth that comes in and out of js-ipfs. It should also support peer and protocol basis metrics.

Some refs:

• per-peer stats ipfs/js-ipfs-bitswap#165
• feat: stats API (stats.bitswap and stats.repo) ipfs/js-ipfs#1198

/cc @pgte @diasdavid

[pgte]

It turns out I also need global stats to implement this connection manager API: https://github.com/libp2p/js-libp2p-connection-manager/tree/master#api

[pgte]

@diasdavid what would be, in your opinion, the best place in the libp2p stack to monitor total traffic (if that's even possible)?

[daviddias]

It can be a module that implements that functionality and gets plugged into js-libp2p.

[pgte]

@diasdavid and how would it hook itself into watching the traffic?

[pgte]

@diasdavid ping :)

[daviddias]

@pgte wanna make a suggestion for that one?

[pgte]

OK, took me a bit to figure out how switch is handling all the outgoing and incoming connections, but here goes:

Some requirements: we need to observe unencrypted traffic in protocol-specific streams so we can compute total (unencrypted) traffic by peer, protocol, direction (upload / download) and the combinations of these ({ peer, protocol }, {peer, direction}, {peer, protocol, direction}).

In my guess, we need the following hooks on js-libp2p-switch:

On self-initiated connections:

• We have the multiplexed protocol-specific connection in this line. We must attach an observer to this stream.

On incoming connections:

• I guess this is a bit more tricky. I think we must wrap each protocol handler function, and one place to do this is before this line: https://github.com/libp2p/js-libp2p-switch/blob/75ba1c156d24c2a4814ef48b1ebdab71f6eaf3d9/src/protocol-muxer.js#L13

@diasdavid thoughts?

[daviddias]

we need to observe unencrypted traffic in protocol-specific streams so we can compute total (unencrypted)

Unencrypted? I guess you mean that you need to observe traffic as it reaches the socket before going to SECIO (which adds a lot of overhead). The traffic won't be in plaintext once the SECIO handshake happens though.

One other option is piggybacking on the fact that we already do Connection wraps to propagate things like Observed Addrs, we can also keep a reference to the socket so that we can measure that directly.

[pgte]

@diasdavid what I mean is the exact opposite: inside the encrypted stream is an unencrypted stream. I think we should not measure the encryption overhead. Also, not sure what go-ipfs does, we should miimc that..

@diasdavid about pigging back on Connection, what would be the insertion points?

[daviddias]

what I mean is the exact opposite: inside the encrypted stream is an unencrypted stream.

I see, you meant to measure the outbound throughput, before encryption and after encryption.

pigging back on Connection, what would be the insertion points?

A Connection can be both the wrap around the socket or a wrap around a muxed stream. We should have a way to visualize how much traffic the stream is contribution to the overall traffic over the socket. libp2p-switch has access to all of these, it can do that work of attaching this readers to the Connectiob object.

[pgte]

@diasdavid yup, and Ithink I want inbound traffic too, also discounting the encryption overhead.

I see the insertion points on dialled connections here, but I don't see that when accepting a new connection. @diasdavid Am I missing something?

[daviddias]

Hope this answers your question:

Incoming Connection via socket (direct from transport):

• https://github.com/libp2p/js-libp2p-switch/blob/master/src/transport.js#L60-L62
• https://github.com/libp2p/js-libp2p-switch/blob/master/src/transport.js#L77

Incoming Connection via stream (from the muxer):

• https://github.com/libp2p/js-libp2p-switch/blob/75ba1c156d24c2a4814ef48b1ebdab71f6eaf3d9/src/dial.js#L191-L193
• https://github.com/libp2p/js-libp2p-switch/blob/75ba1c156d24c2a4814ef48b1ebdab71f6eaf3d9/src/protocol-muxer.js#L13

[pgte]

@diasdavid Ha, multistream-select already returns a Connection. That's what I was missing. Thanks!

[daviddias]

I see this is all done now -- https://github.com/libp2p/js-libp2p#switch-stats-api --, great work @pgte ❤️

[daviddias]

Lot's of relevant Metrics conversations from go-libp2p land

• Metrics instrumentation go-libp2p#522
• What metrics to export? go-libp2p-kad-dht#304
• Add basic opencensus metrics to message handlers go-libp2p-kad-dht#297
• Towards better visibility, debuggability and diagnostics go-libp2p-kad-dht#256
• Port prometheus to opencensus wrapper go-libp2p-daemon#92