[GossipSub 1.3] Topic observation

[ppopth]

Overview

The proposed extension is intended to enable the nodes to get notified when there is a new message in
a topic without actually receiving the actual message.

Four new control messages are introduced: OBSERVE, OBSERVED, UNOBSERVE, and UNOBSERVED. They
are primarily used to notify the peers that the node wants to observe/unobserve if there is a new
message in the topic.

Motivation

There are many use cases of topic observation:

• For a node that just wants to get notified when there is a new message in the topic and doesn't
  want to consume too much bandwidth by directly subscribing to the topic.

• For a node that wants to subscribe to the topic, but has some bandwidth constraint. The node can
  observe the topic and see which peer notifies it first and later send the IWANT to get the
  actual message. This ensures that the bandwidth used is approximately the size of the message.

The downside of topic observation is that the observing node is not part of the mesh network, so it
only consumes the messages, but doesn't contribute to other peers by forwarding the messages.

Relevance to Ethereum

This is useful for the upcoming PeerDAS upgrade (EIP-7594), since it allows a node to get notified when a message is available without bandwidth overhead.

Without this extension, I previously tried to address this Ethereum-specific problem in ethereum/consensus-specs#3717

[Wondertan]

The downside of topic observation is that the observing node is not part of the mesh network, so it
only consumes the messages, but doesn't contribute to other peers by forwarding the messages.

What if an observing node propagated the IHAVE message further to its "observe mesh" peers under a particular condition(like having data locally)? Then, nodes would still be able to contribute to the topic, but keeping bandwidth usage low traded for higher latency.

[ppopth]

What if an observing node propagated the IHAVE message further to its "observe mesh" peers under a particular condition(like having data locally)? Then, nodes would still be able to contribute to the topic, but keeping bandwidth usage low traded for higher latency.

Sending IHAVEs to other peers means you commit that you will send the message to those peers if they send the IWANT back, which requires too much bandwidth.

[nisdas]

In the context of sampling for peerDAS, the flow of data becomes trickier.
-> Node receives its columns via custody subnets.
-> In the following slot(or any N slots into the future) it then decides to sample random peers for any number of columns.
-> It sends our the OBSERVE control message to peers subscribed to the desired column gossip topics. And the peer then sends back OBSERVED

However here is the problem, the peer can only send the control message after the slot has passed or at least after it has received its custody columns. It means that the subcribed peer will not be tracking these particular messages when we actually need them to. So it wont be able to send the IHAVE messages that we want.

The way to get around this would be to have us sending OBSERVE messages for all column subnets all the time. It would mean that every node would constantly have to deal with data that it technically doesn't care about at all until it might need it in the future.

[ppopth]

However here is the problem, the peer can only send the control message after the slot has passed or at least after it has received its custody columns

Why? what I would like to do is to observe the subnets a moment before the slot and you will be notified during the slot. So the flow should be.

-> A moment before the slot, nodes decide the columns they want to sample.
-> Nodes send OBSERVE and receive OBSERVED to/from their peers in the corresponding subnets.
-> In the slot, nodes receives their custody columns
-> Observing nodes get notified

[nisdas]

I think joining a mesh for such a short period of time can be pretty unstable. One issue you will run into is that the peers that you want to Observe might not have the slots that you want. So you would need to constantly send out to all the available peers that you want to 'observe' and by the time you find a peer with a free slot it might be too late . To look at analogue for this, look at what happens when a node first subscribes to any topic. There will be a lot of GRAFT messages and PRUNE messages sent back, before they settle on a peer set that has the appropriate space in their mesh.

[ppopth]

I think joining a mesh for such a short period of time can be pretty unstable

It's very stable because observation is not subscription. You don't actually join the mesh. You are not actually in the middle of the network, but kind of edge of the network (because you don't forward messages to other peers), so how often you observe and unobserve doesn't affect the health of the network.

One issue you will run into is that the peers that you want to Observe might not have the slots that you want. So you would need to constantly send out to all the available peers that you want to 'observe' and by the time you find a peer with a free slot it might be too late . To look at analogue for this, look at what happens when a node first subscribes to any topic. There will be a lot of GRAFT messages and PRUNE messages sent back, before they settle on a peer set that has the appropriate space in their mesh.

Unlike subscription, observation is cheap (because it's only about sending IHAVEs). Together with #617 (comment), I think we can set D_observe to a very high value, or remove it at all. Then, you will very likely have a free slot very fast or do not worry about it at all, if we remove it.

[nisdas]

Yeah but each peer has a limit for how many 'observers' they would allow for each topic. You run into the possibility, the peer has used up all of its slots for 'observation' and there is no more space for your node to observe message ids coming into the remote peer's mesh. Unless the plan is to allow many observers per topic where this stops being an issue.

[Wondertan]

Sending IHAVEs to other peers means you commit that you will send the message to those peers if they send the IWANT back, which requires too much bandwidth

"Too much" depends on how big your "observe mesh" is which when configurable can lead to acceptable bandwidth usage. There is no way to contribute to the topic without spending some bandwidth.

[ppopth]

"Too much" depends on how big your "observe mesh" is which when configurable can lead to acceptable bandwidth usage. There is no way to contribute to the topic without spending some bandwidth.

That's right. It's configurable. But, it will be configured by who?

If it's a global parameter like the normal GossipSub mesh degree, that contradicts with the purpose of topic observation and you will have some bandwidth requirement beforehand. In fact, how is it different from normal GossipSub with low mesh degree to save the bandwidth?

If you can make it configurable independently by each node, that would be great.

[ppopth]

Now I'm not quite sure if we should allow observing nodes to send IWANT. That seems irrelevant. If you want to get the message, you probably should use an out-of-band channel like Req/Resp that is used by Ethereum.