Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[GossipSub 1.3] Topic observation #617

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

[GossipSub 1.3] Topic observation #617

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
166 changes: 166 additions & 0 deletions pubsub/gossipsub/gossipsub-v1.3.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,166 @@
# gossipsub v1.3: Topic observation

| Lifecycle Stage | Maturity | Status | Latest Revision |
|-----------------|---------------|--------|-----------------|
| 1A | Working Draft | Active | r1, 2024-05-28 |

Authors: [@pop]

Interest Group: [@pop]

[@pop]: https://github.com/ppopth

See the [lifecycle document][lifecycle-spec] for context about maturity level and spec status.

[lifecycle-spec]: https://github.com/libp2p/specs/blob/master/00-framework-01-spec-lifecycle.md

<!-- START doctoc generated TOC please keep comment here to allow auto update -->
<!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE -->

- [Overview](#overview)
- [Motivation](#motivation)
- [Parameters](#parameters)
- [Observing a topic](#observing-a-topic)
- [Notifying observing peers](#notifying-observing-peers)
- [Unobserving a topic](#unobserving-a-topic)
- [Limits on observing peers](#limits-on-observing-peers)
- [Control Messages](#control-messages)
- [OBSERVE](#observe)
- [UNOBSERVE](#unobserve)
- [Protobuf](#protobuf)

<!-- END doctoc generated TOC please keep comment here to allow auto update -->

## Overview

This document specifies a minimal extension to the [gossipsub v1.2](gossipsub-v1.2.md) protocol.

The proposed extension is intended to enable the nodes to get notified when there is a new message in
a topic without actually receiving the actual message.

Four new control messages are introduced: `OBSERVE`, `OBSERVED`, `UNOBSERVE`, and `UNOBSERVED`. They
are primarily used to notify the peers that the node wants to observe/unobserve if there is a new
message in the topic.

## Motivation

There are many use cases of topic observation:

- For a node that just wants to get notified when there is a new message in the topic and doesn't
want to consume too much bandwidth by directly subscribing to the topic.

- For a node that wants to subscribe to the topic, but has some bandwidth constraint. The node can
observe the topic and see which peer notifies it first and later send the `IWANT` to get the
actual message. This ensures that the bandwidth used is approximately the size of the message.

The downside of topic observation is that the observing node is not part of the mesh network, so it
only consumes the messages, but doesn't contribute to other peers by forwarding the messages.

## Parameters

This section lists new configurable parameters.

| Parameter | Description | Reasonable Default |
|-------------|------------------------------------------------------------|--------------------|
| `D_observe` | (Optional) the maximum number of observing peers per topic | `D` |

## Observing a topic

A node can observe a topic by sending an `OBSERVE` message to a peer that already subscribes to the topic.
If the observation is successful, the peer will send back an `OBSERVED` message.

If the node already subscribes to the topic, it doesn't make sense for that node to observe the topic so
the node SHOULD NOT send an `OBSERVE` message for that topic.

## Notifying observing peers

When a node receives a message in the topic, it will fordward the message to its mesh peers as usual and
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the context of sampling for peerDAS, the flow of data becomes trickier.
-> Node receives its columns via custody subnets.
-> In the following slot(or any N slots into the future) it then decides to sample random peers for any number of columns.
-> It sends our the OBSERVE control message to peers subscribed to the desired column gossip topics. And the peer then sends back OBSERVED

However here is the problem, the peer can only send the control message after the slot has passed or at least after it has received its custody columns. It means that the subcribed peer will not be tracking these particular messages when we actually need them to. So it wont be able to send the IHAVE messages that we want.

The way to get around this would be to have us sending OBSERVE messages for all column subnets all the time. It would mean that every node would constantly have to deal with data that it technically doesn't care about at all until it might need it in the future.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

However here is the problem, the peer can only send the control message after the slot has passed or at least after it has received its custody columns

Why? what I would like to do is to observe the subnets a moment before the slot and you will be notified during the slot. So the flow should be.

-> A moment before the slot, nodes decide the columns they want to sample.
-> Nodes send OBSERVE and receive OBSERVED to/from their peers in the corresponding subnets.
-> In the slot, nodes receives their custody columns
-> Observing nodes get notified

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think joining a mesh for such a short period of time can be pretty unstable. One issue you will run into is that the peers that you want to Observe might not have the slots that you want. So you would need to constantly send out to all the available peers that you want to 'observe' and by the time you find a peer with a free slot it might be too late . To look at analogue for this, look at what happens when a node first subscribes to any topic. There will be a lot of GRAFT messages and PRUNE messages sent back, before they settle on a peer set that has the appropriate space in their mesh.

Copy link
Contributor Author

@ppopth ppopth Jun 4, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think joining a mesh for such a short period of time can be pretty unstable

It's very stable because observation is not subscription. You don't actually join the mesh. You are not actually in the middle of the network, but kind of edge of the network (because you don't forward messages to other peers), so how often you observe and unobserve doesn't affect the health of the network.

One issue you will run into is that the peers that you want to Observe might not have the slots that you want. So you would need to constantly send out to all the available peers that you want to 'observe' and by the time you find a peer with a free slot it might be too late . To look at analogue for this, look at what happens when a node first subscribes to any topic. There will be a lot of GRAFT messages and PRUNE messages sent back, before they settle on a peer set that has the appropriate space in their mesh.

Unlike subscription, observation is cheap (because it's only about sending IHAVEs). Together with #617 (comment), I think we can set D_observe to a very high value, or remove it at all. Then, you will very likely have a free slot very fast or do not worry about it at all, if we remove it.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah but each peer has a limit for how many 'observers' they would allow for each topic. You run into the possibility, the peer has used up all of its slots for 'observation' and there is no more space for your node to observe message ids coming into the remote peer's mesh. Unless the plan is to allow many observers per topic where this stops being an issue.

, in addition, it will also send the `IHAVE` message to the peers observing that topic.

Unlike normal `IHAVE` messages which are sent at the heartbeats, these `IHAVE`s are sent immediately when
the message arrives.

After the observing peers receive the `IHAVE`, they MAY choose to request the message by sending an `IWANT`.

After the node receives an `IWANT` from an observing (not subscribing) peer, it SHOULD send back the message.

## Unobserving a topic

A node can unobserve a topic by sending an `UNOBSERVE` message to a peer that it previously sent an `OBSERVE`.
If the unobservation is successful, the peer will send back an `UNOBSERVED` message.

## Limits on observing peers

If there are too many observing peers, those peers can send the `IWANT` messages and the node will have to
send too many messages out to those peers which consumes a lot of bandwidth of the node.

The node MUST limit the number of observing peers per topic below or equal to `D_observe`.

If a node receives an `OBSERVE`, but the limit is already reached, it MAY send an `UNOBSERVED` to the peer
with lowest score. Note that an `UNOBSERVED` can be sent, even if there is no `UNOBSERVE`.

## Control Messages

There are four new control messages introduced in this extension: `OBSERVE`, `OBSERVED`,
`UNOBSERVE`, and `UNOBSERVED`.

### OBSERVE

The `OBSERVE` message informs a peer that the node wants to receive an IHAVE message immediately when
there is a new message in the specified topic arriving at the peer.

If the node already subscribes to the topic, the peer MUST ignore the `OBSERVE` message.

### OBSERVED

The `OBSERVED` message informs a peer that the observation is successful.

### UNOBSERVE

The `UNOBSERVE` message informs a peer that the node doesn't want to receive an IHAVE message
immediately for the specified topic anymore. This is like an undo message for the `OBSERVE`.

If there is no previous `OBSERVE` message for the specified topic from the node, the peer MUST ignore
the `UNOBSERVE` message.

### UNOBSERVED

There are two use cases for `UNOBSERVED`:

- To inform a peer that the unobservation is successful.

- To inform a peer that it has been forced to unobserve.

## Protobuf

This extension extends the existing `ControlMessage` structure as follows.

```protobuf
message RPC {
// ... see definition in the gossipsub specification
}

message ControlMessage {
// ... see definition in the gossipsub specification
repeated ControlObserve observe = 6;
repeated ControlObserved observed = 7;
repeated ControlUnobserve unobserve = 8;
repeated ControlUnobserved unobserved = 9;
}

message ControlObserve {
optional string topicID = 1;
}

message ControlObserved {
optional string topicID = 1;
}

message ControlUnobserve {
optional string topicID = 1;
}

message ControlUnobserved {
optional string topicID = 1;
}
```