Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

EIP-7594: Passive sampling #3717

Closed
wants to merge 1 commit into from
Closed

EIP-7594: Passive sampling #3717

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 15 additions & 1 deletion specs/_features/eip7594/das-core.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,8 @@
- [Column gossip](#column-gossip)
- [Parameters](#parameters)
- [Peer sampling](#peer-sampling)
- [Passive sampling](#passive-sampling)
- [Active sampling](#active-sampling)
- [Peer scoring](#peer-scoring)
- [Reconstruction and cross-seeding](#reconstruction-and-cross-seeding)
- [DAS providers](#das-providers)
Expand Down Expand Up @@ -223,7 +225,19 @@ To custody a particular column, a node joins the respective gossip subnet. Verif

## Peer sampling

A node SHOULD maintain a diverse set of peers for each column and each slot by verifying responsiveness to sample queries. At each slot, a node makes `SAMPLES_PER_SLOT` queries for samples from their peers via `DataColumnSidecarsByRoot` request. A node utilizes `get_custody_columns` helper to determine which peer(s) to request from. If a node has enough good/honest peers across all rows and columns, this has a high chance of success.
A node SHOULD maintain a diverse set of peers for each column and each slot by verifying responsiveness to sample queries. After that, there are two kinds of peer sampling a node can do: passive sampling and active sampling. If the node has enough good/honest peers across all columns, this has a high chance of success for both kinds.

### Passive sampling

A few moments before each slot, the node SHOULD be subscribed to `SAMPLES_PER_SLOT` column subnets to receive the samples from their peers. A node utilizes `get_custody_columns` helper to determine which column subnets to be subscribed to. This should be easy to do because the node already has a diverse set of peers.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems to defeat the purpose of only subscribing to gossip from custodied subnets. If you subscribe to an extra SAMPLES_PER_SLOT column subnets before each slot, it is equivalent to increasing the size of the total subnets custodied. I don't think subscribing/unsubscribing quickly makes much of a difference here

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, how about defining "passive sampling" as receiving samples from the custodied subnets (instead of extra subnets) and decide that the data is available if it receives such samples. This hasn't been specified anywhere in the spec yet.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This sounds like the wrong term, I don't think being part of a gossip subnet can be thought as sampling . The level of amplification is 8x on a subnet vs a simple req/resp.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The level of amplification is 8x on a subnet vs a simple req/resp.

It doesn't really have to be 8x. You can just connects to a single node as a mesh peer, so the bandwidth used will be just the same as req/resp.

This sounds like the wrong term, I don't think being part of a gossip subnet can be thought as sampling

I disagree on this. Sampling means take some portion of something. Subscribing to some columns/subnets means taking some columns of all the columns, so I think the term still makes sense.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, how about defining "passive sampling" as receiving samples from the custodied subnets (instead of extra subnets) and decide that the data is available if it receives such samples. This hasn't been specified anywhere in the spec yet.

I would like to change my mind on this. I think it's okay to be subscribed to extra subnets.

it is equivalent to increasing the size of the total subnets custodied

It's not really equivalent because you don't keep the past samples for the extra subnets. You just get them and throw them away.

In terms of the bandwidth usage, as I mentioned in the previous comment, you can reduce the mesh degree to 1 so that you use as much bandwidth as req/resp.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This requires us to be able to have dynamic mesh sizes for separate topics, currently in go-libp2p-pubsub and possibly in other language implementations all topics have the same mesh size. Also having a mesh size of 1 can be problematic for the general network as you would have increased latency on the propagation of a message. A remote peer will not know if a connected peer has a mesh size of 8 or 1. On some paths, data columns might take a lot longer to be propagated

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe for this usecase you would want a new protocol message ? Where instead of random gossip, a peer simply returns the most recent seen message ids for a particular topic.

Copy link
Member Author

@ppopth ppopth May 27, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also having a mesh size of 1 can be problematic for the general network as you would have increased latency on the propagation of a message

I think that's okay because, even though the message is delayed or not received at all, the passive sampling acts as only a complement to active sampling.

Notice that, without passive sampling, the sampling node has to wait until the sampling time to request the samples. With passive sampling (even with a mesh size of 1), there will be a very likely chance that it will get the samples before the sampling time.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe for this usecase you would want a new protocol message ? Where instead of random gossip, a peer simply returns the most recent seen message ids for a particular topic.

In fact, I have an upgrade to GossipSub in mind which will probably help on this issue. I will create a PR on that in a few days.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe for this usecase you would want a new protocol message ? Where instead of random gossip, a peer simply returns the most recent seen message ids for a particular topic.

In fact, I have an upgrade to GossipSub in mind which will probably help on this issue. I will create a PR on that in a few days.

Here it is libp2p/specs#617


After the node finishes sampling, it SHOULD unsubscribe from the subnets.

### Active sampling

If a node fails to do passive sampling, it MAY choose to do active sampling. In addition, if a node wants to do sampling in the past slots, it MUST do active sampling.

The node can do it by making `SAMPLES_PER_SLOT` queries for samples from their peers via `DataColumnSidecarsByRoot` request. A node utilizes `get_custody_columns` helper to determine which peer(s) to request from.

## Peer scoring

Expand Down
Loading