Node: Observation batching #3977
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
bruce-riley
force-pushed
the
node_observation_batching
branch
3 times, most recently
from
4e6c95b
to
fd94f23
Compare
bruce-riley
force-pushed
the
node_observation_batching
branch
5 times, most recently
from
be3eceb
to
a3a513a
Compare
bruce-riley
force-pushed
the
node_observation_batching
branch
2 times, most recently
from
4ca598b
to
2bf941c
Compare
panoel
reviewed
Jul 3, 2024
panoel
reviewed
Jul 3, 2024
panoel
reviewed
Jul 3, 2024
panoel
previously approved these changes
Jul 3, 2024
evan-gray
previously approved these changes
Jul 4, 2024
|
A concern has been raised regarding possible latency impacts of this change. Therefore I am converting this PR back to draft until those concerns can be addressed. |
|
The availability of Pyth's market data with as little latency as possible is a very strong usecase to warrant not being part of observation batching, and so they should be excluded from it. |
bruce-riley
force-pushed
the
node_observation_batching
branch
from
9e3524a
to
dcd325e
Compare
bruce-riley
force-pushed
the
node_observation_batching
branch
from
81554ba
to
601ffc8
Compare
|
This PR is being superseded by PR #4066, so this one is being closed. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR introduces the batching of observations by adding the
SignedObservationBatchgossip message. It modifies the guardian to wait up to one second in order to batch as many observations as possible (up to 4000) into a single gossip message. This not only reduces gossip traffic but also the load on the guardians and applications that listen to observations.This PR is meant to go hand-in-hand with PR #4000, which further improves gossip traffic by splitting it across different topics, allowing the guardians and gossip listeners (like spies and flies) to only subscribe to the topics they need.
NOTE: Once the gossip network cutover is complete, support for
SignedObservation(individual observation) messages will be dropped, but in the meantime, applications that listen to observations need to support both message streams.