Refactor blockpoller initialization to use first streamable block #67
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
maoueh
reviewed
Nov 12, 2024
Refactor BlockPoller to support generic clients with type parameter 'C'. Updated associated methods, test functions, and interfaces to accommodate this change. This enhancement includes moving the retry mechanism and client handling to the poller, improving flexibility for different client types.
Updated several block fetch functions and utility methods to accept a context parameter for better cancellation and timeout control. Enhanced TestBlockClient to handle block fetching delays, and adjusted tests to accommodate these changes.
Updated the Clients struct to include a maxBlockFetchDuration parameter, enhancing flexibility in specifying timeout durations for block fetching. This change was reflected in the initialization function and associated test cases.
Implemented new rolling strategy interfaces and updated RPC clients to utilize the rolling strategies. Introduced two strategies: RoundRobin and AlwaysUseFirst, modifying existing tests to incorporate these changes.
Introduce a MaxStopBlock variable to be used in the BlockPoller. This change helps manage the upper limit for block polling, allowing more flexibility and control over the polling process.
Introduced a sorting mechanism for clients in the `rpc` package. Added interfaces and implementations for sticky rolling strategy, along with comprehensive test cases. Improved thread safety and replaced the old rolling strategy implementations.
Updated the Sort function to accept a SortValueFetcher interface and changed its implementations accordingly. This change improves flexibility and decouples sort value fetching logic from the client type. Also added zap logger dependency to NewClients initialization in tests.
Add a new MaxStopBlock constant set to math.MaxUint64. This change simplifies the code by using a predefined maximum stop block value for the poller logic.
The MaxStopBlock constant was removed because it was not used anywhere in the code. Cleans up unnecessary variables and maintains code readability.
This change makes the method public by capitalizing its name, allowing it to be accessed outside its package. This update is necessary for external packages that need to use this method.
Ensure the rolling strategies reset internal state variables such as `usedClientCount`, `nextClientIndex`, and others after encountering sorting errors. This is to guarantee proper client selection behavior in subsequent retries.
billettc
force-pushed
the
feature/first_streamable_block_refactor
branch
from
597d11c to
3a7656e
Compare
Upgraded the `github.com/streamingfast/google-cloud-go` module to version `v0.0.0-20241202194114-f77ff78d4f66` in both `go.mod` and `go.sum`. This ensures the project is using the latest compatible changes from the dependency.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Replaced start block with first streamable block in BlockPoller to ensure accurate initiation. Added a check for the state file's existence and integrated state initialization with fetching the first streamable block. Updated tests accordingly.