Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: TrustlessGatewayBlockBroker prioritizes & tries all gateways #281

Merged
merged 8 commits into from
Oct 13, 2023

Conversation

SgtPooki
Copy link
Member

@SgtPooki SgtPooki commented Oct 9, 2023

Slight changes on top of #280

  • keep track of individual gateway status and expose reliability rating we can use to sort/prioritize
  • Try all gateways in TrustlessGatewayBlockBroker, sequentially, sorted by reliability

todo

  • fix tests (hasher)
  • Add tests for racing block providers
  • Add tests for prioritizion of trustlessGateways that error/succeed or are untried

achingbrain and others added 2 commits October 9, 2023 18:27
Adds configurable block providers to allow using bitswap but also other
methods such as trustless gateways and any yet-to-be-invented way of
resolving a CID to a block..
@SgtPooki SgtPooki changed the base branch from main to feat/block-providers October 9, 2023 23:22
Comment on lines 99 to 104
// The gateway has > 0 attempts; If we have never encountered an error, consider it 100% reliable.
// Even if a gateway has no successes, it is still considered more reliable than a gateway with errors,
// because it may have been used and aborted, or beaten by another BlockProvider.
if (this.#errors === 0) {
return 1
}
Copy link
Member Author

@SgtPooki SgtPooki Oct 9, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should remove this to ensure slow gateways that never error are deprioritized properly.

@achingbrain
Copy link
Member

Can we scope this PR to improving the TrustlessGatewayProvider? I think all the other points can be taken care of in #280

Base automatically changed from feat/block-providers to main October 10, 2023 16:59
@SgtPooki SgtPooki marked this pull request as ready for review October 10, 2023 17:45
@SgtPooki SgtPooki requested a review from a team as a code owner October 10, 2023 17:45
@SgtPooki SgtPooki self-assigned this Oct 10, 2023
@SgtPooki SgtPooki changed the title feat: improve #280 feat: TrustlessGatewayBlockBroker prioritizes & tries all gateways Oct 10, 2023
@SgtPooki SgtPooki merged commit 9bad21b into main Oct 13, 2023
15 checks passed
@SgtPooki SgtPooki deleted the feat/block-providers-sgtpooki branch October 13, 2023 06:48
achingbrain added a commit that referenced this pull request Oct 13, 2023
This approach is consistent with libp2p components, unixfs, ipns, etc.

Since #281 and #284 were merged without review, this PR implements
suggsestions that would have been in the review of those PRs.

1. Creation of block brokers is done by exported function. If your broker
    takes arguments, pass them to the factory function.  The factory then
    returns a function that accepts helia components and returns the
    broker.
2. Removes BitswapBrokerFactory as it is redunant

The internal API may need some more work but the external API should be
relatively stable.
This was referenced Jan 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
No open projects
Development

Successfully merging this pull request may close these issues.

2 participants