Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

core/node: prioritize announcing pin roots #10376

Merged
merged 7 commits into from
Apr 9, 2024
Merged

Conversation

hacdias
Copy link
Member

@hacdias hacdias commented Mar 22, 2024

This should close #10365.

@hacdias hacdias force-pushed the reprovider-prioritization branch from 926744f to c4ec92b Compare March 22, 2024 12:14
@aschmahmann
Copy link
Contributor

and figure out how on earth I will test that

A couple options that seem reasonable:

  1. Don't fully test end-to-end and just test the functions that get implemented for announcements
  2. Test end-to-end but with a chunk of code running that processes delegated provides (not sure what the state of these is at the moment)

Both of these would let you capture ordering without being that annoying (e.g. you could also run a custom DHT server node instead of using a delegated router for provides).

@hacdias hacdias force-pushed the reprovider-prioritization branch 2 times, most recently from 6f478c3 to 85f8327 Compare March 26, 2024 13:33
@hacdias hacdias force-pushed the reprovider-prioritization branch from 85f8327 to d1b78da Compare March 26, 2024 13:35
@hacdias hacdias marked this pull request as ready for review March 26, 2024 13:37
@hacdias hacdias requested a review from a team as a code owner March 26, 2024 13:37
@hacdias hacdias mentioned this pull request Mar 26, 2024
11 tasks
Copy link
Member

@lidel lidel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lgtm, but we need updates to internals from ipfs/boxo#595 to avoid memory usage when providing non-roots (the remainder of datatstore).

test/cli/provider_test.go Show resolved Hide resolved
core/node/provider.go Show resolved Hide resolved
@hacdias hacdias force-pushed the reprovider-prioritization branch from 4012ea4 to 88b05f4 Compare April 4, 2024 11:02
@hacdias hacdias requested a review from lidel April 4, 2024 11:06
@hacdias hacdias force-pushed the reprovider-prioritization branch 2 times, most recently from 36445c1 to 44d2702 Compare April 4, 2024 13:27
Copy link
Member

@lidel lidel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested with and without accelerated client, seems to work ok.
AFAIK we don't have any announcement benchmarks we could run, but eyeballing ipfs stats provide seems to be no slower than the old all.
I think doing this by default is net positive, but included flat strategy below as a precaution.

core/node/provider.go Show resolved Hide resolved
core/node/provider.go Outdated Show resolved Hide resolved
@hacdias hacdias force-pushed the reprovider-prioritization branch from 8f654b2 to 444f0e8 Compare April 8, 2024 15:26
@hacdias hacdias requested a review from aschmahmann April 8, 2024 15:27
hacdias and others added 6 commits April 9, 2024 08:18
Adding 'flat' strategy which works the same as the old default 'all'.
This way if anyone has memory-related regressions, they can fix without
waiting for a new release.
@hacdias hacdias force-pushed the reprovider-prioritization branch from 444f0e8 to 9434294 Compare April 9, 2024 06:18
@hacdias hacdias enabled auto-merge (squash) April 9, 2024 06:30
@hacdias hacdias merged commit 6f2a61e into master Apr 9, 2024
14 checks passed
@hacdias hacdias deleted the reprovider-prioritization branch April 9, 2024 06:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
No open projects
Status: 🎉 Done
Development

Successfully merging this pull request may close these issues.

Prioritize announcement of pin roots
3 participants