Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Speed up rarity check in index #724

Closed
Tracked by #713
casey opened this issue Oct 27, 2022 · 1 comment · Fixed by #723
Closed
Tracked by #713

Speed up rarity check in index #724

casey opened this issue Oct 27, 2022 · 1 comment · Fixed by #723
Assignees

Comments

@casey
Copy link
Collaborator

casey commented Oct 27, 2022

No description provided.

@casey casey changed the title speed up rarity check Speed up rarity check in index Oct 27, 2022
@casey casey linked a pull request Oct 27, 2022 that will close this issue
@veryordinally
Copy link
Collaborator

veryordinally commented Oct 28, 2022

I performed some analysis of the distribution of ordinal epochs that we encounter in the rarity checking during indexing various segments of the mainnet:
Screenshot 2022-10-28 at 17 33 50

I also performed a head-to-head comparison of binary search and linear search (by running both versions of the method and got these results from the profiler, indexing from blocks 700000 to 701000:

Screenshot 2022-10-28 at 17 39 35

So binary search takes a total time of about 59s, and linear search takes about 7s, indicating a speedup of a factor of more than 8 on this method.

I will investigate now to what amount of savings that translates overall in an index run.

@casey casey self-assigned this Oct 28, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants