Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Consider transparently downloading the DB dump instead of fetching live results by default #78

Open
Shnatsel opened this issue Mar 18, 2022 · 3 comments
Labels
enhancement New feature or request

Comments

@Shnatsel
Copy link
Member

Shnatsel commented Mar 18, 2022

TL;DR: run cargo supply-chain update implicitly from other commands, instead of defaulting to querying the API.

If the cache is expired or nonexistent, and --cache-max-age allows it, we could download the latest DB dump by default instead of fetching live results. This would be a lot faster in the typical case.

We would still need to fall back to querying live data from the API if the latest DB dump published by crates.io is older than --cache-max-age.

@Shnatsel Shnatsel added the enhancement New feature or request label Mar 18, 2022
@Shnatsel
Copy link
Member Author

The part of the dump we need seems to only require a 50Mb download, which is not too bad.

@smoelius
Copy link
Contributor

smoelius commented Apr 2, 2024

@Shnatsel Are there any "gotchas" you would anticipate, were someone to try to implement this?

@Shnatsel
Copy link
Member Author

Shnatsel commented Apr 2, 2024

The database dumps are not officially in a stable format, so I could see the format changing in the future and the tool breaking. However, in practice the parts we care about have not changed in years.

The database download also relies on a somewhat fragile order of the files in the archive to reduce the download size. This could easily change without warning and increase the download size considerably.

Finally, these aren't really live results (up to 48 hours out of date by default), but that is probably fine as long as we display a warning about it.

I don't foresee any issues within the code of cargo supply-chain itself.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants