Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make download_hash.sha1 optional #344

Merged
merged 1 commit into from
Oct 1, 2024

Conversation

HebaruSan
Copy link
Member

@HebaruSan HebaruSan commented Oct 1, 2024

Motivation

Currently we always calculate both the SHA1 and the SHA256 for all mods. This is redundant and wastes CPU time. KSP-CKAN/CKAN#4135 began the process of making the SHA1 optional; the client now ignores download_hash.sha1 for purposes of validating downloads if download_hash.sha256 is set (which is always).

The infra uses download_hash.sha1 in two places:

  • The 8-character file prefix for archive.org upload ZIPs
    (The client also does this and will need to be updated in a subsequent pull request)
  • Comparing to the sha1 received from archive.org to confirm that a previously uploaded file matches the bot's copy

Changes

  • Now if download_hash.sha1 is absent, we use download_hash.sha256 for the 8-character archive.org file prefix instead. This will not cause collisions because the file prefix comes after the identifier+version bucket string, so the exact same mod version would have to have an SHA1 and SHA256 starting with the same 8 characters. And even if that happened, it's perfectly fine for both to point to the same file!
  • Now if download_hash.sha1 is absent, the Mirrorer calculates the SHA1 from the downloaded/cached file to check whether it matches one already on archive.org

This will move us another step closer to the point of being able to drop SHA1 from the metadata completely.

@HebaruSan HebaruSan added Enhancement New feature or request Mirrorer Uploads mods to archive.org labels Oct 1, 2024
@HebaruSan HebaruSan merged commit 093ce5e into KSP-CKAN:master Oct 1, 2024
3 checks passed
@HebaruSan HebaruSan deleted the feature/optional-sha1 branch October 1, 2024 18:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Enhancement New feature or request Mirrorer Uploads mods to archive.org
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant