Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor ZIP importing #4153

Merged
merged 1 commit into from
Aug 9, 2024
Merged

Conversation

HebaruSan
Copy link
Member

@HebaruSan HebaruSan commented Aug 9, 2024

Problems

During and after #4152, I noticed some issues with ZIP importing in GUI:

  • It appears to freeze for quite a while before it shows a popup
  • It's slow
  • The progress bar for importing treats each file as the same fraction of total completion, even if one is 1 GiB and another is 1 KiB, and the updates can be infrequent

Since importing ZIPs is our go-to workaround when users report slow mod download speeds, it should be usable and performant.

Causes

  • GUI.Main.ImportModules calls ModuleInstaller.ImportFiles on a foreground thread, so all UI updates pause until it finishes
  • For mods that share a ZIP, the import stores the ZIP once for every such mod, which can involve deleting and re-writing the same file in the cache multiple times
  • The import will copy each file into the cache and then ask if it can delete them, whereas if you're OK with losing the original, moving would be faster
  • The files' hashes are checked twice, once to find the module associated with each file, and then again to validate them before storing in the cache, which is redundant

Changes

  • Now GUI.Main.ImportModules uses Wait.StartWaiting to do the imports in a background thread, so the UI will be responsive
  • Now the ZIPs are stored to cache once per URL, so mods that share URLs will no longer clobber one another
  • Now the not-indexed files are reported together in one message
  • Now the already-cached mods are reported together in one message
  • Now if the user says it's OK to delete, the files are moved instead of copied into the cache
  • Now the hashes are not re-validated, so storing is faster
  • Now import progress bar updates are proportional to file size, so they feel nice and smooth and linear

This should make importing ZIPs in GUI more pleasant overall.

@HebaruSan HebaruSan added Bug Something is not working as intended Enhancement New features or functionality GUI Issues affecting the interactive GUI Core (ckan.dll) Issues affecting the core part of CKAN Performance Something's slower than it should be labels Aug 9, 2024
@HebaruSan HebaruSan merged commit 9bbff6b into KSP-CKAN:master Aug 9, 2024
3 checks passed
@HebaruSan HebaruSan deleted the fix/importing branch August 9, 2024 21:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Something is not working as intended Core (ckan.dll) Issues affecting the core part of CKAN Enhancement New features or functionality GUI Issues affecting the interactive GUI Performance Something's slower than it should be
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant