Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Export] Read and ZIP project files in 16K chunks instead of reading the whole file at once. #54673

Merged
merged 1 commit into from
Nov 6, 2021

Conversation

bruvzg
Copy link
Member

@bruvzg bruvzg commented Nov 6, 2021

Read and ZIP files in chunks instead of reading the whole file into the single Vector to avoid Vector limitations (and excessive memory usage).

Fixes #54670

@bruvzg bruvzg added bug platform:macos topic:editor cherrypick:3.x Considered for cherry-picking into a future 3.x release topic:export labels Nov 6, 2021
@bruvzg bruvzg added this to the 4.0 milestone Nov 6, 2021
@bruvzg bruvzg requested a review from a team as a code owner November 6, 2021 17:02
@Calinou
Copy link
Member

Calinou commented Nov 6, 2021

Does this slow down exporting times noticeably? If so, a larger buffer (like 64 KB or 256 KB) could be used.

@bruvzg
Copy link
Member Author

bruvzg commented Nov 6, 2021

Does this slow down exporting times noticeably? If so, a larger buffer (like 64 KB or 256 KB) could be used.

I have not noticed any substantial difference between original code and this one with 16 KB, 256 KB and 1 MB buffers. But I guess it might depend on the hardware.

Godot already is using the same code with 16 KB buffer to read and write files to the PCK file.

@akien-mga akien-mga merged commit 67b5577 into godotengine:master Nov 6, 2021
@akien-mga
Copy link
Member

Thanks!

@bruvzg bruvzg deleted the macos_zip_export branch November 6, 2021 18:41
@akien-mga
Copy link
Member

Cherry-picked for 3.5.

@akien-mga akien-mga removed the cherrypick:3.x Considered for cherry-picking into a future 3.x release label Nov 15, 2021
@akien-mga
Copy link
Member

Cherry-picked for 3.4.1.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[3.4] macOS export broken for PCK file over 2GB when exporting to a ZIP archive
3 participants