Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fileupload handling reworked #80

Merged
merged 1 commit into from
Jul 28, 2023

Conversation

KasKatto
Copy link
Contributor

The current implementation for storing file uploads is sub sub-optimal. If a user uploads a large file (say multiple GBs in size) the memory usage of the application will rise very quickly, which is made worse with larger files and/ or more users active at once. This can lead issues for other apps running on the same physical server or even the app itself.

My proposed change is to use File.tempfile instead of IO::Memory for storing the file. To prevent the server from running out of storage from clutter, the controller deletes the file when it is cleaned up by GC only if the file hasnt been moved. i.e. its safe to move them for long term storage.

Ive also changed the body getter to provide backwards compatibility, but its less than optimal as it creates a new io.

Copy link
Member

@stakach stakach left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@stakach
Copy link
Member

stakach commented Jul 28, 2023

We're also working on a direct to cloud upload solution, but it does rely on a fat JS/TS client. I'll send you a nudge once that's completed

@stakach stakach merged commit f125053 into spider-gazelle:master Jul 28, 2023
@KasKatto KasKatto deleted the file-upload-rework branch July 28, 2023 00:12
@KasKatto KasKatto restored the file-upload-rework branch July 28, 2023 00:19
@stakach
Copy link
Member

stakach commented Jul 28, 2023

https://github.com/spider-gazelle/action-controller/releases/tag/v7.1.0

@KasKatto
Copy link
Contributor Author

Thanks for the quick fix, i was just about to say i mixed up my testing and git and forgot a couple lines.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants