-
Notifications
You must be signed in to change notification settings - Fork 5
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
"blob" materializer task #493
Conversation
Codecov ReportPatch coverage:
Additional details and impacted files@@ Coverage Diff @@
## development #493 +/- ##
===============================================
+ Coverage 91.20% 91.47% +0.26%
===============================================
Files 101 102 +1
Lines 16393 16353 -40
===============================================
+ Hits 14952 14959 +7
+ Misses 1441 1394 -47
☔ View full report in Codecov by Sentry. |
dfdd54b
to
196c1fc
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Great! Looking all good.
I would consider adding our own routes and handlers for the /blobs
paths. Both for MIME types and helping with the symlinking workaround. I'm fine to handle this in a separate PR though.
* Introduce blob materializer task * fmt * Materialize blobs to filesystem in "blob" task * Issue a "blob" task when dependencies met. * Some errors are not Critical errors * fmt * Use tempfile for data dir during tests * Test for blob filesystem materialization * fmt
* development: Make sure `/tmp` directory does not run out of scope before application ends (#557) Integrate `Bytes` value (#554) Stream blob data in chunks to files to not occupy too much memory (#551) Blobs directory configuration (#549) Use correct MAX_BLOB_PIECE_LENGTH from p2panda_rs Build a byte buffer over paginated pieces when assembling blobs (#547) HTTP routes to serve files with correct content type and etag headers (#544) Task for automatic garbage collection of unused documents and views (#500) Refactor tmp blob dir creation after rebase Fix after rebase "blob" materializer task (#493) Add static file server to `http` service (#483) Enable deletion of dangling `document_views` and related `document_view_fields` from db (#491) BlobStore for retrieving raw blob data from the db (#484)
Introduces a new "blob" task to the materializer service which:
blob_v1
orblob_piece_v1
documentblob_piece_v1
document triggered the "blob" task) then we persist the blob view to the filesystem at the expected path../<blob_document_id>/<blob_view_id>
Next steps
blob
module which contains allio
stuff📋 Checklist
CHANGELOG.md