-
Notifications
You must be signed in to change notification settings - Fork 124
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
valkyrie: use storage adapters for derivatives #5626
Conversation
c1475c9
to
22bce4e
Compare
not sure why we're seeing these errors again; the |
it might be that
|
back to this error:
|
this looks to be where the File ID gets mangled by AF:
Or more specifically by |
e151a22
to
4a02cd6
Compare
one error left
|
now hang on a second
|
909e8e3
to
2370e14
Compare
Rebased on main and got specs passing and files ingesting. |
it's not handled correctly, and sometimes passed directly to the underlying process (e.g., shrine)
- restore file_identifier set
don't try to infer the id of the file metadata node. these resources are handled differently by different adapters, just making one with a file_identifer set doesn't do the trick. see, e.g., the usage here: https://github.com/samvera/hyrax/blob/main/spec/models/hyrax/file_metadata_spec.rb#L27-L43
all but one of the arguments to `ValkyrieUpload` are arguments to a specific action (an upload), but the `storage_adapter` parameter is more stable and cumbersome to pass per upload. in the default case, this doesn't much matter, but when using one of the alternative adapters (e.g. for derivatives), we can reduce the dependencies for collaborator objects by allowing them to use a preconfigured instance of the upload service. e.g: ```ruby derivatives_uploader.upload(filename: 'test.txt', file_set: my_fs, io: StringIO.new('test')) ``` instead of: ```ruby derivatives_adapter = Hyrax.config.derivatives_storage_adapter Hyrax::ValkyrieUpload.file(filename: 'test.txt', file_set: my_fs, io: StringIO.new('test')) ```
Previously the final group of characters was not included in the extracted id.
Attempting to resave a converted resource resulted in an error due to the attribute setter now being defined by ActiveFedora instead of ActiveTriples.
Add mock ids to satisfy services that need ids. Include mock_thumbnail in file_ids thumbnail_path should include the fileset id
784077f
to
8e6539e
Compare
@dlpierce your additions all look good! |
Currently when derivatives are generated for uploaded files, Hyrax tries to save them directly to the filesystem, and subsequent access also assumes they will exist in a conventional local location:
hyrax/app/services/hyrax/derivative_path.rb
Line 7 in 9ebda1a
When an application is using a different Valkyrie storage adapter, such as valkyrie-shrine for S3, these assumptions fail, and derivative storage doesn't work. This PR teaches Hyrax to use the defined
derivatives_storage_adapter
instead of directly accessing the filesystem.ValkyrieUpload
in order to be used for original and derivative file uploadThumbnailIndexer
forFileSets
(future work should replace all uses ofThumbnailIndexer
in Valkyrie contexts)@samvera/hyrax-code-reviewers