Data seeds improvements #5056

apertome · 2021-07-28T13:35:44Z

Reimplement database seeds using Valkyrie rather than ActiveFedora. Add option for multiple seed files and an environment variable to control whether they are loaded

Changes proposed in this pull request:

Create db seeds using Valkyrie
Split seeds into 2 files, 1 for users, 1 for collections/works
Create collections/works as described in this sheet: https://docs.google.com/spreadsheets/d/1b93me1Gh614eQEWzeqzsSw32PDr_ymIfJ-PvrR7zxDM/edit#gid=291627674

A few caveats:

First attempt at implementing this used Monograph but had some issues in dassie with that. So the work is created as a GenericWork and the valkyrie_resource is retrieved.
Not all the categories from the sheet are implemented yet
File attachment is not implemented yet
The script attempts to set the collections/works as public, but this doesn't seem to actually work as implemented

@samvera/hyrax-code-reviewers

jlhardes · 2021-09-13T17:17:56Z

Work on #4788 Hyrax::Works do not show thumbnail in Dashboard -> Works also connected to this PR work.

Co-authored-by: Chris Colvard <chris.colvard@gmail.com>

add basic required metadata to the pcdm collection form

Add administrative set form with minimal metadata

add config for setting admin set model

existing collections indexed as `'Collection'` and those created with `Wings` should still turn up in queries. probably we should be using the indexed `generic_type` for these filters instead of a list of models, but this keeps compatibility until we can work that out.

updates metadata in the model and indexer

For some reason there were two spec files testing `Hyrax::AdministrativeSetIndexer`. One in `hyrax/administrative_set_indexer_spec.rb`. The other in `hyrax/valkyrie_administrative_set_indexer_spec.rb`. The tests are the same in both spec files. I removed the one starting with `valkyrie` because the other one follows the normal naming conventions.

we want these stubs to apply for whatever collection class is in use in the test suite.

this partial has apparently never resolved properly; it's test was stubbed. this fixes the reference and forces the template to resolve to pass the view spec

this presenter method was previously deprecated in favor of a helper. it looks like this view got missed in that update.

setup these tests with valkyrie.

update these expectations to be sensitive to current model class filter considerations. use the Valkyrie test class `Monograph` as the dummy work class, since `GenericWork` is likely to be removed from the test app at some point.

this is a case where we are doing a whole collection lookup just to then pass the id to the collaborator service. skip this and save the DB round trip.

`Dashboard::CollectionsController` tests still have quite a few hard-coded class expectations. this loosens a few of them up.

we want to be able to take actions (e.g. to log, or send timeseries data), when a collection changes its membership. add a hook for this.

when using a collection class other than `::Collection`, apply Hyrax's curation_concern permissions. (e.g. this allows `#create`).

map `Hyrax::PcdmCollection` to `Hyrax::Forms::PcdmCollectionForm` for the factory method's purposes.

`Dashboard::CollectionsController#create` has some behavior to avoid CanCan's load_resource hook. the reasons for avoiding this are a bit opaque, but we definitely don't want to carry this forward. disabling it immediately does actually fail some tests, so instead limit the skip to cases where an ActiveFedora collection class is in use.

if a ChangeSet has a `#collection_type_gid` value, never override it with the default.

Valkyrie models don't index automatically on save, so setting a "re-index extent" on the model doesn't make much sense. it's honestly not clear to me that this line does anything at all. the collection doesn't appear to be saved (or indexed) as part of this controller's logic. whatever the case, it turns out that just not failing on this assignment makes this controller's unit tests pass. we'll definitely want to look closer at it, since the controller is doing `load_and_authorize_resource` on the whole collection just to then pass through the id to a collaborator---do we need to load the object, or just authorize based on id and move on?

this actor won't be updated for compatibility with Valkyrie-based collection models. skip its tests in the case that Valkyrie is in use for Collection.

these specs stubbed ActiveFedora in unnecessary places, and just generally had convoluted setup and expectations (one test memoized an empty array, used a before block to populate it, and then checked its contents as the only expectation!). don't exercise code under test in `subject` or `before`.

query collection from Valkyrie and use it to handle PermissionsCreateService logic.

a first hack at editing admin sets using Valkyrie

initial take on support create for valkyrie administrative sets

Valkyrized applications can't use the legacy MemberPresenterFactory, which depends on the AF `list_source`

Valkyrized FileSets don't have a `#parent` method, but the FileSetController provides a helper method we can use for compatibility

chart: document chart publishing process

…board->Collections

Collection type gid is required for listing admin sets at Dashboard -> Collections. This is what identifies this as an admin set.

add collection_type_gid attribute to Hyrax::AdministrativeSets

related to #5385.

…default admin set

houndci-bot · 2022-01-31T19:34:05Z

app/assets/stylesheets/_bootstrap-default-overrides.scss

+// Date picker on safari placeholder text was misaligned
+@media screen {
+  @media screen and (-webkit-min-device-pixel-ratio: 0) {
+    input[type="date"].form-control {


Avoid qualifying attribute selectors with an element.
Avoid qualifying class selectors with an element.

apertome · 2022-01-31T21:22:37Z

Pushed some updates to this, but I'm getting some flakiness still.

Current behavior looks like:

Some seed collections and works are created
Currently works are being created as GenericWorks. I think this is mostly working but no image attachments yet -- still need to figure out how to do that.
The seeds script is able to create Works and automatically assicate with Default Admin Set

It can also associate Works with Collections:

The seed script is not creating Monograph works, but when I try to add some manually via the GUI, and upload files, it seems like it recognized that there are supposed to be files but any time you try to access them (either to view, download, or edit) there are errors like this:

dlpierce · 2022-03-02T18:18:31Z

Closing in favor of #5472

dlpierce self-assigned this Aug 11, 2021

elrayle and others added 27 commits October 8, 2021 11:46

Update lib/hyrax/configuration.rb

450b659

Co-authored-by: Chris Colvard <chris.colvard@gmail.com>

Update lib/hyrax/configuration.rb

5cc2e8f

Co-authored-by: Chris Colvard <chris.colvard@gmail.com>

Merge pull request #5176 from samvera/col_form

2326e63

add basic required metadata to the pcdm collection form

Merge pull request #5177 from samvera/af_form

3682bb9

Add administrative set form with minimal metadata

Merge pull request #5175 from samvera/as_model

e3dba86

add config for setting admin set model

include core metadata in administrative sets

f7782bc

updates metadata in the model and indexer

adjust stubs in WorkShowPresenter specs

01fe01c

we want these stubs to apply for whatever collection class is in use in the test suite.

squash

dbad658

squash

6237fd6

fix out-of-path reference to _view_type_group.html.erb partial

2da50a2

this partial has apparently never resolved properly; it's test was stubbed. this fixes the reference and forces the template to resolve to pass the view spec

switch call to deprecated presenter method with a helper call

a5ad618

this presenter method was previously deprecated in favor of a helper. it looks like this view got missed in that update.

change ManagedCollectionsServiceSpec to use valkyrie models in setup

f393838

setup these tests with valkyrie.

update WorkSearchBuilder specs for Valkyrie

2e2729f

update these expectations to be sensitive to current model class filter considerations. use the Valkyrie test class `Monograph` as the dummy work class, since `GenericWork` is likely to be removed from the test app at some point.

load collection from CanCan tooling in CollectionMembersController

8f5d634

don't look up destination collection for 'move' action to access id

662e311

this is a case where we are doing a whole collection lookup just to then pass the id to the collaborator service. skip this and save the DB round trip.

don't hardcode collection class expectations in some specs

f6bb8af

`Dashboard::CollectionsController` tests still have quite a few hard-coded class expectations. this loosens a few of them up.

add a publishable event for collection membership changes

9986429

we want to be able to take actions (e.g. to log, or send timeseries data), when a collection changes its membership. add a hook for this.

apply general curation_concerns permissions to configured collection

57cc27f

when using a collection class other than `::Collection`, apply Hyrax's curation_concern permissions. (e.g. this allows `#create`).

allow ResourceForms #for` factory method to find Collection forms

27e5fa6

map `Hyrax::PcdmCollection` to `Hyrax::Forms::PcdmCollectionForm` for the factory method's purposes.

don't override a ChangeSet's existing data with a generic default

50213ff

if a ChangeSet has a `#collection_type_gid` value, never override it with the default.

skip CollectionsMembershipActor specs if using Valkyrie collections

96f60a2

this actor won't be updated for compatibility with Valkyrie-based collection models. skip its tests in the case that Valkyrie is in use for Collection.

use valkyrie for PermissionsCreateService

fc0d4a0

query collection from Valkyrie and use it to handle PermissionsCreateService logic.

elrayle and others added 24 commits January 27, 2022 14:15

Merge pull request #5392 from samvera/admin-set-edit-valkyrie

984dde5

a first hack at editing admin sets using Valkyrie

Merge pull request #5394 from samvera/admin-set-create-valkyrie

c959fef

initial take on support create for valkyrie administrative sets

use helper to get permission template for collections and admin sets

8e41759

WorkShowPresenter#member_presenter_factory: pick based on solr model

bb9f393

Valkyrized applications can't use the legacy MemberPresenterFactory, which depends on the AF `list_source`

use valkyrie-compatible controller methods in FileSet views

4d0a998

Valkyrized FileSets don't have a `#parent` method, but the FileSetController provides a helper method we can use for compatibility

chart: document chart publishing process

d858666

Merge pull request #5415 from samvera/chart-push

ccbb0e3

chart: document chart publishing process

add Hyrax::AdministrativeSets to the set of classes displayed on Dash…

6792f5f

…board->Collections

inculde permissions and visibility in AdministrativeSet solr doc

0954b08

Fix nokogiri on arm64 alpine

01606e3

Update FITS

54b6f04

add collection_type_gid attribute to Hyrax::AdministrativeSets

838a554

Collection type gid is required for listing admin sets at Dashboard -> Collections. This is what identifies this as an admin set.

Merge pull request #5422 from samvera/as_index_gid

f01616d

add collection_type_gid attribute to Hyrax::AdministrativeSets

AdministrativeSet#description should be single-valued

8d84eaa

related to #5385.

Split off hyrax-work-base image and bundle install from dassie

19bee57

Skip including the dassie Gemfile when building in docker-compose

d81a025

Create new Valkyrie-based seed script

0e38ff0

Implement creating basic works and collections using Valkyrie

db9779b

Handle existing works/collections better; create more works

4aca3ae

Integrate seed dirs; add environment variable

31a510f

Cleanup

f20b254

Use real usernames rather than examples; add a child collection; use …

75c88f0

…default admin set

Set works/collections to public

a47fa40

Merge

d6d255b

houndci-bot reviewed Jan 31, 2022

View reviewed changes

dlpierce closed this Mar 2, 2022

elrayle mentioned this pull request Mar 21, 2022

add infrastructure for clearing repository and running seeds #5472

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Data seeds improvements #5056

Data seeds improvements #5056

apertome commented Jul 28, 2021 •

edited by no-reply

Loading

jlhardes commented Sep 13, 2021

houndci-bot Jan 31, 2022

apertome commented Jan 31, 2022

dlpierce commented Mar 2, 2022

Data seeds improvements #5056

Data seeds improvements #5056

Conversation

apertome commented Jul 28, 2021 • edited by no-reply Loading

jlhardes commented Sep 13, 2021

houndci-bot Jan 31, 2022

Choose a reason for hiding this comment

apertome commented Jan 31, 2022

dlpierce commented Mar 2, 2022

apertome commented Jul 28, 2021 •

edited by no-reply

Loading