Obsoleted resource maps are shown in the download dropdown menu

[vchendrix]

If a user creates data derivations, thus resulting in multple resource maps for a data package why would the user be allowed to download all the different resulting data packages and not the most current one?

Multiple data package downloads

This is confusing. It seems to me that the user should only be able to download the current data package.

[laurenwalker]

If a metadata document is in multiple resource maps that are not in the same obsolescence chain, that dropdown menu will appear so they can choose which package to download.

[vchendrix]

@laurenwalker What I am confused by is how does the user know which package they want? It just numbers them 1-3.

[laurenwalker]

Yeah, it's not a very informative interface. It has been a rare case in our repositories so we haven't dedicated the time to improving it.

[vchendrix]

The problem with this is that it is not clear which dataset is the current one. Is the last dataset the current one?

…

On Wed, Mar 28, 2018 at 12:18 PM, Lauren Walker ***@***.***> wrote: Yeah, it's not a very informative interface. It has been a rare case in our repositories so we haven't dedicated the time to improving it. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#537 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABfGcVxCRXFrJiQPyZ2YDphlWPO8KUs_ks5ti-IagaJpZM4S-9T6> .

[vchendrix]

Based on my observations it looks like the resource maps are ordered in upload date in ascending order. Is that correct?

…

On Fri, Mar 30, 2018 at 7:48 AM, Valerie Hendrix ***@***.***> wrote: The problem with this is that it is not clear which dataset is the current one. Is the last dataset the current one? On Wed, Mar 28, 2018 at 12:18 PM, Lauren Walker ***@***.***> wrote: > Yeah, it's not a very informative interface. It has been a rare case in > our repositories so we haven't dedicated the time to improving it. > > — > You are receiving this because you authored the thread. > Reply to this email directly, view it on GitHub > <#537 (comment)>, > or mute the thread > <https://github.com/notifications/unsubscribe-auth/ABfGcVxCRXFrJiQPyZ2YDphlWPO8KUs_ks5ti-IagaJpZM4S-9T6> > . >

[laurenwalker]

I believe they are ordered semi-arbitrarily based on the ordering received from the Solr index resourceMap field. They are all "current versions" in that none of the resource maps listed in that dropdown are obsoleted. It would make sense to order them by uploadDate and give them a better label than "Dataset 1", "Dataset 2", etc. Maybe we could show the citation for that package with a link to that package's MetadataView (#view).

[laurenwalker]

This is a symptom of having a metadata-centric system - the search on the DataCatalogView (#data) returns science metadata objects, so when a science metadata object is in more than one package/resource map, we don't really have a way of determining which package is the "current one" that the user is interested in downloading. So ultimately they need to choose which package they want to download.

This has been a rare case in the DataONE world since science metadata documents are usually only in one dataset, so we haven't flushed out that dataset selection UI. Is this is a common occurrence in your repository?

[vchendrix]

From my tests using the Metacat UI derivations interface, I have confirmed that if I make an number of new derivations that the obsoleted resource maps show up in the index search results.

Example of multiple derivations

This is an image taken from my localhost

Three obsoleted resourceMaps

This document returns four resource maps. The first three in the list are obsoleted. The last one is the current one. Is this an issue with metacat or metacat UI. It seems to me that either Metacat should not return the obsoleted resource maps OR the query in Metacat UI needs to filter out the obsoleted resource maps.

Search by ID: ess-dive-7d51d34fce3ce31-20180330T175118791
id=ess-dive-7d51d34fce3ce31-20180330T175118791 seriesId=
	formatType=METADATA
	title=FOOOOOOO
	obsoletes=
	obsoletedBy=
	resourceMap=['urn:uuid:6d8641e5-1882-4415-b4ee-b3052b8104bb', 
               'ess-dive-6a6fdd22f6b14d6-20180409T163002986', 
               'ess-dive-12f541e7f9f6116-20180409T163011714', 
               'ess-dive-edd489d38bd083b-20180409T163022313']

[laurenwalker]

@vchendrix - Ok, I see the issue you are talking about now. I'll create a separate ticket for the download dropdown menu UI improvements, since that is a different issue than this bug.

@gothub - It looks like Val found a bug in the prov editor where a new resource map is created and the old resource maps are not obsoleted in the system metadata. Let me know if you have the time to look into this and if not, I can.

[laurenwalker]

Ack, sorry it looks like I didn't investigate far enough before I commented on this ticket. The resource maps ARE obsoleted correctly, it's just the dropdown menu that is wrong. I'll slate this into a 2.0.1 release.

[laurenwalker]

I created the above referenced Metacat issue to solve this bug, which is ultimately caused by a Solr index bug, not a MetacatUI bug. When the Solr index is fixed, the messed-up download buttons will resolve themselves. Will keep track of the progress there.

[laurenwalker]

After some discussion with the dev team, this may not be a Metacat issue after all. Reopening.

[laurenwalker]

Download Dropdown Menu Issue

Issue description: When a metadata document is aggregated by more than one resource map in the same obsolence chain, both obsoleted packages and their newer versions are shown in the download dropdown menu.

This issue is primarily caused after a package is updated with the provenance editor.

Original github issue: #537

Expected behavior: The search result row should show a dropdown menu so the user can select which package s/he wants to download. Only the most recent version of the packages that contains this metadata document should be shown in the menu.

Example:

Metadata document M is aggregated by resource map A. The Solr document for metadata M looks like this:

id: M
resourceMap: A

Provenance relationships are added to the resource map and this new version is given id B. M is now also aggregated by resource map B. The Solr document for metadata M looks like this:

id: M
resourceMap: A, B

In this example, the search result row in MetacatUI would have a download dropdown menu with two choices: A or B.

A and B are technically unique packages available for download, but in this scenario, there doesn't seem to be a good reason to ever download package A. It would make most sense to have the download button only download package B.

Now, let's say the data owner creates a new resource map, Z, which aggregates M, but it contains different data files and is a totally separate package. Z does not obsolete B. Now the Solr document for metadata M looks like this:

id: M
resourceMap: A, B, Z

Ideally, the download menu would have the choice to download either B or Z, so those are the most recent resource maps that aggregate M.

Solutions

There are a few different options to solve this issue:

A. MetacatUI sends another (sigh) Solr query to determine if any of the resource maps listed in the Solr resourceMap field are obsoleted by another id in that list. This would be one more query per search results page. It would not hold up the rendering of the search result row in the UI, but the download button will have a slight delay rendering if there are multiple resource maps.

B. We add a new Solr field that contains only the newest resource map identifiers. In the example above, the Solr document for M could look like:

id: M
resourceMap: A, B, Z
newestResourceMap: B, Z

C. When there are multiple packages, we disable the download button, and add a tooltip that instructs users to click to go to the dataset landing page and see the list of packages to choose from. The MetadataView renders a table for each unobsoleted resource map.

---

Anyone feel free to share input if interested. I am leaning towards solution C

[vchendrix]

Option C seems good to me.

…

On Tue, Jul 31, 2018 at 1:31 PM Lauren Walker ***@***.***> wrote: I typed up a description of this bug and some solutions here: https://hackmd.io/kcnViO5ZSEGx1u2L7aonBw?view Anyone feel free to share input if interested. I am leaning towards solution C in the hackpad above. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#537 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABfGcSfnjrlGnbtiL6gLpMvHCRvOGT8Rks5uML6agaJpZM4S-9T6> .

[laurenwalker]

I am going to move forward with Option C.

[laurenwalker]

This has been fixed by changing the resultsItem template. @vchendrix, just make sure that the ESS-DIVE theme resultsItem.html template merges these changes in. If you'd like me to do that, let me know!