2.23.0

New features 🎉

MetacatUI has a DOI 📖

MetacatUI's new DOI is doi:10.18739/A25M6282K. The software can be cited as:

Matthew B. Jones, Chris Jones, Lauren Walker, Robyn Thiessen-Bock, Ben Leinfelder, Peter Slaughter, Bryce Mecum, Rushiraj Nenuji, Hesham Elbashandy, Val Hendrix. 2022. MetacatUI: A client-side web interface for DataONE data repositories (version 2.23.0). Arctic Data Center. doi:10.18739/A25M6282K

Improved Citations Display 📃

When displaying citations, MetacatUI now uses a standard format that more closely aligns with APA 7 style formatting. The new citation formatting is used throughout MetacatUI, including on dataset landing pages, search result lists, portals, and other areas. (See #777 and #2106).

On dataset landing pages, the citation in the header has been replaced with a header-style view that is more in line with other online science publications. Previously, only up to five authors were listed in the header. The new view shows at least 20 authors by default, with the option to expand the list to show all authors. The "Copy citation" button has been replaced with a "Cite this dataset" button, which opens a popup with the full citation and includes the same "copy citation" functionality as before. (See #264)

Misc improvements

• Show the mini-legend in the layer details panel in Cesium #2091
• Add project search filter option #2044
• Include a default imagery layer for Cesium #1938
• Bump activesupport from 6.0.3.6 to 6.0.6.1 in /docs #2087
• Bump nokogiri from 1.13.6 to 1.13.9 in /docs #2072
• Bump @actions/core from 1.8.2 to 1.9.1 #2047
• Update JSDocs to 4.0.0 #2089

Bug fixes 🛠️

• Metrics displayed in the modal should be in the same order as the the metrics buttons in the dataset landing page #2100
• Resolves issues with infinite spinners for portal metrics #2088
• Remove references to the deprecated dataCatalogMap app config option #2075
• Update theme routers to conditionally use the Cesium map instead of Google map on the data page #2064
• UserGroupView missing alertTemplate #2062
• Fix issues with the display of the continuous/gradient legend in the Cesium map #2049
• Typo in Ethical Research Practices help text #2048
• portal list fails with 500 service failure #2042
• Multiple portal metric bugs are persisting #2041
• Can't set a data object private by the share button #2009

Developer notes

The CitationView and CitationModel have been changed extensively in order to improve the citations display and to eventually support displaying citations in multiple styles. This may cause conflicts for repositories that have customized or extended this view or model. A guide to customizing the CitationView is included with the docs.

See the complete changelog: 2.22.0...2.23.0

MetacatUI 2.22.0

New features 🎉

Beta: Google Map search now replaced with Cesium 3D maps

The main data search for years has used Google Maps for visualizing and filtering dataset search results. This has now been replaced with Cesium, a more powerful 3D map visualization.

Note: This feature is in Beta, which means it isn't quite ready for production yet. We suggest you keep using Google Maps for now but look forward to supporting Cesium in the coming months.

By default, Cesium in the data search is disabled until this feature is out of Beta. Use the app configuration useDeprecatedDataCatalogView to enable it.

Basic support for your custom projects API

If you have a database of research projects that is accessible via a REST API, you can now use the new Project model and ProjectsView to fetch those projects and display them in the UI. This is a skeleton implementation that is meant for MetacatUI owners to extend. Further support for Projects may be added in future versions.

See #1945, Project, ProjectsView

Bug fixes 🛠️

• Repositories using custom object identifiers were producing invalid EML if there was an annotation in it #2038

Developer notes

New UserGroupView

Managing user groups in the Settings tab of the user profile was previously part of the UserView. This made the view huge and convoluted so it has been refactored out to it's own view, UserGroupView. Group management can now be inserted easily into other views.

Default bagit format is now set to application/bagit-1.0.

If you are using an older version of Metacat you may need to change this to a bagit format that your Metacat version supports.

New configuration options

• useDeprecatedDataCatalogView - As of 2.22.0, the DataCatalogView is being soft-deprecated and replaced with the new CatalogSearchView. To give MetacatUI operators time to transition to the new CatalogSearchView, this configuration option can be enabled (by setting to true) and will tell MetacatUI to use the legacy link DataCatalogView. It is highly suggested that MetacatUI operators switch to supporting the new CatalogSearchView as soon as possible as the legacy link DataCatalogView will be fully deprecated and removed in the future.
• defaultSearchFields - The document fields to return when conducting a search. This is the list of fields returned by the main catalog search view.

See the complete changelog: 2.21.0...2.22.0

2.21.0

New features

Semantic annotation filters in the portal collection builder

Filter datasets in a portal collection by semantic annotation. This new filter type allows researches to use indexed annotations like Measurement Type annotations in their collection query via an easy-to-use ontology tree interface (the same ontology tree interface used in the main search view).

EML Party types and EML Party attributes can now be required in the editor

Specific EML Party types (e.g. PIs, Co-PIs, etc.) can now be required. In addition, specific EML Party fields for each type can be required, such as requiring all Creators and Contacts to provide a full name, email, and ORCID.

This configuration is stored in emlEditorRequiredFields and emlEditorRequiredFields_EMLParty.

Misc improvements

• Unit and integration tests can now be run and the results parsed server-side. CI tools can now fail builds because of failed Mocha tests. (#1966)
• Set id, system, and scope attributes on EMLProjects (#2000)
• Support defining a view rectangle for imagery layers in Cesium (#2021)

Bug fixes

• Sometimes the abstract was duplicated (#1047, #782)
• Fix EML validation error: At least one method step is required (#2017)
• User dropdown menus were cut off (#2010)
• Cross-origin MetacatUI setups were not displaying image previews (#1982 )

2.20.0

New features 🎉

Support for downloading datasets with folder structures

The DataONE API, implemented by Metacat, now supports adding folders to datasets so that when users download a dataset via the Download All button, the data directory in the BagIt .zip file has user-defined folders to organize the data.

This is currently only supported for packages that add the folder structure by editing the resource map RDF XML. Support for displaying and adding folders with MetacatUI is in the works for the future.

By default, this option is turned off since it requires an upgrade to Metacat 2.17.0. This allows MetacatUI owners to upgrade MetacatUI before Metacat is upgraded. To use this new download BagIt format, set AppConfig#packageFormat to application%2Fbagit-1.0

Several Cesium improvements

Show notifications in Cesium layers

Add a notification to a Cesium layer to show visitors to a portal that a data alyer in Cesium is New, just a Preview, or whatever custom notification you'd like.

This adds a badge element next to the layer name in the layers list, and a longer message in the layer details panel:

Five styles of notifications are available:

See #1981

Made the zoom button in Cesium more obvious

See #1976

Add option to have layer panel opened by default

Set the MapConfig#toolbarOpen option to true to have the layer panel opened by default when Cesium is first loaded onto the page.

See #1971

Misc. imporvements

• The project field is now retrieved during searches from the DataCatalogViewWithFilters, which is used by the portal searches #1992

Bug fixes 🛠️

• 'Cancel' button for filter editor was not working #1975
• Handle empty EML individualName nodes #1958
• Fix failing tests in EMLNonNumericDomain and EMLEntity suites. These tests were trying to test behavior of EML that isn't valid (adding an XML id to the EMLNonNumericDomain) or that isn't handled by MetacatUi at all, but now is (adding an XML id to entities when there isn't a connected DataONEObject) #1416
• Fixed error in test suite where the global variable MetacatUI was not accessible, preventing much of the app from being covered by unit tests. #509

Developer notes

Important information that software developers who use MetacatUI for their repository should know

Possible breaking changes in deployments with a customized index.html or AppView

In order to support full-app unit tests (#509), MetacatUI needs to be intialized on the page without rendering the views and visual elements of the app. This requires the AppView.render() and AppView.initiliaze() methods to check that Appview.el exists on the page before rendering. If there is no AppView element on the page, the function will exit without rendering anything, causing the application to be a blank white page.

This does mean there may be a breaking change for MetacatUI owners who have changed their AppView.el to a different element (.e.g body or a different id. The default is #metacatui-app) OR have changed their index.html enough to remove the #metacatui-app element.

However this is only for advanced owners who have made significant changes to their MetacatUI deployments and should already be aware of that tech debt.

New configuration options

• MapConfig#Notification - A notification displays a badge in the LayerListView and a message in the LayerDetailsView. This is useful for indicating some special status of the layer: "new", "under development", etc.
• MapConfig#toolbarOpen - Whether or not the toolbar is open when the map is initialized. Set to false by default, so that the toolbar is hidden by default.

See the complete changelog: 2.19.0...2.20.0

2.19.0

New features 🎉

Create custom Methods in the submission form

The dataset editor now supports guided questions in the Methods section in addition to the generic numbered Method Steps. This is useful for repositories that want to get specific methods information from their researchers. (#1864)

These custom method questions are saved to the EML document as an EML Method Step with a custom title.

For example, the Arctic Data Center will be using this feature to collect information about Ethical Research Practices.

By default, custom methods are disabled. See the customEMLMethods configuration documentation for info on how to enable these in your MetacatUI.

New Data Sensitivity question in submission form

To further support best practices about storing sensitive data (particularly sensitive information about humans), the dataset editor has a new question that asks the user to specify the sensitivity level of their data. This uses a new ontology, SENSO, to create a semantic annotation in the EML at the dataset level. DataTags are also displayed for each category. If the researcher picks a minimal or significant risk level, they are warned to not upload that data yet. (#1863)

To enable this feature, set the enableDataSensitivityInEditor config to true. It is enabled by default. To require this field in the editor, set emlEditorRequiredFields.dataSensitivity to true. It is optional by default.

Search for private datasets

You can now filter your dataset search to only show private datasets with the new Access filter. When checked, any public datasets are excluded from the search results. This should be particularly useful for researchers looking for in-progress datasets shared by their project team and for data managers to find in-progress datasets. (#1902)

To enable this feature, add the isPrivate filter name to the defaultSearchFilters configuration. It is enabled by default.

New Cesium features:

New "Home" button to reset the map

Next to the layers panel is a Home button that zooms the map back to the original position. (#1903)

Support geoJSON as a data layer

Data layers can now be added to Cesium that use geoJSON (in addition to the already supported Cesium 3D Tiles layers and raster image layers.) (#1779)

Feature Info Panels for each data layer are now customizable with templates

In Cesium, clicking on a data entity (e.g. a map marker) shows an "info box" containing the description property or attribute table of that entity. This information is now configurable (e.g. for a geoJSON layer you may want to show info from the JSON such as a date, an image, a link, logo, etc.).

These feature info panel templates are customized using the FeatureTemplate configuration set on each MapAsset
(#1797)

Continuous color palettes

Cesium can now color map features on a gradient based on numeric properties like area, height, etc. Each color and value pair would correspond to a point in a gradient, and any other values between configured values would use the interpolated color, e.g.:

Use the ColorPaletteConfig in the Cesium MapConfig to enable this. (#1952)

Bug fixes 🛠️

• Multiple method steps couldn't be removed at the same time (#1447)
• EML documents with spatialVector or spatialRaster entities were failing to save. #1457 #1633
• EML drafts retrieved via the Copy/Download button in the DraftsView were unescaping XML entities (#1944)
• EML drafts without a title were causing the DraftsView to crash

New App configuration options

• enableDataSensitivityInEditor
• dataSensitivityInfoURL
• customEMLMethods

See the complete changelog: 2.18.1...2.19.0

2.18.1

Bug fix

This patch release includes a fix for a bug that was introduced in 2.18.0 where the tiles on the search map were not showing up.

2.18.0

MetacatUI 2.18.0

New features 🎉

🌎 Introducing Cesium maps for MetacatUI 🌏

MetacatUI can now display Cesium 3D Tiles in an interactive Cesium 3D world map. Data is added to the map as layers which display in a layer selection list.

To see a live example of Cesium in MetacatUI portals, take a look at the Permafrost Discovery Gateway portal in the Arctic Data Center (After Monday, Dec 6).

Cesium maps are configured for MetacatUI using JSON. Cesium 3D Tiles can be hosted on Cesium Ion or self-hosted.

Currently, Cesium maps are only supported in portals as a new page type (in addition to the already supported Data, Metrics, and Freeform pages). Portals can use this new page type by manually editing the portal XML document with the Cesium configuration. (Cesium pages cannot be added to portals via the portal editor yet.)

In future versions, MetacatUI will replace the Google Maps interactive map on the search pages with Cesium. Track progress of Cesium features in MetacatUI in Github by following the cesium label.

See the full documentation for Cesium on the MetacatUI website.

Other improvements

• If someone tries to submit a dataset with missing or invalid information, the edtior will now display a list of everything that needs to be fixed in addition to highlighting the items in the editor. #1915

• The default maximum size for package downloads was increased from 3 GB to 100 GB because of new improvements to Metacat packaging times.

• The Submit button in the dataset editor is now disabled until all files are done uploading. This should prevent people from submitting broken datasets with missing files. #662

Bug fixes 🛠️

• Sometimes changes to the metadata title were being reset when the access policy was updated #1912
• Data attributes in the editor were loading slowly when semantic annotations were enabled and there were many attributes. Annotation widgets are now lazily loaded. #1896
• /signin route for at least the Arctic theme was not working #1893
• Sometimes the editor wouldn't save because data attributes were missing required id attributes #1895
• If a dataset fails to save in the editor, the "replacement" emergency resource map will have the default repository access policy instead of inheriting the access policy of the metadata like it should. This has been fixed. #1878

Developer notes

• To start using the new Cesium features, see the detailed Cesium documentation on the MetacatUI website.

New configuration options

• enableCesium - If enabled (by setting to true), Cesium maps will be used in the interface.
• cesiumToken - Your Access Token for the Cesium API, which can be retrieved from https://cesium.com/ion/tokens.
• bingMapsKey - Your Access Token for the Bing Maps Imagery API, which can be retrieved from https://www.bingmapsportal.com/. Required if any Cesium layers use imagery directly from Bing.

See the complete changelog: 2.17.0...2.18.0

2.17.0

Release Notes

This is a very exciting release of MetacatUI that includes several major new features.

New features 🎉

Custom filter builder for portals!

This long-awaited feature is finally here! You can now build custom filters for the 'Data' page in portals so that portal visitors can filter data in the collection by relevant filters.

For example, the Distributed Biological Observatory project, which is a ship-based research program, may want their community to search for data by vessel or transect.

These custom filters are now easily creatable in the portal editor. The 70+ metadata fields that are already indexed by Metacat can be used to create a filter.

Four filter types are supported:

• Free text
• Date
• Dropdown menus
• Toggles

Create more complex queries in the portal query builder

The capabilities of the portal query builder have been dramatically expanded. It's now easy to create the exact query you need to create a dataset collection.

New capabilities:

• Group related query rules
• Select whether to search for ALL or ANY of the rules in the group (i.e. AND or OR), using a dropdown menu.
• Change whether the search fields and values are ANDed or ORed together by simply clicking the or keyword shown.
• Exclude specific data from the collection by creating a rule group and selecting the Exclude option.

Add EML 2.2.0 Measurement Type semantic annotations

EML 2.2.0 semantic annotations for Measurement Type can now be added and edited in the dataset editor. If MetacatUI is configured with a Bioportal API key, an ontology tree widget is displayed in the EML Entity editor (behind the data file Describe button). Researchers can find the relevant measurement type and add an annotation entirely through MetacatUI.

Note that this feature is disabled by default, since it requires use of the Bioportal service. Set AppConfig.enableMeasurementTypeView to true to enable.

The semantic annotations also have an improved display.

The annotations have separate popovers for the annotation property and value, making annotations easier to understand and read.

Collections-Portals schema 1.1.0 support

Earlier this year, DataONE released the 1.1.0 version of the collections-portals XML schema, which is used by all portals in the DataONE ecosystem. This new version of the schema is the reason why the two above portal filter features are now available!

All portals created in MetacatUI will now be v1.1.0. 1.0.0 portal documents will still be viewable and editable, but they will automatically be upgraded to 1.1.0 any time they are edited.

Other new features from the 1.1.0 release will be in the future for MetacatUI (e.g. custom icons for dropdown menu items, more complex toggle filters, images for associated parties, etc.)

Bug fixes 🛠️

• Fixed weird display of Clear all button on the search page #1834
• JS error after saving a package #1882

Developer notes

• See the how-to guide for customizing filters in the new query builder and filter builder.

We strongly suggest you upgrade Metacat anytime you upgrade to a minor version of MetacatUI, however to be precise:

• The new portal features require a Metacat upgrade to Metacat 2.14.1
• The new semantic annotation display requires an upgrade to Metacat 2.15.1

New configuration options

• enableMeasurementTypeView - Enable or disable showing the MeasurementTypeView in the Editor's attribute modal dialog. The AppModel#bioportalAPIKey must be set to a valid Bioportal API key for the ontology tree widget to work.
• portalSearchFiltersInfoURL - The URL for a webpage where people can learn more about custom portal search filters. If no URL is provided, links to more info about portals will be omitted.

2.16.0

New features 🎉

A new view that displays a list of all portals

Previously, portals could only be accessed by a direct link or by the "My portals" view in the user menu. A new view shows a list of all private and public portals. This is the first step towards creating a full-fledged portals search view so researchers can find data portals by searching by keyword, people, topic, etc.

To access this new view, go to /portals in your MetacatUI deployment. (#1708)

Other improvements

• Combine the Funding Description and Award Number fields in the portal query builder #1757

• Added help text to the provenance editor selection modals, based on user feedback. #1822
  

• Data objects can now be access with URLs such as: view/{metadata-id}#data-id. Example:
  view/urn%3Auuid%3Ad64bca79-7bb7-4eb1-9b25-047fd64f32a9#urn%3Auuid%3Ad64bca79-7bb7-4eb1-9b25-047fd64f32a9. We have always supported and will continue to support direct navigation to data objects by using only the data identifier in the URL, e.g. view/{data-id} #1408

• The user navigation menu was sometimes overlapping other elements on the portals views #1725, #1649

• Tweak BioPortal tree widget behavior to require fewer clicks #1659

• Added fallback behavior in JSON-LD routines for missing abstracts/descriptions. #1803 When the abstract field is missing, you get a description like:

  "description": "No description is available. Visit https://dataone.org/datasets/urn%3Auuid%3A1161a3af-27f0-49ce-be93-515016cb6b75 for complete metadata about this dataset.",

• Added support for Zotero's ZoteroItemUpdated event #1804

Bug fixes 🛠️

• Fix performance issue when MetadataView has many AnnotationViews to render #1813
• renderMyProfile method broken in arctic theme router #1810
• Provenance charts weren't displaying in the DataONE theme sometimes #1823
• The portals list was failing with a red error message when the user was in many groups #1712
• The Clear All button in portals was leaving the last filter selections greyed-out #1570
• New portals were not using the default access policy after 2.15.0 update #1802

Developer notes

New configuration options

• provenanceInfoURL - A URL of a webpage for people to learn more about provenance. If no URL is provided, links to more info about provenance will be omitted.

2.15.1

Bug fixes

• Urgent bug fixes related to 2.15.0 where the access policy on resource maps and data objects were not always set correctly.
• Minor fix to portal label URL that shows up in the Settings tab of the portal editor #1717