Skip to content

Latest commit

 

History

History
101 lines (73 loc) · 5.44 KB

20200818-minutes.md

File metadata and controls

101 lines (73 loc) · 5.44 KB
Raw

Metadata Templates Working Group Minutes

Tuesday, 2020/08/18

10:00 am ET

Attendees:

Terrell Russell, Kory Draughn, Alan King, Dave Fellinger, Ton Smeele (Utrecht), Deep Patel (NIEHS), Mike Conway (NIEHS)

Minutes

Update/Discussion:

  • YODA has incorporated atomic endpoint - looking good so far - part of an update to manage users and groups
  • CEDAR based on JSON-LD, not just JSON-Schema - could be additional requirements
  • COVID-related research... https://www.go-fair.org/implementation-networks/overview/vodan/ - defining metadata vocabularies for COVID research in Netherlands - one is CEDAR-based and one is YODA-based
  • Mike/Kory to work on Jargon testing framework to use atomic metadata API endpoint

USE CASES

  • GETTING DATA FROM CATALOG INTO GUI

    • From GUI - Pass a collection, and 'resolve metadata templates'
    • Look at AVUs associated with that collection
      • AVU each would say I am attribute x of element y
    • Json Array of templates used, and any leftover/orphan elements
      • Return this to the GUI

  • SENDING UPDATES FROM GUI INTO CATALOG

    • Gui, click pencil/edit icon… then editor would be informed by the template information about what kind of drop downs / counters to draw for each item
    • Submitting, would go back through converter to get applied/saved in the database

  • Get/show all templates above this point that are involved in this point of the tree

  • Allow optional / available (not required) templates to be included/used

  • Possible labels / classification around multiple templates themselves - for findability / organization

ELEMENTS OF ARCHITECTURE

  • CREATION/DEFINITION of templates (punt to CEDAR / others)
  • HOSTING of templates (perhaps CEDAR, perhaps irods.org or github)
  • BINDING/MANAGEMENT of templates to collections/data (part of MTWG MVP)
  • USE of templates in GUI (part of MTWG MVP)

VERBS

  • BIND - external reference, cache it, version it…
    • imports json, marks its origin, creates a guid
Puts it into irods
    • template binding is then always to an irods object via path or avu

  • RESOLVE - given logical path, give me universe of templates bound there and above
  • MERGE - template + content -> AVU and back again
    • EITHER... "ikea" bag of templates and data pairs… or pre-assembled merged single json payload…
      • Probably go with the first, so applications can decide to show/ignore some of the metadata
      • Or BOTH available in the API, so applications can choose which they want to use?

TODO: Generate first draft of Swagger MT-iRODS API

  • List overall available MT in the pool
  • Add external MT into 'pool'
  • List attached MTs on an object/collection
  • Attach/Apply MT to an object/collection as required/optional
  • Remove MT from an object/collection
  • Resolve instance of MT based on an object/collection JSON
  • Resolve json schema(s) that defines the metadata to be applied via template X to collection Y
  • BONUS - Rasterize / Pre-assemble? Set of nested/attached schemas down into a single schema

YODA is using server-side functions, but exposed via Swagger API for lightweight clients - all consistent. We would like to emulate that approach with MT...

To implement the Swagger documented metadata template interface, one could prototype reusing components that we have developed for use in Yoda.

The basic idea is that Python rules residing in the iRODS server are tagged as part of an API using a decorator. iRODS clients, e.g. browser or likewise, can call the rule in a standardized way. The processing includes argument handling/validation and error handling.

To give you a flavor of what we are doing, the Yoda API is being defined in OpenAPI format, this can be rendered in Swagger: https://petstore.swagger.io/?url=https://utrechtuniversity.github.io/irods-ruleset-uu/api.json#/browse/post_browse_folder NB: this is in development, we are refactoring relevant Yoda functions to be part of the API.

Now the relevant API components are:

  1. CLIENT/BROWSER: some javascript code to execute client side, wraps an Ajax POST call to the web server
  1. WEB SERVER: passes the rule call onto iRODS
  • calls requested rule
  • passes the JSON encoded argument object to the rule
  • returns STDOUT from rule back to client (not much magic in here, just calling the rule)
  1. iRODS PYTHON RULE ENGINE: processes the api call

The decorator code and the javascript wrapper code are generic elements that help to make life very easy for the client application developer and for rule developers.