Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

(re)design metadata extractors and "harmonization" across multiple extractors whenever needed #701

Closed
1 task
yarikoptic opened this issue Jul 6, 2021 · 1 comment

Comments

@yarikoptic
Copy link
Member

follow up to #698 on indeed us coming up on time to RF our metadata extraction etc.
Insofar we relied heavily on extraction only from NWB files and otherwise just providing a very minimal common metadata (see e.g. https://github.com/dandi/dandi-cli/pull/693/files#diff-efe33fb24af3f1d5572d20580fa315cc3b2a8d6a69992c260fd900f643f2e924R663) .

And even thinking about BIDS support brings interesting use-cases/decisions to make in how we should (re-)design metadata extraction

  • BIDS adheres to "inheritance" principle so having the data file (and even its immediate sidecar .json) might not be sufficient! It would require reading out some .json side car file level above.
    • I think we should defer on all such logic to PyBIDS. FTR, that is what used by datalad-neuroimaging bids metadata extractor (see here) adding only some lean harmonization on top (here we might need to do more)
    • Thus metadata extractor might need to be "dandiset-aware" (unlike for pure .nwb file - we just need that file alone)
  • BIDS supports NWB files. So when extracting metadata for an .nwb file within BIDS dataset, what should take precedence and/or could overload - metadata in nwb or BIDS?
    • Since BIDS allows for more flexible metadata changes (just change a side car) I guess we should allow BIDS to overload metadata extracted from NWB.
    • Will we ever want to contain metadata fields coming from NWB and BIDS?

I even wonder how much we could "harmonize" DANDI metadata extraction/extractors with what DataLad is doing (or going todo, metadata extractors were refactored in https://github.com/datalad/datalad-metalad/tree/master/datalad_metalad/extractors, I am yet to familiarize myself with details).

@satra and @jwodder please chime in.

@yarikoptic
Copy link
Member Author

I will consider this superseded by later #1082

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant