[ENH] Add BIDS URIs and deprecate relative paths, RawSources and (possibly unused) BasedOn #918

effigies · 2021-10-29T16:54:41Z

This PR replaces #820, which I'm leaving open for comparison purposes.

I have moved BIDS URI as a sub-section of URI, and drastically reduced the amount of text, including justification and examples. I have also reduced the scope to purely include files in BIDS datasets, as standard URIs are sufficient for other resources.

I am happy to consider re-adding the justification and some examples to an appendix, but I think keeping Common Principles lean is a good idea.

One significant semantic change: Reading up on file:/// URLs, there is no scope for these to be relative. Relative paths are simply that, and are resolved based on context. I have thus permitted DatasetLinks to include derivatives/xyz as a valid location.

closes #790
closes #471
closes #718
#757 (or at least, pave the way for it to be closed)

src/02-common-principles.md

sappelhoff

thanks for narrowing the scope of this and writing it up so clearly. I am gonna do a double check of all fields that would need to be adjusted - but perhaps you already caught them all.

Some comments meanwhile.

src/02-common-principles.md

sappelhoff

I think the BasedOn field is the only one that remains to be adjusted (right now).

Taken from my list in #820 ... Some of the items I had on there were invalid, one of them is now deprecated, and the rest should be in this PR - and only BasedOn is missing as far as I can see.

IntendedFor
SpatialReference (invalid)
AssociatedEmptyRoom
B0FieldIdentifier (invalid)
B0FieldSource (invalid)
BasedOn
RawSources (deprecated)
Sources

effigies · 2021-11-08T13:53:04Z

Should BasedOn be Sources? It's only specified in an appendix as a qMRI-specific derivative metadata field, and seems nearly identical to RawSources (which will be merged into Sources).

cc @agahkarakuzu @Gilles86 for input.

agahkarakuzu · 2021-11-08T22:00:57Z

Thanks @effigies ! BasedOn may be a field introduced early in BEP001, then not adapted to following conventions. As you noted, RawSources (or Sources after the merge) is identical. I am OK for the modifications this requires, i.e. changing BasedOn --> Sources in the qMRI appendix. Example datasets will need to be updated as well.

effigies · 2021-11-09T01:26:00Z

Thanks, Agah! @sappelhoff I guess technically we should probably deprecate BasedOn like we will RawSources?

sappelhoff · 2021-11-09T09:55:14Z

Thanks, Agah! @sappelhoff I guess technically we should probably deprecate BasedOn like we will RawSources?

Sounds good! Too bad we didn't catch this earlier - but better late than never.

PS: Re: adjusting examples --> running grep -irn "BasedOn" . on bids-examples does not yield anything. So perhaps this field was never used?

nicholst · 2021-11-10T13:19:41Z

I've reviewed this at a high level, and have no comments except to say that issues that I caught in #820 have been addressed here.

effigies · 2021-11-10T14:48:15Z

@sappelhoff I guess for something this big, we should probably patch the validator and update some examples to ensure that somebody following these rules won't be unable to validate.

sappelhoff · 2021-11-11T21:28:48Z

@sappelhoff I guess for something this big, we should probably patch the validator and update some examples to ensure that somebody following these rules won't be unable to validate.

agreed 👍 we should have a PR ready for the validator to merge once this is merged/released ... and updating the examples is a good idea as well.

VisLab · 2021-11-19T16:47:28Z

I have gone over the changes and support this pull request.

commit 4fef34d Author: Christopher J. Markiewicz <markiewicz@stanford.edu> Date: Fri Oct 29 11:50:22 2021 -0400 RF: Narrow scope and reorganize BIDS URI section commit 7f04af7 Merge: 704bedc f21e9b2 Author: Stefan Appelhoff <stefan.appelhoff@mailbox.org> Date: Thu Sep 2 17:29:48 2021 +0200 Merge branch 'master' into enh/point_to_data commit 704bedc Author: Stefan Appelhoff <stefan.appelhoff@mailbox.org> Date: Thu Aug 12 10:36:25 2021 +0200 address wording suggested by Kay commit 0c21ae5 Author: Stefan Appelhoff <stefan.appelhoff@mailbox.org> Date: Wed Aug 11 11:40:29 2021 +0200 clarify derivatives DATASET versus FOLDER commit 554e1e1 Author: Stefan Appelhoff <stefan.appelhoff@mailbox.org> Date: Wed Aug 11 11:30:57 2021 +0200 extend examples, docs, and recommendations for REMOTE bids uris commit 2d49121 Author: Stefan Appelhoff <stefan.appelhoff@mailbox.org> Date: Wed Aug 11 10:03:03 2021 +0200 change order: within, remote, outside -> within, outside, remote commit 55f988d Author: Stefan Appelhoff <stefan.appelhoff@mailbox.org> Date: Tue Aug 10 13:31:36 2021 +0200 improve 'outside of ds' example commit 475b4ca Author: Stefan Appelhoff <stefan.appelhoff@mailbox.org> Date: Tue Aug 10 12:12:57 2021 +0200 deriv3 -> deriv4 for 4th example commit b82f7ad Author: Stefan Appelhoff <stefan.appelhoff@mailbox.org> Date: Tue Aug 10 11:56:40 2021 +0200 consistently refer to data as dataset where appropriate commit 8bdd55d Author: Stefan Appelhoff <stefan.appelhoff@mailbox.org> Date: Mon Aug 9 14:34:30 2021 +0200 semantic line breaks and add RECOMMENDATIONS commit 1a7d870 Author: Stefan Appelhoff <stefan.appelhoff@mailbox.org> Date: Mon Aug 9 13:59:53 2021 +0200 fix link syntax commit fe547c9 Author: Stefan Appelhoff <stefan.appelhoff@mailbox.org> Date: Mon Aug 9 13:50:44 2021 +0200 fix schema links commit 8a36aa2 Author: Stefan Appelhoff <stefan.appelhoff@mailbox.org> Date: Mon Aug 9 13:30:39 2021 +0200 fix Windows spec to contain a colon: /C/... --> /C:/... commit 3acd887 Author: Stefan Appelhoff <stefan.appelhoff@mailbox.org> Date: Mon Aug 9 13:25:04 2021 +0200 change BIDS URI reserved keyword: bids:local:/... -> bids::/... commit 742db02 Author: Stefan Appelhoff <stefan.appelhoff@mailbox.org> Date: Mon Aug 9 13:15:59 2021 +0200 make spatialReference example URI based again (not BIDS URI) commit 42dd6cd Author: Stefan Appelhoff <stefan.appelhoff@mailbox.org> Date: Mon Aug 9 12:24:30 2021 +0200 misc fixes and additions commit 91e332d Merge: 4b0a18f 9a79dff Author: Stefan Appelhoff <stefan.appelhoff@mailbox.org> Date: Mon Aug 9 11:35:42 2021 +0200 Merge branch 'master' into enh/point_to_data commit 4b0a18f Author: Stefan Appelhoff <stefan.appelhoff@mailbox.org> Date: Mon Jun 28 14:21:41 2021 +0200 improve brainplot example commit 955f78f Author: Stefan Appelhoff <stefan.appelhoff@mailbox.org> Date: Mon Jun 28 14:12:22 2021 +0200 add info on what to do with nested derivatives dirs commit 5ba3b6e Author: Stefan Appelhoff <stefan.appelhoff@mailbox.org> Date: Mon Jun 28 13:58:45 2021 +0200 clarify derivatives spatialreferences commit 29db468 Author: Stefan Appelhoff <stefan.appelhoff@mailbox.org> Date: Mon Jun 28 13:55:21 2021 +0200 update qmri BasedOn to use BIDS URIs commit 79b7e84 Author: Stefan Appelhoff <stefan.appelhoff@mailbox.org> Date: Mon Jun 28 13:50:11 2021 +0200 adjust Sources and RawSources to use BIDS URIs commit 7760f02 Author: Stefan Appelhoff <stefan.appelhoff@mailbox.org> Date: Mon Jun 28 13:44:26 2021 +0200 complete SpatialReferences example commit 78de170 Author: Stefan Appelhoff <stefan.appelhoff@mailbox.org> Date: Wed Jun 23 22:02:13 2021 +0200 fix: add missing word commit 4a87df8 Author: Stefan Appelhoff <stefan.appelhoff@mailbox.org> Date: Mon Jun 14 11:49:30 2021 +0200 address misc feedback on wording by Kay. commit dd08ced Author: Stefan Appelhoff <stefan.appelhoff@mailbox.org> Date: Fri Jun 11 10:03:01 2021 +0200 add missing newline commit 6263793 Author: Stefan Appelhoff <stefan.appelhoff@mailbox.org> Date: Fri Jun 11 10:01:42 2021 +0200 clarify local paths: WINDOWS, UNIX commit e62ee75 Author: Stefan Appelhoff <stefan.appelhoff@mailbox.org> Date: Fri Jun 11 09:49:31 2021 +0200 add warning about non-portable BIDS URIs commit 1872fe7 Author: Stefan Appelhoff <stefan.appelhoff@mailbox.org> Date: Fri Jun 11 09:44:39 2021 +0200 clarify REMOTE example, RECOMMEND DOI over arbitrary servers commit e3d094b Author: Stefan Appelhoff <stefan.appelhoff@mailbox.org> Date: Fri Jun 11 09:28:29 2021 +0200 reorder example sections from easy to hard commit 68821ec Author: Stefan Appelhoff <stefan.appelhoff@mailbox.org> Date: Fri Jun 11 09:26:33 2021 +0200 clarify local references WITHIN ds commit 2385745 Author: Stefan Appelhoff <stefan.appelhoff@mailbox.org> Date: Fri Jun 11 09:14:33 2021 +0200 deprecate at 1.7.0, not 1.6.1 commit 49b6662 Merge: 430429d 17b5444 Author: Stefan Appelhoff <stefan.appelhoff@mailbox.org> Date: Tue Jun 8 20:48:38 2021 +0200 Merge branch 'master' into enh/point_to_data commit 430429d Author: Stefan Appelhoff <stefan.appelhoff@mailbox.org> Date: Tue Jun 8 14:23:16 2021 +0200 links commit 9e72f28 Author: Stefan Appelhoff <stefan.appelhoff@mailbox.org> Date: Tue Jun 8 14:18:09 2021 +0200 more links and typos commit 97176ef Author: Stefan Appelhoff <stefan.appelhoff@mailbox.org> Date: Tue Jun 8 14:10:03 2021 +0200 fix more links commit 8d799b4 Author: Stefan Appelhoff <stefan.appelhoff@mailbox.org> Date: Tue Jun 8 14:07:05 2021 +0200 fix links commit 2c80316 Author: Stefan Appelhoff <stefan.appelhoff@mailbox.org> Date: Tue Jun 8 14:04:56 2021 +0200 pacify linter, fix build commit 4446df7 Author: Stefan Appelhoff <stefan.appelhoff@mailbox.org> Date: Tue Jun 8 13:57:00 2021 +0200 fix typos commit 0f3a278 Author: Stefan Appelhoff <stefan.appelhoff@mailbox.org> Date: Tue Jun 8 13:49:58 2021 +0200 overhaul AssociatedEmptyroom fields across the spec commit f2f502e Author: Stefan Appelhoff <stefan.appelhoff@mailbox.org> Date: Tue Jun 8 13:47:35 2021 +0200 overhaul SpatialReference fields across the spec commit 60b437f Author: Stefan Appelhoff <stefan.appelhoff@mailbox.org> Date: Tue Jun 8 13:44:41 2021 +0200 overhaul IntendedFor fields across the spec commit 80a6011 Author: Stefan Appelhoff <stefan.appelhoff@mailbox.org> Date: Tue Jun 8 13:28:49 2021 +0200 add DatasetLinks field to dataset_description.json docs commit b320026 Author: Stefan Appelhoff <stefan.appelhoff@mailbox.org> Date: Tue Jun 8 13:21:08 2021 +0200 add examples for within, outside, remote commit a159e53 Author: Stefan Appelhoff <stefan.appelhoff@mailbox.org> Date: Tue Jun 8 11:33:16 2021 +0200 add reasoning on why relative path is deprecated

…lash

src/05-derivatives/02-common-data-types.md

Co-authored-by: Stefan Appelhoff <stefan.appelhoff@mailbox.org>

src/05-derivatives/02-common-data-types.md

src/02-common-principles.md

sappelhoff

apart from my code suggestion on leading slashes (based on bids-standard/bids-examples#304 (comment)), I think this can finally go in

Co-authored-by: Stefan Appelhoff <stefan.appelhoff@mailbox.org>

effigies requested review from chrisgorgo and sappelhoff as code owners October 29, 2021 16:54

effigies force-pushed the enh/bids_uri branch from e487716 to 78553ae Compare October 29, 2021 16:58

effigies mentioned this pull request Oct 29, 2021

[ENH] Major update to pointing to files within, outside of, or remote to a current BIDS dataset #820

Closed

satra reviewed Oct 30, 2021

View reviewed changes

src/02-common-principles.md Outdated Show resolved Hide resolved

sappelhoff reviewed Nov 8, 2021

View reviewed changes

src/02-common-principles.md Outdated Show resolved Hide resolved

src/02-common-principles.md Show resolved Hide resolved

src/02-common-principles.md Outdated Show resolved Hide resolved

src/02-common-principles.md Outdated Show resolved Hide resolved

sappelhoff mentioned this pull request Nov 8, 2021

[ENH] Add Sources link to all raw datatypes sidecar JSON #906

Closed

sappelhoff reviewed Nov 8, 2021

View reviewed changes

sappelhoff mentioned this pull request Nov 8, 2021

[MISC] add "forward slash" requirement for paths to common principles #867

Merged

effigies changed the title ~~ENH: Add BIDS URIs, deprecate relative paths~~ ENH: Add BIDS URIs and deprecate relative paths, RawSources and (possibly unused) BasedOn Nov 9, 2021

sappelhoff added this to the 1.7.0 milestone Nov 9, 2021

Remi-Gau mentioned this pull request Nov 11, 2021

[FIX] Update synthetic derivatives bids-standard/bids-examples#298

Merged

6 tasks

effigies added 8 commits November 19, 2021 12:02

RF: Refer to dataset-relative paths as relative paths, drop leading s…

b84f935

…lash

ENH: Mark dataset-name as optional, consistently use <relative-path>

f65fb36

ENH: Clarify sheme-less URIs in DatasetLinks

6ef0073

FIX: Direct readers to resolution section for local URIs

a311eac

FIX: Use relative paths in all BIDS URIs

f383385

ENH: Qualify datasets as BIDS datasets when not absolutely clear

300771b

RF: Deprecate BasedOn, recommend Sources + BIDS URI

d9733f0

effigies force-pushed the enh/bids_uri branch from c53ba6d to d9733f0 Compare November 19, 2021 17:05

effigies mentioned this pull request Dec 7, 2021

BIDS URI support bids-standard/legacy-validator#1393

Closed

2 tasks

sappelhoff mentioned this pull request Jul 21, 2022

Relax NIfTI IntendedFor check when BIDS-URIs are used bids-standard/legacy-validator#1409

Merged

sappelhoff requested a review from erdalkaraca as a code owner July 21, 2022 17:27

fix typo: absolute -> relative

e13ab6b

sappelhoff requested a review from ezemikulan as a code owner July 21, 2022 17:27

Merge branch 'master' into enh/bids_uri

74b5012

sappelhoff changed the title ~~ENH: Add BIDS URIs and deprecate relative paths, RawSources and (possibly unused) BasedOn~~ [ENH] Add BIDS URIs and deprecate relative paths, RawSources and (possibly unused) BasedOn Jul 21, 2022

sappelhoff added 4 commits July 21, 2022 19:39

add display_name

8747b7d

prepend SPEC_ROOT to links from schema to spec

85159d5

fix links: html -> md

63513b8

missed a few

afb2e4e

sappelhoff reviewed Jul 22, 2022

View reviewed changes

src/05-derivatives/02-common-data-types.md Show resolved Hide resolved

Update src/05-derivatives/02-common-data-types.md

7cb10e0

Co-authored-by: Stefan Appelhoff <stefan.appelhoff@mailbox.org>

effigies commented Jul 22, 2022

View reviewed changes

src/05-derivatives/02-common-data-types.md Outdated Show resolved Hide resolved

effigies and others added 2 commits July 22, 2022 12:43

Update src/05-derivatives/02-common-data-types.md

7c202a9

remove trailing '/' from '../../', add link to URIs definition

81fb972

sappelhoff reviewed Jul 23, 2022

View reviewed changes

src/02-common-principles.md Outdated Show resolved Hide resolved

sappelhoff approved these changes Jul 23, 2022

View reviewed changes

Update src/02-common-principles.md

13e5f71

Co-authored-by: Stefan Appelhoff <stefan.appelhoff@mailbox.org>

sappelhoff merged commit 6e2a0b0 into bids-standard:master Jul 23, 2022

bids-maintenance added a commit that referenced this pull request Jul 23, 2022

[DOC] Auto-generate changelog entry for PR #918

a3e5df2

yarikoptic mentioned this pull request Jul 25, 2022

[ENH] Allow for - (dash) (and/or +) in <label> #1165

Open

VisLab mentioned this pull request Jul 28, 2022

Need more generalized schema loading hed-standard/hed-python#525

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[ENH] Add BIDS URIs and deprecate relative paths, RawSources and (possibly unused) BasedOn #918

[ENH] Add BIDS URIs and deprecate relative paths, RawSources and (possibly unused) BasedOn #918

effigies commented Oct 29, 2021 •

edited by sappelhoff

Loading

sappelhoff left a comment

sappelhoff left a comment

effigies commented Nov 8, 2021

agahkarakuzu commented Nov 8, 2021

effigies commented Nov 9, 2021

sappelhoff commented Nov 9, 2021 •

edited

Loading

nicholst commented Nov 10, 2021

effigies commented Nov 10, 2021

sappelhoff commented Nov 11, 2021

VisLab commented Nov 19, 2021

sappelhoff left a comment

[ENH] Add BIDS URIs and deprecate relative paths, RawSources and (possibly unused) BasedOn #918

[ENH] Add BIDS URIs and deprecate relative paths, RawSources and (possibly unused) BasedOn #918

Conversation

effigies commented Oct 29, 2021 • edited by sappelhoff Loading

sappelhoff left a comment

Choose a reason for hiding this comment

sappelhoff left a comment

Choose a reason for hiding this comment

effigies commented Nov 8, 2021

agahkarakuzu commented Nov 8, 2021

effigies commented Nov 9, 2021

sappelhoff commented Nov 9, 2021 • edited Loading

nicholst commented Nov 10, 2021

effigies commented Nov 10, 2021

sappelhoff commented Nov 11, 2021

VisLab commented Nov 19, 2021

sappelhoff left a comment

Choose a reason for hiding this comment

effigies commented Oct 29, 2021 •

edited by sappelhoff

Loading

sappelhoff commented Nov 9, 2021 •

edited

Loading