[SCHEMA] Define common derivatives rules #1072

rwblair · 2022-04-14T16:23:49Z

cherry-picked commits from schema/derivatives branch

…r files

codecov · 2022-04-14T16:28:14Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 71.50%. Comparing base (dca1564) to head (833074a).

Additional details and impacted files

@@              Coverage Diff               @@
##           schema-sprint    #1072   +/-   ##
==============================================
  Coverage          71.50%   71.50%           
==============================================
  Files                  9        9           
  Lines                930      930           
==============================================
  Hits                 665      665           
  Misses               265      265

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

src/schema/rules/datatypes/derivatives.yaml

src/schema/rules/sidecars/common_derivative.yaml

src/schema/rules/datatypes/derivatives.yaml

src/schema/rules/sidecars/common_derivative.yaml

src/schema/rules/common_derivatives_validation.yaml

src/schema/rules/datatypes/derivatives.yaml

…cation into schema/common_derivatives

…pecification into schema/common_derivatives

effigies · 2022-04-14T22:09:18Z

anat_parametric:
 $ref: schema.rules.datatypes.anat.parametric
 entities:
   $ref: schema.rules.datatypes.anat.parametric.entities
   space: OPTIONAL
   desc: OPTIONAL

…es. Add anatomical volumetric, mask and segmentation derivatives

src/schema/rules/datatypes/derivatives.yaml

tsalo · 2022-04-15T17:57:14Z

src/schema/rules/datatypes/common_derivatives.yaml

@@ -0,0 +1,413 @@
+---
+anat_nonparametric_common:
+  $ref: schema.rules.datatypes.anat.nonparametric


Part of the goal of #1012 was to eliminate the dependency on the folder structure and filenames for datatype rules. If the group names are unique, then I think we could have schema.rules.datatypes.nonparametric.

@effigies were the names unique?

No, the names weren't unique, but we could make them unique if we reorganize. I think fully-qualified is fine as long as we update the paths when things get moved.

tsalo · 2022-04-15T17:58:05Z

src/schema/rules/datatypes/common_derivatives.yaml

+    desc: OPTIONAL
+
+anat_parametric_common:
+  $ref: schema.rules.datatypes.anat.parametric


Since the group has an entities field, will it conflict with the entities defined below when the ref is filled out?

The idea is that the below entities will override the original. To make it a patch, we pull in the original entities and patch that. So we go from:

anat_parametric_common: $ref: schema.rules.datatypes.anat.parametric entities: $ref: schema.rules.datatypes.anat.parametric.entities space: OPTIONAL desc: OPTIONAL

to

anat_parametric_common: $ref: schema.rules.datatypes.anat.parametric entities: subject: required session: optional run: optional acquisition: optional ceagent: optional reconstruction: optional space: optional desc: optional

to

anat_parametric_common: suffixes: - T1map - T2map - T2starmap - R1map - R2map - R2starmap - PDmap - MTRmap - MTsat - UNIT1 - T1rho - MWFmap - MTVmap - PDT2map - Chimap - S0map - M0map extensions: - .nii.gz - .nii - .json datatypes: - anat entities: subject: required session: optional run: optional acquisition: optional ceagent: optional reconstruction: optional space: optional desc: optional

Co-authored-by: Taylor Salo <tsalo006@fiu.edu>

…cation into schema/common_derivatives

src/schema/rules/tabular_data/derivatives/common_derivatives.yaml

effigies · 2022-04-15T21:40:17Z

src/schema/rules/sidecars/derivatives/common_derivatives.yaml

+SpatialReferenceNoEntity:
+  selectors:
+  - dataset.DatasetType == "derivative"
+  - not(entities contains "space")
+  fields:
+    SpatialReference: REQUIRED


Might be an inconsistency in the spec. I believe that a missing "space" entity is assumed to be space-orig.

src/schema/rules/datatypes/derivatives/common_imaging_derivatives.yaml

src/schema/rules/datatypes/derivatives/common_derivatives.yaml

…g derivatives add missing comma in tabular common_derivatives

Remi-Gau · 2022-04-16T07:51:31Z

src/schema/rules/datatypes/derivatives/common_derivatives.yaml

@@ -0,0 +1,413 @@
+---
+anat_nonparametric_common:
+  $ref: schema.rules.datatypes.anat.nonparametric


Quick FYI as I browsing the several schema related PRs

starting those keys with a $ will for sure not play nice with MATLAB in the end, unless I change them to something else when converting from yaml to json in the bids-matlab repo but I would prefer to avoid having different flavor of the schema.

_ref would presumably work in all languages and be unlikely to collide with valid keys anywhere we might use this construct.

This reverts commit d9e38fa.

…check

…cation into schema/common_derivatives

rwblair added 4 commits April 14, 2022 08:58

common derivatives schema start with no good very bad yaml formatting

df339be

update common derivatives sidecar yaml to look more like other sideca…

e2ba28c

…r files

split derivative checks into own files outside sidecars

4cc1c37

remove nifti_header schemam

acb2d92

rwblair requested a review from tsalo as a code owner April 14, 2022 16:23

effigies changed the title ~~Schema/common derivatives~~ [SCHEMA] Define common derivatives rules Apr 14, 2022

effigies added schema Issues related to the YAML schema representation of the specification. Patch version release. derivatives labels Apr 14, 2022

create derivatives datatypes for use in sidecar rule application

c900dea

rwblair commented Apr 14, 2022

View reviewed changes

src/schema/rules/datatypes/derivatives.yaml Outdated Show resolved Hide resolved

rwblair commented Apr 14, 2022

View reviewed changes

src/schema/rules/sidecars/common_derivative.yaml Outdated Show resolved Hide resolved

rwblair commented Apr 14, 2022

View reviewed changes

src/schema/rules/datatypes/derivatives.yaml Outdated Show resolved Hide resolved

rwblair commented Apr 14, 2022

View reviewed changes

src/schema/rules/sidecars/common_derivative.yaml Outdated Show resolved Hide resolved

effigies reviewed Apr 14, 2022

View reviewed changes

src/schema/rules/common_derivatives_validation.yaml Outdated Show resolved Hide resolved

src/schema/rules/common_derivatives_validation.yaml Outdated Show resolved Hide resolved

Update src/schema/rules/sidecars/common_derivative.yaml

a5ecdc8

effigies reviewed Apr 14, 2022

View reviewed changes

src/schema/rules/datatypes/derivatives.yaml Outdated Show resolved Hide resolved

rwblair and others added 4 commits April 14, 2022 13:16

Merge branch 'schema-sprint' of github.com:bids-standard/bids-specifi…

57f8a21

…cation into schema/common_derivatives

Merge branch 'schema-sprint' of github.com:bids-standard/bids-specifi…

fa3d0cd

…cation into schema/common_derivatives

Hostile takeover

6a32925

Merge branch 'schema/common_derivatives' of github.com:rwblair/bids-s…

4a0a124

…pecification into schema/common_derivatives

generate common_derivatives datatypes from all other existing datatyp…

eb58a91

…es. Add anatomical volumetric, mask and segmentation derivatives

tsalo reviewed Apr 15, 2022

View reviewed changes

src/schema/rules/datatypes/derivatives.yaml Outdated Show resolved Hide resolved

yamllint fix for sidecars common_derivatives

75153bd

tsalo reviewed Apr 15, 2022

View reviewed changes

rwblair and others added 3 commits April 15, 2022 11:00

Update src/schema/rules/datatypes/derivatives.yaml

318e5ad

Co-authored-by: Taylor Salo <tsalo006@fiu.edu>

Merge branch 'schema-sprint' of github.com:bids-standard/bids-specifi…

e834851

…cation into schema/common_derivatives

continue to bring derivatives schema in line with spec.

e230fd8

rwblair added 2 commits April 15, 2022 13:38

fix datatypes/derivatives/common_derivatives.yaml header

425fee6

Merge branch 'schema-sprint' of github.com:bids-standard/bids-specifi…

e3d460c

…cation into schema/common_derivatives