Merge branch 'bids-standard:master' into master

.pre-commit-config.yaml

@@ -33,12 +33,12 @@ repos:
         files: tools/schemacode
         args: ["--settings-file", "tools/schemacode/pyproject.toml"]
   - repo: https://github.com/pyCQA/flake8
-    rev: 6.0.0
+    rev: 6.1.0
     hooks:
       - id: flake8
         args: [--config=tools/schemacode/setup.cfg]
   - repo: https://github.com/pre-commit/mirrors-prettier
-    rev: v3.0.0
+    rev: v3.0.3
     hooks:
       - id: prettier
         files: src/schema/.*/.*\.yaml
@@ -48,7 +48,7 @@ repos:
       - id: codespell
         args: ["--config=.codespellrc", "--dictionary=-", "--dictionary=.codespell_dict"]
   - repo: https://github.com/pre-commit/mirrors-mypy
-    rev: v1.4.1
+    rev: v1.5.1
     hooks:
       - id: mypy
         # Sync with project.optional-dependencies.typing

DECISION-MAKING.md

@@ -30,6 +30,7 @@ Current and past members of the steering group can be found
 | Eric Earl ([@ericearl](https://github.com/ericearl))                      | 2h/week         |                                       | Dec 2021 |
 | Christine Rogers ([@christinerogers](https://github.com/christinerogers)) | 2h/mo           | Interoperability, EEG and multi-modal | Apr 2023 |
 | Nell Hardcastle ([@nellh](https://github.com/nellh))                      | 2h/week         |                                       | Jul 2023 |
+| Kimberly Ray ([@KimberlyLRay](https://github.com/KimberlyLRay))                      | 1h/week         |                                       | Nov 2022 |
 
 In addition to the [BIDS Governance](https://bids.neuroimaging.io/governance.html#bids-maintainers-group)
 classification of a maintainer, maintainers may declare a limited scope of responsibility.

mkdocs.yml

@@ -26,6 +26,7 @@ nav:
       - Glossary: glossary.md
       - BIDS Extension Proposals: extensions.md
       - Appendix:
+        - Schema: appendices/schema.md
         - Contributors: appendices/contributors.md
         - Licenses: appendices/licenses.md
         - Entity table: appendices/entity-table.md

src/CHANGES.md

@@ -3,6 +3,7 @@
 ## [Unreleased](https://github.com/bids-standard/bids-specification/tree/HEAD)
 
 -   \[FIX] Update links to BIDS examples [#1545](https://github.com/bids-standard/bids-specification/pull/1545) ([Remi-Gau](https://github.com/Remi-Gau))
+-   \[ENH] Clarify meaning of raw vs derivative datasets [#1537](https://github.com/bids-standard/bids-specification/pull/1537) ([CPernet](https://github.com/CPernet))
 -   \[FIX] match subject label in folder and filename in func example [#1536](https://github.com/bids-standard/bids-specification/pull/1536) ([Remi-Gau](https://github.com/Remi-Gau))
 -   FIX: Trail MEG directory formats with `/` [#1534](https://github.com/bids-standard/bids-specification/pull/1534) ([effigies](https://github.com/effigies))
 -   \[ENH] Permit CITATION.cff as structured alternative to some dataset_description fields [#1525](https://github.com/bids-standard/bids-specification/pull/1525) ([effigies](https://github.com/effigies))

src/appendices/schema.md

@@ -0,0 +1,19 @@
+# BIDS schema
+
+The BIDS schema is a machine readable representation of the BIDS standard,
+written in a custom YAML format.
+The goal of the schema is to provide a single source for rendering the specification
+and validating BIDS datasets, reducing the scope for inconsistencies.
+Third party tools may also use the schema to write code that will adapt to additions
+to the BIDS standard.
+
+The BIDS schema is available in two machine readable formats:
+
+-   as a set of [YAML](https://en.wikipedia.org/wiki/YAML) files in the [BIDS specification repository](https://github.com/bids-standard/bids-specification/src/schema)
+-   as a [single dereferenced json file](https://bids-specification.readthedocs.io/en/stable/schema.json)
+
+A didactic walkthrough of the schema can be found in the [BEP Guide](https://bids-extensions.readthedocs.io/en/latest/schema/),
+and a complete description is available in the [`bidsschematools` documentation](https://bidsschematools.readthedocs.io/en/latest/).
+[`bidsschematools`](https://pypi.org/project/bidsschematools/) is a Python package sourced
+from the specification repository, and includes the necessary code to render the specification
+and filename validation.

src/derivatives/introduction.md

@@ -4,9 +4,30 @@ Derivatives are outputs of common processing pipelines, capturing data and
 meta-data sufficient for a researcher to understand and (critically) reuse those
 outputs in subsequent processing.
 Standardizing derivatives is motivated by use cases where formalized
-machine-readable access to processed data enables higher level processing.
+machine-readable access to processed data enables higher-level processing.
 
 The following sections cover additions to and divergences from "raw" BIDS.
+Raw data are data that have been curated into BIDS from a non-BIDS source.
+If a dataset is derived from at least one other valid BIDS dataset, then it is a derivative dataset.
+
+Examples:
+
+A defaced T1w image would typically be made during the curation process and is thus under raw
+
+```Text
+sourcedata/private/sub-01/anat/sub-01_T1w.nii.gz
+sub-01/anat/sub-01_T1w.nii.gz
+```
+
+A defaced T1w image could also, in theory, be derived from a BIDS dataset and would thus be under derivatives
+
+```Text
+sub-01/anat/sub-01_T1w.nii.gz
+derivatives/sub-01/anat/sub-01_desc-defaced_T1w.nii.gz
+```
+
+## Derivatives storage and directory structure
+
 Placement and naming conventions for derived datasets are addressed in
 [Storage of derived datasets][storage], and dataset-level metadata is included
 in [Derived dataset and pipeline description][derived-dataset-description].

src/longitudinal-and-multi-site-studies.md

@@ -53,7 +53,8 @@ A guide for using macros can be found at
                 "sub-control01_ses-postdrug_phasediff.json": "",
                 "sub-control01_ses-postdrug_magnitude1.nii.gz": "",
                 }
-            }
+            },
+        "sub-control01_sessions.tsv": "",
         },
     "participants.tsv": "",
     "dataset_description.json": "",
@@ -62,6 +63,19 @@ A guide for using macros can be found at
     }
 ) }}
 
+`sub-control01_sessions.tsv` content:
+
+```Text
+session_id	acq_time	systolic_blood_pressure
+ses-predrug	2009-06-15T13:45:30	120
+ses-postdrug	2009-06-16T13:45:30	100
+```
+
+See this [example dataset](https://github.com/bids-standard/bids-examples/tree/master/7t_trt)
+that has been formatted
+using this specification and can be used
+for practical guidance when curating a new longitudinal dataset.
+
 ## Multi-site or multi-center studies
 
 This version of the BIDS specification does not explicitly cover studies with

src/modality-agnostic-files.md

@@ -5,7 +5,8 @@
 Templates:
 
 -   `dataset_description.json`
--   `README`
+-   `README[.md|.rst|.txt]`
+-   `CITATION.cff`
 -   `CHANGES`
 -   `LICENSE`
 

src/modality-specific-files/magnetoencephalography.md

@@ -89,17 +89,20 @@ which some installations impose to be run on raw data prior to analysis.
 Such processing steps are needed for example because of active shielding software corrections
 that have to be performed to before the MEG data can actually be exploited.
 
-### Recording EEG simultaneously with MEG
+### Recording (i)EEG simultaneously with MEG
 
-Note that if EEG is recorded with a separate amplifier,
+Note that if (i)EEG is recorded with a separate amplifier,
 it SHOULD be stored separately under a new `/eeg` data type
-(see [the EEG specification](electroencephalography.md)).
+(see the [EEG](electroencephalography.md) and
+[iEEG](intracranial-electroencephalography.md) specifications).
 
-If however EEG is recorded simultaneously **with the same MEG system**,
+If however (i)EEG is recorded simultaneously **with the same MEG system**,
 it MAY be stored under the `/meg` data type.
 In that case, it SHOULD have the same sampling frequency as MEG (see `SamplingFrequency` field below).
-Furthermore, the EEG sensor coordinates SHOULD be specified using MEG-specific coordinate
-systems (see [coordinates section](#coordinate-system-json-_coordsystemjson) below and
+Furthermore, (i)EEG sensor coordinates MAY be recorded in an
+[`electrodes.tsv`](electroencephalography.md#electrodes-description-_electrodestsv)
+file using MEG-specific coordinate systems
+(see [Coordinate System JSON](#coordinate-system-json-_coordsystemjson) below and
 the [Coordinate Systems Appendix](../appendices/coordinate-systems.md)).
 
 ### Sidecar JSON (`*_meg.json`)

src/modality-specific-files/task-events.md

@@ -272,7 +272,9 @@ in the accompanying JSON sidecar as follows (based on the example of the previou
         "ScreenDistance": 1.8,
         "ScreenRefreshRate": 60,
         "ScreenResolution": [1920, 1200],
-        "ScreenSize": [0.472, 0.295]
-    }
+        "ScreenSize": [0.472, 0.295],
+        "HeadStabilization": "none"
+    },
+    "VisionCorrection": "lenses"
 }
 ```

src/schema/README.md

@@ -957,7 +957,7 @@ EventsMissing:
     level: warning # could be an error with the proper selectors, I think
   selectors:
     - '"task" in entities'
-    - '!matches(entities.task, "rest")'
+    - '!match(entities.task, "rest")'
     - suffix != "events"
   checks:
     - '"events" in associations'

src/schema/meta/associations.yaml

@@ -24,7 +24,7 @@ events:
 aslcontext:
   selectors:
     - suffix == 'asl'
-    - matches(extension, '\.nii(\.gz)?$')
+    - match(extension, '\.nii(\.gz)?$')
   target:
     suffix: aslcontext
     extension: .tsv
@@ -33,7 +33,7 @@ aslcontext:
 m0scan:
   selectors:
     - suffix == 'asl'
-    - matches(extension, '\.nii(\.gz)?$')
+    - match(extension, '\.nii(\.gz)?$')
   target:
     suffix: m0scan
     extension: [.nii, .nii.gz]
@@ -42,16 +42,16 @@ m0scan:
 magnitude:
   selectors:
     - suffix == 'fieldmap'
-    - matches(extension, '\.nii(\.gz)?$')
+    - match(extension, '\.nii(\.gz)?$')
   target:
     suffix: magnitude
     extension: [.nii, .nii.gz]
   inherit: false
 
 magnitude1:
   selectors:
-    - matches(suffix, 'phase(diff|1)$')
-    - matches(extension, '\.nii(\.gz)?$')
+    - match(suffix, 'phase(diff|1)$')
+    - match(extension, '\.nii(\.gz)?$')
   target:
     suffix: magnitude1
     extension: [.nii, .nii.gz]
@@ -60,15 +60,15 @@ magnitude1:
 bval:
   selectors:
     - suffix == 'dwi'
-    - matches(extension, '\.nii(\.gz)?$')
+    - match(extension, '\.nii(\.gz)?$')
   target:
     extension: .bval
   inherit: true
 
 bvec:
   selectors:
     - suffix == 'dwi'
-    - matches(extension, '\.nii(\.gz)?$')
+    - match(extension, '\.nii(\.gz)?$')
   target:
     extension: .bvec
   inherit: true

src/schema/objects/columns.yaml

@@ -37,6 +37,9 @@ age:
   display_name: Subject age
   description: |
     Numeric value in years (float or integer value).
+
+    It is recommended to tag participant ages that are 89 or higher as 89+,
+    for privacy purposes.
   type: number
   unit: year
 cardiac:

src/schema/objects/entities.yaml

@@ -199,7 +199,7 @@ part:
     `part-real`/`part-imag` pairs of files.
 
     Phase images MAY be in radians or in arbitrary units.
-    The sidecar JSON file MUST include the units of the `phase` image.
+    The sidecar JSON file MUST include the `"Units"` of the `phase` image.
     The possible options are `"rad"` or `"arbitrary"`.
 
     When there is only a magnitude image of a given type, the `part` entity MAY be
@@ -356,7 +356,7 @@ stain:
     although the label may be different.
 
     Descriptions of antibodies SHOULD also be indicated in the `"SamplePrimaryAntibodies"`
-    and/or `"SampleSecondaryAntobodies"` metadata fields, as appropriate.
+    and/or `"SampleSecondaryAntibodies"` metadata fields, as appropriate.
   type: string
   format: label
 subject:

src/schema/objects/metadata.yaml

@@ -500,6 +500,8 @@ ContrastBolusIngredient:
     - CARBON DIOXIDE
     - BARIUM
     - XENON
+    - UNKNOWN
+    - NONE
 DCOffsetCorrection:
   name: DCOffsetCorrection
   display_name: DC Offset Correction
@@ -1326,6 +1328,14 @@ HeadCoilFrequency:
       items:
         type: number
         unit: Hz
+HeadStabilization:
+  name: HeadStabilization
+  display_name: Head stabilization
+  description: |
+    Head restraint method used during the experiment
+    to prevent rotation and/or translation of the head.
+    Example: "chin-rest", "head-rest", "bite-bar", "chin-rest and head-rest", "none"
+  type: string
 HowToAcknowledge:
   name: HowToAcknowledge
   display_name: How To Acknowledge
@@ -3311,6 +3321,7 @@ StimulusPresentation:
     - SoftwareRRID
     - SoftwareVersion
     - Code
+    - HeadStabilization
   properties:
     OperatingSystem:
       $ref: objects.metadata.OperatingSystem
@@ -3543,6 +3554,13 @@ VELChannelCount:
     Number of linear velocity channels.
   type: integer
   minimum: 0
+VisionCorrection:
+  name: VisionCorrection
+  display_name: Vision correction
+  description: |
+    Equipment used to correct participant vision during an experiment.
+    Example: "spectacles", "lenses", "none".
+  type: string
 VolumeTiming:
   name: VolumeTiming
   display_name: Volume Timing