One modality to rule them all?

[VisLab]

I hesitate to open this can of worms, but as more auxiliary modalities (e.g., motion, eye-tracking) are incorporated into BIDS, the use of multiple modalities in conjunction for analysis becomes problematic for BIDS, and this is of concern for downstream users of the data.

As far as I can tell, each modality is treated completely independently of each other with no standardized information about how they sync up. I think the motion BEP included time-stamps at one point (I don't know if it was in the final version being incorporated into BIDS), but this is insufficient for analysis in conjunction with modalities such as EEG since the time clocks don't run evenly and these recordings can run for a reasonable length of time.

I would like to raise this issue for consideration.... Does anyone have any thoughts on how this might be addressed in BIDS?
Is anyone interested on working on this?

The only idea that I have thought of so far is to have a "Sync" modality whose data consists of the sync markers for the individual data streams that could be synchronized downstream for analysis (something like available from LSL).
This would preserve the current model of modality independence, and allow the actual imaging data to be as is, but allow tools to synchronize for downstream analysis.

[sappelhoff]

cc @sjeung @JuliusWelzel

[JuliusWelzel]

Dear Kay,
first of all, great thread title ;)

You are absolutely right, it would be great to clarify this in more detail across multiple modalities. For MOTION-BIDS we currently plan to advise using the scans.tsv to synchronize between multiple recordings:
"Synchronising motion data with other modalities can be achieved by using the scans.tsv file, which contains the offsets
between shared files. No main modality is defined which assigns each data type equal importance.
When multiple data types are simultaneously recorded, it may be necessary to match events.tsv files shared alongside one
modality with another." If timestamps per sample are available, we also drafted a paragraph in the spec on why it makes sense to share these as a dedicated channel.
However, I would be happy to help drafting a more general solution.

Best,
Julius

[Remi-Gau]

@VisLab I think this very closely related to this #86

[VisLab]

@VisLab I think this very closely related to this #86

Yes....it is telling that this issue is from 2018.

[VisLab]

For MOTION-BIDS we currently plan to advise using the scans.tsv to synchronize between multiple recordings...

This is a start, but not really the solution. As a side note.... I would strongly lobby to make the scans.tsv file and the acq_time strongly recommended. Subjects do have circadian rhythms which affect the brain and downstream analysts might like to control for that. Further, sometimes we would like to know how much time has elapsed in the session for time-on-task issues, so knowing when the various scans started in the session can be quite important.