`validate_nii` node failing when using SYN SDC #3046

smeisler · 2023-06-16T12:06:43Z

What happened?

I am processing some 7T data (Natural Scenes Dataset), and get unusable SDC when trying to incorporate their fieldmaps (see below), so I am trying to use SYN SDC on the data on a fresh working directory and consistently get the following error.

What command did you use?

singularity run -e --containall -B ${scratch},${templateflow_dir} $fmriprep_IMG $scratch/data $scratch/data/derivatives participant --participant_label ${subject:4} -w $scratch --fs-license-file ${scratch}/license.txt --project-goodvoxels --use-syn-sdc --output-layout legacy --notrack --mem_mb 60000 --nprocs 16 --omp-nthreads 8 --output-spaces T1w fsnative

What version of fMRIPrep are you running?

23.1.1

How are you running fMRIPrep?

Singularity

Is your data BIDS valid?

Yes

Are you reusing any previously computed results?

FreeSurfer

Please copy and paste any relevant log output.

230616-07:51:32,914 nipype.workflow ERROR:
	 Saving crash info to /om2/scratch/tmp/smeisler/fsub_proc/sub-04/data/derivatives/fmriprep/sub-04/log/20230616-075050_0666cd17-27f9-45dc-89e5-2eaba19359d1/crash-20230616-075132-smeisler-validate_nii-ae539a9d-d097-4275-b6f1-093c66d0ef96.txt
Traceback (most recent call last):
  File "/opt/conda/envs/fmriprep/lib/python3.10/site-packages/nipype/pipeline/plugins/multiproc.py", line 292, in _send_procs_to_workers
    num_subnodes = self.procs[jobid].num_subnodes()
  File "/opt/conda/envs/fmriprep/lib/python3.10/site-packages/nipype/pipeline/engine/nodes.py", line 1309, in num_subnodes
    self._check_iterfield()
  File "/opt/conda/envs/fmriprep/lib/python3.10/site-packages/nipype/pipeline/engine/nodes.py", line 1332, in _check_iterfield
    raise ValueError(
ValueError: Input in_file was not set but it is listed in iterfields.


When creating this crashfile, the results file corresponding
to the node could not be found.
230616-07:51:32,926 nipype.workflow ERROR:
	 Node validate_nii failed to run on host node103.
230616-07:51:32,927 nipype.workflow ERROR:
	 Saving crash info to /om2/scratch/tmp/smeisler/fsub_proc/sub-04/data/derivatives/fmriprep/sub-04/log/20230616-075050_0666cd17-27f9-45dc-89e5-2eaba19359d1/crash-20230616-075132-smeisler-validate_nii-a2adfcd2-cda9-4426-b57e-bd4ef81e4b4b.txt
Traceback (most recent call last):
  File "/opt/conda/envs/fmriprep/lib/python3.10/site-packages/nipype/pipeline/plugins/multiproc.py", line 292, in _send_procs_to_workers
    num_subnodes = self.procs[jobid].num_subnodes()
  File "/opt/conda/envs/fmriprep/lib/python3.10/site-packages/nipype/pipeline/engine/nodes.py", line 1309, in num_subnodes
    self._check_iterfield()
  File "/opt/conda/envs/fmriprep/lib/python3.10/site-packages/nipype/pipeline/engine/nodes.py", line 1332, in _check_iterfield
    raise ValueError(
ValueError: Input in_file was not set but it is listed in iterfields.


When creating this crashfile, the results file corresponding
to the node could not be found.
230616-07:51:32,938 nipype.workflow ERROR:
	 Node validate_nii failed to run on host node103.
230616-07:51:32,938 nipype.workflow ERROR:
	 Saving crash info to /om2/scratch/tmp/smeisler/fsub_proc/sub-04/data/derivatives/fmriprep/sub-04/log/20230616-075050_0666cd17-27f9-45dc-89e5-2eaba19359d1/crash-20230616-075132-smeisler-validate_nii-8f97a736-e372-43e9-b4d0-33b0616bb984.txt
Traceback (most recent call last):
  File "/opt/conda/envs/fmriprep/lib/python3.10/site-packages/nipype/pipeline/plugins/multiproc.py", line 292, in _send_procs_to_workers
    num_subnodes = self.procs[jobid].num_subnodes()
  File "/opt/conda/envs/fmriprep/lib/python3.10/site-packages/nipype/pipeline/engine/nodes.py", line 1309, in num_subnodes
    self._check_iterfield()
  File "/opt/conda/envs/fmriprep/lib/python3.10/site-packages/nipype/pipeline/engine/nodes.py", line 1332, in _check_iterfield
    raise ValueError(
ValueError: Input in_file was not set but it is listed in iterfields.

Additional information / screenshots

I am using the Natural Scenes Dataset, which has 7T fMRI and fmap data. I get lots of warping when trying the original fmaps, so when running SDC I ignore the fMRI fmaps.

Tree of subject dir

├── anat
│   ├── sub-04_run-01_T1w.json
│   ├── sub-04_run-01_T1w.nii.gz
│   ├── sub-04_run-01_T2w.json
│   ├── sub-04_run-01_T2w.nii.gz
│   ├── sub-04_run-02_T1w.json
│   ├── sub-04_run-02_T1w.nii.gz
│   ├── sub-04_run-02_T2w.json
│   ├── sub-04_run-02_T2w.nii.gz
│   ├── sub-04_run-03_T1w.json
│   ├── sub-04_run-03_T1w.nii.gz
│   ├── sub-04_run-03_T2w.json
│   ├── sub-04_run-03_T2w.nii.gz
│   ├── sub-04_run-04_T1w.json
│   ├── sub-04_run-04_T1w.nii.gz
│   ├── sub-04_run-05_T1w.json
│   ├── sub-04_run-05_T1w.nii.gz
│   ├── sub-04_run-06_T1w.json
│   └── sub-04_run-06_T1w.nii.gz
├── dwi
│   ├── sub-04_acq-98_dir-AP_dwi.bval
│   ├── sub-04_acq-98_dir-AP_dwi.bvec
│   ├── sub-04_acq-98_dir-AP_dwi.json
│   ├── sub-04_acq-98_dir-AP_dwi.nii.gz
│   ├── sub-04_acq-98_dir-PA_dwi.bval
│   ├── sub-04_acq-98_dir-PA_dwi.bvec
│   ├── sub-04_acq-98_dir-PA_dwi.json
│   ├── sub-04_acq-98_dir-PA_dwi.nii.gz
│   ├── sub-04_acq-99_dir-AP_dwi.bval
│   ├── sub-04_acq-99_dir-AP_dwi.bvec
│   ├── sub-04_acq-99_dir-AP_dwi.json
│   ├── sub-04_acq-99_dir-AP_dwi.nii.gz
│   ├── sub-04_acq-99_dir-PA_dwi.bval
│   ├── sub-04_acq-99_dir-PA_dwi.bvec
│   ├── sub-04_acq-99_dir-PA_dwi.json
│   └── sub-04_acq-99_dir-PA_dwi.nii.gz
├── fmap
│   ├── sub-04_acq-DWI_dir-AP_epi.json
│   ├── sub-04_acq-DWI_dir-AP_epi.nii.gz
│   ├── sub-04_acq-DWI_dir-PA_epi.json
│   ├── sub-04_acq-DWI_dir-PA_epi.nii.gz
│   ├── sub-04_acq-fMRI_run-01_magnitude1.json
│   ├── sub-04_acq-fMRI_run-01_magnitude1.nii.gz
│   ├── sub-04_acq-fMRI_run-01_magnitude2.json
│   ├── sub-04_acq-fMRI_run-01_magnitude2.nii.gz
│   ├── sub-04_acq-fMRI_run-01_phasediff.json
│   ├── sub-04_acq-fMRI_run-01_phasediff.nii.gz
│   ├── sub-04_acq-fMRI_run-02_magnitude1.json
│   ├── sub-04_acq-fMRI_run-02_magnitude1.nii.gz
│   ├── sub-04_acq-fMRI_run-02_magnitude2.json
│   ├── sub-04_acq-fMRI_run-02_magnitude2.nii.gz
│   ├── sub-04_acq-fMRI_run-02_phasediff.json
│   └── sub-04_acq-fMRI_run-02_phasediff.nii.gz
└── func
    ├── sub-04_task-floc_run-01_bold.json
    ├── sub-04_task-floc_run-01_bold.nii.gz
    ├── sub-04_task-floc_run-01_events.tsv
    ├── sub-04_task-floc_run-01_sbref.json
    ├── sub-04_task-floc_run-01_sbref.nii.gz
    ├── sub-04_task-floc_run-02_bold.json
    ├── sub-04_task-floc_run-02_bold.nii.gz
    ├── sub-04_task-floc_run-02_events.tsv
    ├── sub-04_task-floc_run-02_sbref.json
    ├── sub-04_task-floc_run-02_sbref.nii.gz
    ├── sub-04_task-floc_run-03_bold.json
    ├── sub-04_task-floc_run-03_bold.nii.gz
    ├── sub-04_task-floc_run-03_events.tsv
    ├── sub-04_task-floc_run-03_sbref.json
    ├── sub-04_task-floc_run-03_sbref.nii.gz
    ├── sub-04_task-floc_run-04_bold.json
    ├── sub-04_task-floc_run-04_bold.nii.gz
    ├── sub-04_task-floc_run-04_events.tsv
    ├── sub-04_task-floc_run-04_sbref.json
    └── sub-04_task-floc_run-04_sbref.nii.gz

Original warping when using the fmaps

Only non-BIDS valid file is my fitlins statsmodel. No other BIDS warning or errors:

	1: [ERR] Files with such naming scheme are not part of BIDS specification. This error is most commonly caused by typos in file names that make them not BIDS compatible. Please consult the specification and make sure your files are named correctly. If this is not a file naming issue (for example when including files not yet covered by the BIDS specification) you should include a ".bidsignore" file in your dataset (see https://github.com/bids-standard/bids-validator#bidsignore for details). Please note that derived (processed) data should be placed in /derivatives folder and source data (such as DICOMS or behavioural logs in proprietary formats) should be placed in the /sourcedata folder. (code: 1 - NOT_INCLUDED)
		./models/model-floc_desc-6MP5ACompCor_smdl.json
			Evidence: model-floc_desc-6MP5ACompCor_smdl.json

	Please visit https://neurostars.org/search?q=NOT_INCLUDED for existing conversations about this issue.

        Summary:                Available Tasks:                     Available Modalities: 
        77 Files, 1.92GB        NSD Functional Localizer Task        MRI                   
        1 - Subject                                                                        
        1 - Session                                                                        


	If you have any questions, please post on https://neurostars.org/tags/bids.

The text was updated successfully, but these errors were encountered:

smeisler · 2023-06-16T12:11:33Z

P.S. I have this data on OSF (currently a private repo) and can share it if you would want to use it for any tests. Can also add the FreeSurfer outputs or anything else that would expedite testing.

effigies · 2023-06-16T14:03:56Z

What fieldmaps did you try to use before? Phasediff or pepolar?

P.S. I have this data on OSF (currently a private repo) and can share it if you would want to use it for any tests. Can also add the FreeSurfer outputs or anything else that would expedite testing.

Yeah, that'd be good. I'm pretty unlikely to get to anything until after next week, but I can at least run some datasets so I have something to come back to.

smeisler · 2023-06-16T14:07:01Z

What fieldmaps did you try to use before? Phasediff or pepolar?

Phasediff

Yeah, that'd be good. I'm pretty unlikely to get to anything until after next week, but I can at least run some datasets so I have something to come back to.

Sounds good, I'll make sure it's all uploaded and send you a link privately via email or Slack.

marcelzwiers · 2023-06-18T14:47:43Z

I get similar results with phasediff fieldmaps, however when I mask the phasediff images first with mri_synthstrip --border 0 [..] the results look much better (pretty good I'd say)

smeisler · 2023-06-18T14:51:52Z

Thanks @marcelzwiers I will give that a try!

amirhusseinab · 2023-07-03T13:34:12Z

@marcelzwiers thanks for the solution, I'll give it a try. Any suggestions about the fieldmapless method? I was thinking of probably running the fmriprep once, interrupt the process after the synthetic fmap generation, apply masking, but I realized the "magnitude" image is the culprit; it is the actual distorted one.

smeisler · 2023-07-03T16:41:24Z

@amirhusseinab the original error described here happened during the SYN fieldmapless workflow, unfortunately, so I am not sure you will be able to try that. Neat idea though!

smeisler added the bug label Jun 16, 2023

effigies mentioned this issue Jul 14, 2023

FIX: Pass sbref files to SyN workflow #3060

Merged

effigies closed this as completed in fdfb699 Aug 1, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

`validate_nii` node failing when using SYN SDC #3046

`validate_nii` node failing when using SYN SDC #3046

smeisler commented Jun 16, 2023

smeisler commented Jun 16, 2023 •

edited

Loading

effigies commented Jun 16, 2023

smeisler commented Jun 16, 2023 •

edited

Loading

marcelzwiers commented Jun 18, 2023

smeisler commented Jun 18, 2023

amirhusseinab commented Jul 3, 2023

smeisler commented Jul 3, 2023

validate_nii node failing when using SYN SDC #3046

validate_nii node failing when using SYN SDC #3046

Comments

smeisler commented Jun 16, 2023

What happened?

What command did you use?

What version of fMRIPrep are you running?

How are you running fMRIPrep?

Is your data BIDS valid?

Are you reusing any previously computed results?

Please copy and paste any relevant log output.

Additional information / screenshots

smeisler commented Jun 16, 2023 • edited Loading

effigies commented Jun 16, 2023

smeisler commented Jun 16, 2023 • edited Loading

marcelzwiers commented Jun 18, 2023

smeisler commented Jun 18, 2023

amirhusseinab commented Jul 3, 2023

smeisler commented Jul 3, 2023

`validate_nii` node failing when using SYN SDC #3046

`validate_nii` node failing when using SYN SDC #3046

smeisler commented Jun 16, 2023 •

edited

Loading

smeisler commented Jun 16, 2023 •

edited

Loading