🐛 No resources for warp_longitudinal_T1w_to_template

[shnizzedy]

Describe the bug

241023-08:14:13,673 nipype.workflow INFO:
     Connecting warp_longitudinal_T1w_to_template...
2024-10-23 08:14:13,673 [    INFO] Connecting warp_longitudinal_T1w_to_template...
241023-08:14:13,674 nipype.workflow INFO:
     Connecting warp_longitudinal_T1w_to_template...
2024-10-23 08:14:13,674 [    INFO] Connecting warp_longitudinal_T1w_to_template...
Traceback (most recent call last):
  File "/code/run.py", line 833, in <module>
    raise exception
  File "/code/run.py", line 827, in <module>
    run_main()
  File "/code/run.py", line 796, in run_main
    exitcode = CPAC.pipeline.cpac_runner.run(
  File "/code/CPAC/pipeline/cpac_runner.py", line 425, in run
    run_T1w_longitudinal(sublist, c)
  File "/code/CPAC/pipeline/cpac_runner.py", line 209, in run_T1w_longitudinal
    anat_longitudinal_wf(subject_id, sub_list, cfg)
  File "/code/CPAC/longitudinal_pipeline/longitudinal_workflow.py", line 659, in anat_longitudinal_wf
    wf = connect_pipeline(wf, config, rpool, pipeline_blocks)
  File "/code/CPAC/pipeline/cpac_pipeline.py", line 1127, in connect_pipeline
    wf = nb.connect_block(wf, cfg, rpool)
  File "/code/CPAC/pipeline/engine.py", line 1499, in connect_block
    for pipe_idx, strat_pool in rpool.get_strats(
  File "/code/CPAC/pipeline/engine.py", line 558, in get_strats
    raise LookupError('\n\n[!] C-PAC says: None of the listed '
LookupError: When trying to connect node block 'warp_longitudinal_T1w_to_template' to workflow 'cpac_sub-70009_ses-2' at beginning: 

[!] C-PAC says: None of the listed resources in the node block being connected exist in the resource pool.

Resources:
[]

To reproduce

Try to run a longitudinal pipeline

Custom pipeline configuration

pipeline_config_rbc-options_addRegressors_noBBR_bugfix_funcmask_FSL_long.yml

%YAML 1.1
---
# CPAC Pipeline Configuration YAML file
# Version 1.8.7.dev1
#
# http://fcp-indi.github.io for more info.
#
# Tip: This file can be edited manually with a text editor for quick modifications.
FROM: fmriprep-options

pipeline_setup:

  # Name for this pipeline configuration - useful for identification.
  # This string will be sanitized and used in filepaths
  pipeline_name: RBC_addRegressors_noBBR_bugfixBPF_funcmask_FSL
  output_directory:

    # Quality control outputs
    quality_control:

      # Generate eXtensible Connectivity Pipeline-style quality control files
      generate_xcpqc_files: On

      # Generate quality control pages containing preprocessing and derivative outputs.
      generate_quality_control_images: Off

  system_config:

    # Stop worklow execution on first crash?
    fail_fast: On

    # Random seed used to fix the state of execution.
    # If unset, each process uses its own default.
    # If set, a `random.log` file will be generated logging the random seed and each node to which that seed was applied.
    # If set to a positive integer (up to 2147483647), that integer will be used to seed each process that accepts a random seed.
    # If set to 'random', a random positive integer (up to 2147483647) will be generated and that seed will be used to seed each process that accepts a random seed.
    random_seed: 77742777

longitudinal_template_generation:

  # If you have multiple T1w's, you can generate your own run-specific custom
  # T1w template to serve as an intermediate to the standard template for
  # anatomical registration.

  # This runs before the main pipeline as it requires multiple T1w sessions
  # at once.
  run: On

  # Freesurfer longitudinal template algorithm using FSL FLIRT
  # Method to average the dataset at each iteration of the template creation
  # Options: median, mean or std
  average_method: median

  # Degree of freedom for FLIRT in the template creation
  # Options: 12 (affine), 9 (traditional), 7 (global rescale) or 6 (rigid body)
  dof: 12

  # Interpolation parameter for FLIRT in the template creation
  # Options: trilinear, nearestneighbour, sinc or spline
  interp: trilinear

  # Cost function for FLIRT in the template creation
  # Options: corratio, mutualinfo, normmi, normcorr, leastsq, labeldiff or bbr
  cost: corratio

  # Number of threads used for one run of the template generation algorithm
  thread_pool: 2

  # Threshold of transformation distance to consider that the loop converged
  # (-1 means numpy.finfo(np.float64).eps and is the default)
  convergence_threshold: -1


registration_workflows:
  anatomical_registration:

    # Template to be used during registration.
    # It is not necessary to change this path unless you intend to use a non-standard template.
    T1w_brain_template: $FSLDIR/data/standard/MNI152_T1_${resolution_for_anat}_brain.nii.gz

    # Template to be used during registration.
    # It is not necessary to change this path unless you intend to use a non-standard template.
    T1w_template: $FSLDIR/data/standard/MNI152_T1_${resolution_for_anat}.nii.gz

    # Template to be used during registration.
    # It is not necessary to change this path unless you intend to use a non-standard template.
    T1w_brain_template_mask: $FSLDIR/data/standard/MNI152_T1_${resolution_for_anat}_brain_mask.nii.gz

  functional_registration:
  
    coregistration:

      # functional (BOLD/EPI) registration to anatomical (structural/T1)
      run: On
      func_input_prep:

        # Choose whether to use the mean of the functional/EPI as the input to functional-to-anatomical registration or one of the volumes from the functional 4D timeseries that you choose.
        # input: ['Mean_Functional', 'Selected_Functional_Volume', 'fmriprep_reference']
        input: [Mean_Functional] #changed from fmriprep_reference
        

      boundary_based_registration:

        # this is a fork point
        #   run: [On, Off] - this will run both and fork the pipeline
        run: [Off]

        # reference for boundary based registration
        # options: 'whole-head' or 'brain'
        reference: brain

        # choose which FAST map to generate BBR WM mask
        # options: 'probability_map', 'partial_volume_map'
        bbr_wm_map: partial_volume_map

        # optional FAST arguments to generate BBR WM mask
        bbr_wm_mask_args: -bin

    func_registration_to_template:
      output_resolution:

        # The resolution (in mm) to which the preprocessed, registered functional timeseries outputs are written into.
        # NOTE:
        #   selecting a 1 mm or 2 mm resolution might substantially increase your RAM needs- these resolutions should be selected with caution.
        #   for most cases, 3 mm or 4 mm resolutions are suggested.
        # NOTE:
        #   this also includes the single-volume 3D preprocessed functional data,
        #   such as the mean functional (mean EPI) in template space
        func_preproc_outputs: 2mm

        # The resolution (in mm) to which the registered derivative outputs are written into.
        # NOTE:
        #   this is for the single-volume functional-space outputs (i.e. derivatives)
        #   thus, a higher resolution may not result in a large increase in RAM needs as above
        func_derivative_outputs: 2mm

      target_template:
        T1_template:

          # Standard Skull Stripped Template. Used as a reference image for functional registration.
          # This can be different than the template used as the reference/fixed for T1-to-template registration.
          T1w_brain_template_funcreg: $FSLDIR/data/standard/MNI152_T1_${func_resolution}_brain.nii.gz

          # Standard Anatomical Brain Image with Skull.
          # This can be different than the template used as the reference/fixed for T1-to-template registration.
          T1w_template_funcreg: $FSLDIR/data/standard/MNI152_T1_${func_resolution}.nii.gz

          # Template to be used during registration.
          # It is not necessary to change this path unless you intend to use a non-standard template.
          T1w_brain_template_mask_funcreg: $FSLDIR/data/standard/MNI152_T1_${func_resolution}_brain_mask.nii.gz

          # a standard template for resampling if using float resolution
          T1w_template_for_resample: $FSLDIR/data/standard/MNI152_T1_${func_resolution}_brain.nii.gz

functional_preproc:
  # distortion_correction:

    # using: ['PhaseDiff', 'Blip', 'Blip-FSL-TOPUP']
    #   PhaseDiff - Perform field map correction using a single phase difference image, a subtraction of the two phase images from each echo. Default scanner for this method is SIEMENS.
    #   Blip - Uses AFNI 3dQWarp to calculate the distortion unwarp for EPI field maps of opposite/same phase encoding direction.
    #   Blip-FSL-TOPUP - Uses FSL TOPUP to calculate the distortion unwarp for EPI field maps of opposite/same phase encoding direction.
    # using: [PhaseDiff, Blip-FSL-TOPUP]

  func_masking:
    run: On

    # using: ['AFNI', 'FSL', 'FSL_AFNI', 'Anatomical_Refined', 'Anatomical_Based', 'Anatomical_Resampled', 'CCS_Anatomical_Refined']
    # FSL_AFNI: fMRIPrep-style BOLD mask. Ref: https://github.com/nipreps/niworkflows/blob/a221f612/niworkflows/func/util.py#L246-L514
    # Anatomical_Refined: 1. binarize anat mask, in case it is not a binary mask. 2. fill holes of anat mask 3. init_bold_mask : input raw func → dilate init func brain mask 4. refined_bold_mask : input motion corrected func → dilate anatomical mask 5. get final func mask
    # Anatomical_Based: Generate the BOLD mask by basing it off of the anatomical brain mask. Adapted from DCAN Lab's BOLD mask method from the ABCD pipeline.
    # Anatomical_Resampled: Resample anatomical brain mask in standard space to get BOLD brain mask in standard space. Adapted from DCAN Lab's BOLD mask method from the ABCD pipeline. ("Create fMRI resolution standard space files for T1w image, wmparc, and brain mask […] don't use FLIRT to do spline interpolation with -applyisoxfm for the 2mm and 1mm cases because it doesn't know the peculiarities of the MNI template FOVs")
    # CCS_Anatomical_Refined: Generate the BOLD mask by basing it off of the anatomical brain. Adapted from the BOLD mask method from the CCS pipeline.
    # this is a fork point
    using: ['FSL']

    FSL-BET:

      # Set an intensity threshold to improve skull stripping performances of FSL BET on rodent scans.
      functional_mean_thr:
        run: Off
        threshold_value: 98

      # Bias correct the functional mean image to improve skull stripping performances of FSL BET on rodent scans
      functional_mean_bias_correction: On

      # Apply to 4D FMRI data, if bold_bet_functional_mean_boolean : Off.
      # Mutually exclusive with functional, reduce_bias, robust, padding, remove_eyes, surfaces
      # It must be 'on' if select 'reduce_bias', 'robust', 'padding', 'remove_eyes', or 'bet_surfaces' on
      functional_mean_boolean: Off

      # Set the threshold value controling the brain vs non-brain voxels.
      frac: 0.3

      # Mesh created along with skull stripping
      mesh_boolean: Off

      # Create a surface outline image
      outline: Off

      # Add padding to the end of the image, improving BET.Mutually exclusive with functional,reduce_bias,robust,padding,remove_eyes,surfaces
      padding: Off

      # Integer value of head radius
      radius: 0

      # Reduce bias and cleanup neck. Mutually exclusive with functional,reduce_bias,robust,padding,remove_eyes,surfaces
      reduce_bias: Off

      # Eyes and optic nerve cleanup. Mutually exclusive with functional,reduce_bias,robust,padding,remove_eyes,surfaces
      remove_eyes: Off

      # Robust brain center estimation. Mutually exclusive with functional,reduce_bias,robust,padding,remove_eyes,surfaces
      robust: Off

      # Create a skull image
      skull: Off

      # Gets additional skull and scalp surfaces by running bet2 and betsurf. This is mutually exclusive with reduce_bias, robust, padding, remove_eyes
      surfaces: Off

      # Apply thresholding to segmented brain image and mask
      threshold: Off

      # Vertical gradient in fractional intensity threshold (-1,1)
      vertical_gradient: 0.0

    FSL_AFNI:
      brain_mask: $FSLDIR/data/standard/MNI152_T1_${resolution_for_anat}_brain_mask.nii.gz
      brain_probseg: $FSLDIR/data/standard/MNI152_T1_${resolution_for_anat}_brain_mask.nii.gz

    Anatomical_Refined:

      # Choose whether or not to dilate the anatomical mask if you choose 'Anatomical_Refined' as the functional masking option. It will dilate one voxel if enabled.
      anatomical_mask_dilation: Off

    # Apply functional mask in native space
    apply_func_mask_in_native_space: On
    
  generate_func_mean:

    # Generate mean functional image
    run: On

  truncation:

    # First timepoint to include in analysis.
    # Default is 0 (beginning of timeseries).
    # First timepoint selection in the scan parameters in the data configuration file, if present, will over-ride this selection.
    # Note: the selection here applies to all scans of all participants.
    start_tr: 2

  despiking:

    # Run AFNI 3dDespike
    # this is a fork point
    #   run: [On, Off] - this will run both and fork the pipeline
    run: [On]
    space: template

nuisance_corrections:
  2-nuisance_regression:

    # this is a fork point
    #   run: [On, Off] - this will run both and fork the pipeline
    run: [On]

    # Select which nuisance signal corrections to apply
    Regressors:
      - Name: 36P
        Bandpass:
          bottom_frequency: 0.01
          top_frequency: 0.1
        CerebrospinalFluid:
          erode_mask: Off
          extraction_resolution: 2
          include_delayed: On
          include_delayed_squared: On
          include_squared: On
          summary: Mean
        GlobalSignal:
          include_delayed: On
          include_delayed_squared: On
          include_squared: On
          summary: Mean
        Motion:
          include_delayed: On
          include_delayed_squared: On
          include_squared: On
        WhiteMatter:
          erode_mask: Off
          extraction_resolution: 2
          include_delayed: On
          include_delayed_squared: On
          include_squared: On
          summary: Mean
      - Name: aCompCor24P
        Bandpass:
          bottom_frequency: 0.01
          top_frequency: 0.1
        Motion:
          include_delayed: On
          include_delayed_squared: On
          include_squared: On
        aCompCor:
          summary:
            method: DetrendPC
            components: 5
          tissues:
            - WhiteMatter
            - CerebrospinalFluid
          extraction_resolution: 2    

    # Process and refine masks used to produce regressors and time series for
    # regression.
    regressor_masks:
      erode_csf:

        # Erode cerebrospinal fluid mask in millimeters, default for cerebrospinal fluid is 30mm
        # Cerebrospinal fluid erosion default is using millimeters.
        csf_mask_erosion_mm:

        # Target volume ratio, if using erosion.
        # Default proportion is None for cerebrospinal fluid mask.
        # If using erosion, using both proportion and millimeters is not recommended.
        csf_erosion_prop: 0.9

timeseries_extraction:
  run: On
  connectivity_matrix:

    # Create a connectivity matrix from timeseries data
    # Options:
    #  ['AFNI', 'Nilearn', 'ndmg']
    using: [Nilearn]

  # Enter paths to region-of-interest (ROI) NIFTI files (.nii or .nii.gz) to be used for time-series extraction, and then select which types of analyses to run.
  # Denote which analyses to run for each ROI path by listing the names below. For example, if you wish to run Avg and SpatialReg, you would enter: '/path/to/ROI.nii.gz': Avg, SpatialReg
  # available analyses:
  #   /path/to/atlas.nii.gz: Avg, Voxel, SpatialReg
  tse_roi_paths:
    /ndmg_atlases/label/Human/Glasser_space-MNI152NLin6_res-2x2x2.nii.gz: Avg
    /cpac_templates/Schaefer2018_space-FSLMNI152_res-2mm_desc-200Parcels17NetworksOrder.nii.gz: Avg
    /cpac_templates/Schaefer2018_space-FSLMNI152_res-2mm_desc-400Parcels17NetworksOrder.nii.gz: Avg
    /ndmg_atlases/label/Human/Yeo-17-liberal_space-MNI152NLin6_res-2x2x2.nii.gz: Avg
    /ndmg_atlases/label/Human/Yeo-17_space-MNI152NLin6_res-2x2x2.nii.gz: Avg
    /ndmg_atlases/label/Human/Yeo-7-liberal_space-MNI152NLin6_res-2x2x2.nii.gz: Avg
    /ndmg_atlases/label/Human/Yeo-7_space-MNI152NLin6_res-2x2x2.nii.gz: Avg
    /cpac_templates/Sch400TianS1.nii.gz: Avg
    /cpac_templates/Sch400TianS2.nii.gz: Avg
    /cpac_templates/Sch400TianS3.nii.gz: Avg
    /cpac_templates/Sch400TianS4.nii.gz: Avg
  
amplitude_low_frequency_fluctuation:

  # ALFF & f/ALFF
  # Calculate Amplitude of Low Frequency Fluctuations (ALFF) and fractional ALFF (f/ALFF) for all voxels.
  run: Off

  # space: Template or Native
  target_space: [Template]

regional_homogeneity:

  # ReHo
  # Calculate Regional Homogeneity (ReHo) for all voxels.
  run: Off

  # space: Template or Native
  target_space: [Template]

network_centrality:

  # Calculate Degree, Eigenvector Centrality, or Functional Connectivity Density.
  run: Off

  # Full path to a NIFTI file describing the mask. Centrality will be calculated for all voxels within the mask.
  template_specification_file: /cpac_templates/Mask_ABIDE_85Percent_GM.nii.gz
  degree_centrality:

    # Enable/Disable degree centrality by selecting the connectivity weights
    #   weight_options: ['Binarized', 'Weighted']
    # disable this type of centrality with:
    #   weight_options: []
    weight_options: [Binarized]

  eigenvector_centrality:

    # Enable/Disable eigenvector centrality by selecting the connectivity weights
    #   weight_options: ['Binarized', 'Weighted']
    # disable this type of centrality with:
    #   weight_options: []
    weight_options: []

  local_functional_connectivity_density:

    # Enable/Disable lFCD by selecting the connectivity weights
    #   weight_options: ['Binarized', 'Weighted']
    # disable this type of centrality with:
    #   weight_options: []
    weight_options: [Binarized, Weighted]

# OUTPUTS AND DERIVATIVES
# -----------------------
post_processing:
  spatial_smoothing:

    # Smooth the derivative outputs.
    # Set as ['nonsmoothed'] to disable smoothing. Set as ['smoothed', 'nonsmoothed'] to get both.
    #
    # Options:
    #     ['smoothed', 'nonsmoothed']
    output: [smoothed, nonsmoothed]

    # Tool to use for smoothing.
    # 'FSL' for FSL MultiImageMaths for FWHM provided
    # 'AFNI' for AFNI 3dBlurToFWHM for FWHM provided
    smoothing_method: [AFNI]

    # Full Width at Half Maximum of the Gaussian kernel used during spatial smoothing.
    # this is a fork point
    #   i.e. multiple kernels - fwhm: [4,6,8]
    fwhm: [4,6]

  z-scoring:

    # z-score standardize the derivatives. This may be needed for group-level analysis.
    # Set as ['raw'] to disable z-scoring. Set as ['z-scored', 'raw'] to get both.
    #
    # Options:
    #     ['z-scored', 'raw']
    output: [z-scored, raw]

Run command

#!/usr/bin/bash

#SBATCH --nodes=1
#SBATCH --job-name=prek_rbc
#SBATCH --partition=RM-shared
#SBATCH --ntasks=10
#SBATCH --time=24:00:00
#SBATCH --error=/ocean/projects/med220004p/jclucas/UserSupport/Shinwon/longitudinal/batch_log/1_%x_%A_%a.err
#SBATCH --output=/ocean/projects/med220004p/jclucas/UserSupport/Shinwon/longitudinal/batch_log/1_%x_%A_%a.out
#SBATCH --mail-user=g3q1r0a9i9v0b0f7@cmi-cnl.slack.com
#SBATCH --mail-type=BEGIN
#SBATCH --mail-type=END
#SBATCH --mail-type=FAIL

export DATA="/ocean/projects/med220004p/spark9/02_data/prek_updated_10m/"
export IMAGE="/ocean/projects/med220004p/spark9/01_project/01_prek/02_preproc_all/03_longitudinal_10m/c-pac:release-v1.8.7.post1.dev2.sif"
export ROOT_DIR="/ocean/projects/med220004p/jclucas/UserSupport/Shinwon/longitudinal"
export SUBJECT="70009"
export NCPUS=1         # Disable nipype graph parallelism
export NTHREADS=10
export MEMGB=20        # 2x --ntasks; PSC provisions 2GB per cpu

CMD="apptainer run -B /ocean/projects/med220004p/jclucas/C-PAC/CPAC:/code/CPAC:ro -B $DATA:$DATA:ro -B $ROOT_DIR:$ROOT_DIR $IMAGE $DATA $ROOT_DIR/outputs participant --participant_label $SUBJECT --n_cpus=$NCPUS --num_ants_threads $NTHREADS --skip_bids_validator --save_working_dir --fail_fast True --mem-gb $MEMGB --pipeline_file=$ROOT_DIR/pipeline_config_rbc-options_addRegressors_noBBR_bugfix_funcmask_FSL_long.yml"
echo $CMD
eval $CMD
exitcode=$?
exit exitcode

Expected behavior

Longitudinal pipeline runs or a helpful error message is provided.

Acceptance criteria

• longitudinal pipeline runs to completion

C-PAC version

v1.8.7.post1.dev2

Container platform

Apptainer

Docker and/or Singularity version(s)

Apptainer v1.3.4-1.el8

Additional context

Reported by @Shinwon Park on Slack