-
Notifications
You must be signed in to change notification settings - Fork 1
/
studies:7t:processingpipelines.dwiki
130 lines (89 loc) · 6.99 KB
/
studies:7t:processingpipelines.dwiki
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
====== Processing Pipelines ======
===== Overview =====
=== File Locations ===
^ ^ Location ^
^ Raw | ''/Volumes/Hera/Raw/MRprojects/7TBrainMech/'' |
^ funcs | ''/Volumes/Hera/preproc/7TBrainMech_{mgsencmemm,rest}'' |
^ subjs | ''/Volumes/Hera/Projects/7TBrainMech/subjs/'' |
^ scripts | ''/Volumes/Hera/Projects/7TBrainMech/scripts/'' |
=== Pipeline ===
- Getting Raw Data & BIDS
- MR preprocessing (anat, rest, and task)
- Freesurfer (anatomical ROIs)
- MRSI grid approach
- MRSI roi/atlas approach
Scripts referenced below are relative to ''/Volumes/Hera/Projects/7TBrainMech/scripts/mri/''
=== Links ===
* [[https://github.com/LabNeuroCogDevel/7TBrainMech_scripts|github repo]] - source code management
* [[http://arnold.wpic.upmc.edu/dokuwiki/doku.php?id=studies:7t:processingpipelines|internal wiki]] - this document rendered nicely
* [[https://docs.google.com/spreadsheets/d/1_EdqA8ObwPaqeLd-BJuvXhimotkHG8zvHMTICmGEVyM/edit#gid=2066974508|status gsheet]] - ''status.csv'' upload to google sheets
===== Raw Data to BIDS =====
* Data transfer is over USB sneakernet (2019-08: XNAT may soon change this) because network infrastructure did not allow for remote file access
* raw DICOMS are not organized and have unusual participant IDs. The scanner syncs all dicoms for each session (all protocols) to a single directory. Because of the large number of dicoms for 3depi, this makes folders with >40,000 files in them! As of sometime in 2019-08, the MRRC has started organizing dicoms into per protcol directoires by hand.
* BIDS is not used for scout or R2'
Organization and BIDS conversion can be done by hand or with scripts. The end product is to have folders like
* raw re-organized ''/Volumes/Hera/Raw/BIDS/7TBrainMech/rawlinks/${lunaid}_${yyymmdd}/${seqno}_${protocol_name}_${dcm_cnt}/'' and
* BIDS ''/Volumes/Hera/Raw/BIDS/7TBrainMech/sub-${lunaid}/${yyymmdd}/anat/sub-$lunaid}_T1w.nii.gz''
==== Mirror ====
The USB enclosed hard drive we get usually has the structure ''/Data/Luna/$SCANID''. We want to sync all the new subjects into our local mirror ''/Volumes/Hera/Raw/MRprojects/7TBrainMech/$SCANID''. ''$SCANID'' is created/used by the MRRC and like ''20190125Luna2'' -- the 8 digit scan date, "Luna" and the visit number for that day.
The easiest and slower way to mirror files is to drag and drop.
- Connect the data drive to your local computer
- Drag and drop ''Data/Luna/$SCANID'' to windows share mounted ''/Volumes/Hera/Raw/MRprojects/7TBrainMech/''
The scripted way uses ''000_rsync_fetch_from_usb.bash'':
an rsync command to automate sync when the USB enclosure is connected directly to rhea in the server room.
Running this requires
- the key to access the server room and
- USB mounted to /mnt/usb. see the output of the
* sync script for mount suggestions and
* ''sudo dmesg|tail'' from rhea after inserting USB for the path to the device (likely ''/dev/sdb2'').
==== Parse ====
Once the data is mirrored locally (in ''/Volumes/Hera/Raw/MRprojects/7TBrainMech/$SCANID''), we need to organize it.
Scripts in ''BIDS/'' directory are helpful here.
- Identify the ''$RAWDCMDIR'' folder for a session
* all dicoms in one folder (older format, e.g. ''20180907Luna1/20180908LUNA1DCMALL1A'')
* hand organized probably in ''DCM'' or ''TIEJUN*'' (eg. ''20190719Luna1/20190719LUNA1/DCM/'', ''20190719Luna2/20190719LUNA2/TIEJUN_JREF-LUNA_20190719_163248_312000/'')
- Run linker script
* ''010_gen_raw_links.bash $RAWDCMDIR'' for older all in one folder
* ''000_dcmfolder_201906fmt.bash $RAWDCMDIR'' for newer per protocol folder
- Run BIDS conversion script
* ''BIDS/020_mkBIDS.R'' with no arguments to run against all links
* e.g. ''BIDS/020_mkBIDS.R rawlinks/11735_20190719'' to try just one (see ''ls /Volumes/Hera/Raw/BIDS/7TBrainMech/rawlinks'')
==== Parse Common Failures ====
* No raw directory: there is no folder within ''$SCANID'' containing all the dicoms. Need to talk to MRRC
* Cannot find id. Hard code ''$SCANID'' to ''$LUNAID'' conversion in ''BIDS/func.bash'' (see source for ''getld8_hardcoded()'' function). A postgres database rebuild will also help.
===== Pre/process =====
Scripts can be much friendlier after the data has been organized. Manual steps can mostly be avoided until working with per subject coordinates in ''MRSI_roi''
==== FreeSurfer ====
Freesurfer is done on the Pittsburgh Super Computer. Two scripts manage this. They run without arguments.
- ''FS/001_toPSC.bash'' - copy BIDS t1 anat to PSC and start batch process
- ''FS/002_fromPSC.bash'' - copy FS output from PSC to local
Neither script is destructive (run one at a time but any time). The scripts assume ''~/.ssh/config'' includes a ''host PSC'' section.
==== Functional ====
* ''010_preproc.bash'' handles preprocessing of T1, task, and rest data. It uses the local preprocessing wrapper tool ''pp''.
* see ''pp_status 7TBrainMech_rest MHRest_nost diff'' for status.
==== MRSI ====
=== Grid ===
MRSI depends on FreeSurfer ((anatomical ROIs, done on PSC see ''mri/FS'')) and functional preprocessing ((T1->T2, T1->MNI warps, done by ''mri/010_preproc.bash'')).
* ''mri/MRSI/90_doall.bash'' - will try to run everyone. Processing errors should also suggest work around/break point.
* For specific files used, see ''mri/MRSI/readme.md''
== Grid Common Failures ==
* missing MRSI csv files (probably sync from box data). Talk to MRRC
* MRRC box update format may have changed. see ''mri/001_rsync_MRSI_from_box.bash'' and ''/Volumes/Hera/Raw/MRprojects/7TBrainMech/MRSI_BrainMechR01''
* to organize by hand
- move participant's ''spreadsheet.csv'' to ''/Volumes/Hera/Raw/MRprojects/7TBrainMech/MRSI_BrainMechR01/$MRID/SI1/spreadsheet.csv'' where MRID is what the MR has called the scan, like 20190209Luna1
* Too many/too few B0 scouts. Need to explicitly identify the PFC scout. Hard code in ''force_dir'' array defined by ''mri/MRSI/01_get_slices.bash''
* Resolution of scout is off or SPM causes segfault. add explicit ignore in ''mri/MRSI/Codes_yj/spm_reg_ROIs.m'' to the ''KNOWN_ISSUES'' cell.
* MRID <-> LunaID lookup in ''mri/MRSI/txt/id.txt'' (manual changes will eventually be overwritten by ''mri/MRSI/id_list.bash'')
* unless typo is on MRSI uploaded data side, missing conversion is probably issue upstream. see [[#Parse Common Failures]]
* Freesurfer or preprocessing failed. To fix, fix at previous step.
=== ROI voxel ===
ROI voxel based approach depends on warps generated by the grid based approach. ''./mri/MRSI_roi/readme.md'' has much more information.
This method requires
- organizing files
- identifying center coordinates per subject.
- sending spec. at those coordinates back to MRRC for LC Model
- retrieving model out csv files
==== EEG ====
^ raw | ''/Volumes/Hera/Raw/EEG/7TBrainMech'' ((also linked from ''/Volumes/Hera/Projects/7TBrainMech/raw_eeg'')) |
^ github mirror | https://github.com/LabNeuroCogDevel/7TBrainMech_scripts/tree/master/eeg |
^ box mirror | |