Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[FIX] Normalize data to zero mean and unit variance before dimension estimation #636

Merged
merged 11 commits into from
Feb 5, 2021
Merged

[FIX] Normalize data to zero mean and unit variance before dimension estimation #636

Show file tree
Hide file tree
Changes from 7 commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
65917dd
Normalize data to zero mean and unit variance in ma_pca
notZaki Dec 28, 2020
d9c3dc2
Use normalized data as input to ma_pca
notZaki Jan 15, 2021
c43860d
Revert first change: do not re-scale input in ma_pca
notZaki Jan 15, 2021
34c2836
Use consistent normalization strategy in ma_pca
notZaki Jan 16, 2021
c7e475e
Remove outdated comments
notZaki Jan 16, 2021
68e455e
Merge branch 'master' into normalizeBeforeMAPCA
notZaki Jan 16, 2021
9b72780
Update regression test data; normalization produces fewer figures
notZaki Jan 16, 2021
c60d57e
Merge branch 'master' into normalizeBeforeMAPCA
notZaki Jan 25, 2021
1d89983
Merge branch 'master' into normalizeBeforeMAPCA
notZaki Jan 30, 2021
9f3f952
Normalize along spatial dimension in mapca; Matches GIFT
notZaki Jan 30, 2021
2b385db
Update test output; updated mapca finds 3 more components
notZaki Jan 30, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 2 additions & 4 deletions tedana/decomposition/ma_pca.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -478,9 +478,7 @@ def ma_pca(data_nib, mask_nib, criteria='mdl'):
maskvec = np.reshape(mask_nib, Nx * Ny * Nz, order='F')
data_non_normalized = data_nib_V[maskvec == 1, :]
scaler = StandardScaler(with_mean=True, with_std=True)
# TODO: determine if tedana is already normalizing before this
data = scaler.fit_transform(data_non_normalized) # This was X_sc
data = data_non_normalized
data = scaler.fit_transform(data_non_normalized.T).T # This was X_sc

LGR.info('Performing SVD on original OC data...')
V, EigenValues = _icatb_svd(data, Nt)
Expand Down Expand Up @@ -549,7 +547,7 @@ def ma_pca(data_nib, mask_nib, criteria='mdl'):
dat[:, i] = dat0[mask_s_1d == 1]

# Perform Variance Normalization
dat = scaler.fit_transform(dat)
dat = scaler.fit_transform(dat.T).T

# (completed)
LGR.info('Performing SVD on subsampled i.i.d. OC data...')
Expand Down
3 changes: 0 additions & 3 deletions tedana/tests/data/cornell_three_echo_outputs.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -66,9 +66,6 @@ figures/comp_061.png
figures/comp_062.png
figures/comp_063.png
figures/comp_064.png
figures/comp_065.png
figures/comp_066.png
figures/comp_067.png
dn_ts_OC.nii.gz
feats_OC2.nii.gz
figures
Expand Down
5 changes: 0 additions & 5 deletions tedana/tests/data/nih_five_echo_outputs_verbose.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -110,8 +110,3 @@ figures/comp_062.png
figures/comp_063.png
figures/comp_064.png
figures/comp_065.png
figures/comp_066.png
figures/comp_067.png
figures/comp_068.png
figures/comp_069.png
figures/comp_070.png