FIX: Revision of `antsBrainExtraction`, better handling edge cases #567

oesteban · 2020-09-16T18:16:58Z

This PR introduces changes to better handle the copying of headers within the workflow, which is a potential source for nipreps/smriprep#127, and addresses some problems that surface when the ATROPOS refinement step fails (nipreps/smriprep#125).

This refactor could be considered a preliminary work on #531.

Updates the nodes with pure python interfaces based on nibabel, minimizing the need for the new copy_header of ANTs' nipype interfaces.
Reorganizes the workflow so that the Atropos refinement is completely self-contained.
If a WM prior is found within the template's structure, then it is mapped on to the individual's brain and fed as weight_image in the N4 correction.
Adds regularization of the WM posterior estimated with ATROPOS, by multiplying with the WM prior mapped from the template. In the future, an easy check could be implemented here to dismiss the ATROPOS refinement if the prior and the posterior do not overlap sufficiently (i.e., the WM was not correctly labeled or segmented)

This commit: - [x] Updates the nodes with pure python interfaces based on nibabel, minimizing the need for the new ``copy_header`` of ANTs' nipype interfaces. - [x] Reorganizes the workflow so that the Atropos refinement is completely self contained. These are the first two steps to address nipreps/smriprep#125.

pull-assistant · 2020-09-16T18:17:07Z

Best reviewed: commit by commit

Optimal code review plan (2 warnings)

fix: refactor the brain extraction workflow

niworkflows/anat/ants.py 40% changes removed in fix: do not run subp...

enh: increase the utilization of the WM prior

enh: select the posterior from ATROPOS with maximal overlap with wm

enh: add brainstem as part of the prior & brainmask for final N4

doc: state that the workflow is not a nipype version of ``antsBrainExt...

sty: run black

fix: do not run subprocess at import

niworkflows/anat/ants.py 57% changes removed in sty: run black [skip...

Apply suggestions from code review [skip ci]

enh: use 32bit precision where @mgxd's suggested

sty: run black [skip ci]

Powered by Pull Assistant. Last update 0eb96fb ... ed4d06e. Read the comment docs.

This commit introduces two changes: - [x] If a WM prior is found within the template's structure, then it is mapped on to the individual's brain and fed as ``weight_image`` in the N4 correction. - [x] Adds regularization of the WM posterior estimated with ATROPOS, by multiplying with the WM prior mapped from the template. In the future, an easy check could be implemented here to dismiss the ATROPOS refinement if the prior and the posterior do not overlap sufficiently (i.e., the WM was not correctly labeled or segmented) Resolves: nipreps/smriprep#125 (at least for ds005/sub-08).

codecov · 2020-09-16T22:38:53Z

Codecov Report

Merging #567 into master will decrease coverage by 5.38%.
The diff coverage is 6.25%.

@@            Coverage Diff             @@
##           master     #567      +/-   ##
==========================================
- Coverage   64.62%   59.23%   -5.39%     
==========================================
  Files          43       42       -1     
  Lines        5215     5264      +49     
  Branches      760      765       +5     
==========================================
- Hits         3370     3118     -252     
- Misses       1688     2009     +321     
+ Partials      157      137      -20

Flag	Coverage Δ
#documentation	`?`
#reportlettests	`?`
#travis	`59.23% <6.25%> (-0.65%)`	⬇️
#unittests	`?`

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
niworkflows/anat/ants.py	`9.70% <6.25%> (-60.92%)`	⬇️
niworkflows/func/util.py	`21.17% <0.00%> (-58.83%)`	⬇️
niworkflows/anat/freesurfer.py	`39.13% <0.00%> (-52.18%)`	⬇️
niworkflows/anat/skullstrip.py	`30.00% <0.00%> (-50.00%)`	⬇️
niworkflows/interfaces/itk.py	`26.31% <0.00%> (-12.04%)`	⬇️
niworkflows/interfaces/fixes.py	`44.44% <0.00%> (-11.12%)`	⬇️
niworkflows/interfaces/bids.py	`87.58% <0.00%> (-8.97%)`	⬇️
niworkflows/interfaces/utils.py	`37.76% <0.00%> (-4.99%)`	⬇️
niworkflows/__init__.py

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9c3eb81...ed4d06e. Read the comment docs.

niworkflows/anat/ants.py

effigies · 2020-09-17T15:03:10Z

niworkflows/anat/ants.py

        (init_aff, norm, [("output_transform", "initial_moving_transform")]),
        (norm, map_brainmask, [
            ("reverse_transforms", "transforms"),
            ("reverse_invert_flags", "invert_transform_flags"),
        ]),
        (map_brainmask, thr_brainmask, [("output_image", "input_image")]),
-        (thr_brainmask, dil_brainmask, [("output_image", "op1")]),
-        (dil_brainmask, get_brainmask, [("output_image", "op1")]),
+        (map_brainmask, inu_n4_final, [("output_image", "weight_image")]),


Having trouble tracing how this originally connected through...

Although not ideal, this might actually help -> https://1942-74174878-gh.circle-artifacts.com/0/docs/_build/no_version_html/html/api/niworkflows.anat.ants.html#niworkflows.anat.ants.init_brain_extraction_wf

Okay, so it looks like we're no longer using Atropos to select white matter at all. What purpose does it now serve?

Yes, we still use ATROPOS:

To refine the binary mask (as originally done by antsBrainExtraction.sh)

To obtain the WM posterior (i.e., select the WM).

In some sense, now we use it even more :D

However, it is a bit implicit - see the disconnect() call below, to replace the final outputs of the brain-extraction workflow with those coming from the ATROPOS workflow.

oesteban · 2020-09-19T07:33:51Z

It seems this refactor still has some problems:

ds054/sub-100003:

I might have to roll back the WM prior utilization and change it to using the brain mask from the template. Look at the remaining inhomogeneity around the cerebellum in ds054/sub-100057:

BTW, this seems pretty idiosyncratic of ds054. But in hindsight, I think I can propose a much cleverer usage of priors that works for ds054 and all others I'm testing on (ds003, ds005, ds051, ds114).

Addresses issues like nipreps#567 (comment)

see nipreps/niworkflows#567 (comment)

oesteban · 2020-09-21T20:18:16Z

The latest commit includes changes to make the workflow more reliable overall, fixing the problems (at least for sub-100003) I detected on ds054.

I will re-run a good bunch of datasets, including the two originally identified as failing (ds005/sub-08 and ds051/sub-06) and ds054.

oesteban · 2020-09-21T20:24:50Z

I have visually confirmed that after a05dddc, the posterior most overlapping with the WM prior is correctly selected for ds051/sub-06.

and set it as weights image in the final N4.

Addresses issues like nipreps#567 (comment)

pep8speaks · 2020-09-22T09:13:41Z

Hello @oesteban, Thank you for updating!

Cheers! There are no style issues detected in this Pull Request. 🍻 To test for issues locally, pip install flake8 and then run flake8 niworkflows.

Comment last updated at 2020-09-22 18:51:16 UTC

oesteban · 2020-09-22T09:14:10Z

Okay, it seems to be working correctly now.

ds005

Checked visually and all subjects (incl. 08) look great now.

ds051

Checked visually and all subjects (incl. 06) look great now.

ds054/sub-100003

Before:

After:

ds054/sub-100057

Before:

After:

ds054/sub-100042

Before:

After:

…traction``

oesteban · 2020-09-22T10:22:00Z

This is ready for merge - final comments @effigies @mgxd ?

niworkflows/anat/ants.py

mgxd

This looks pretty good (looking at the before/after pictures)

niworkflows/anat/ants.py

Co-authored-by: Mathias Goncalves <goncalves.mathias@gmail.com>

oesteban · 2020-09-22T18:57:07Z

Good to go?

oesteban · 2020-09-22T19:24:52Z

Hey @bpinsard, I'd like to squish this into an RC2 before the LTS is cut (and preferably, cut the RC as soon as possible). Do you want to take a look into it before we go ahead? We would like to hear your opinion before merging since you'll be maintaining the LTS.

effigies

I think I roughly follow what happened here. No obvious problems.

oesteban · 2020-09-23T07:46:08Z

I'll take @bpinsard's silence as no-opposition. The sooner we start testing this, the better.

1.3.1 Bug-fix release in the 1.3.x series. Addresses longstanding issues in the anatomical MRI brain extraction workflow. * FIX: Revision of ``antsBrainExtraction``, better handling edge cases (nipreps#567)

oesteban requested a review from mgxd September 16, 2020 18:16

oesteban changed the base branch from master to maint/1.2.x September 16, 2020 18:17

oesteban changed the base branch from maint/1.2.x to master September 16, 2020 18:17

oesteban force-pushed the fix/smriprep-125 branch from 1e42b52 to d4ea1f5 Compare September 16, 2020 21:41

effigies reviewed Sep 17, 2020

View reviewed changes

niworkflows/anat/ants.py Show resolved Hide resolved

effigies reviewed Sep 17, 2020

View reviewed changes

niworkflows/anat/ants.py Show resolved Hide resolved

effigies reviewed Sep 17, 2020

View reviewed changes

oesteban mentioned this pull request Sep 17, 2020

Simple Nibabel + Numpy interfaces to replace those from ANTs #568

Open

oesteban added a commit to oesteban/niworkflows that referenced this pull request Sep 21, 2020

enh: add brainstem as part of the prior & brainmask for final N4

89d3531

Addresses issues like nipreps#567 (comment)

oesteban added a commit to nipreps/fmriprep that referenced this pull request Sep 21, 2020

fix: address issues such as ds054/sub-100003

ee0d054

see nipreps/niworkflows#567 (comment)

oesteban added 2 commits September 22, 2020 01:20

enh: select the posterior from ATROPOS with maximal overlap with wm

18639b7

and set it as weights image in the final N4.

enh: add brainstem as part of the prior & brainmask for final N4

a89688e

Addresses issues like nipreps#567 (comment)

oesteban force-pushed the fix/smriprep-125 branch from 89d3531 to 3464b24 Compare September 22, 2020 08:23

doc: state that the workflow is not a nipype version of ``antsBrainEx…

ea72430

…traction``

oesteban force-pushed the fix/smriprep-125 branch from 76445d5 to eecfc6b Compare September 22, 2020 09:15

sty: run black

7b9ea4f

oesteban force-pushed the fix/smriprep-125 branch from eecfc6b to 7b9ea4f Compare September 22, 2020 09:16

effigies reviewed Sep 22, 2020

View reviewed changes

niworkflows/anat/ants.py Outdated Show resolved Hide resolved

oesteban force-pushed the fix/smriprep-125 branch 2 times, most recently from e5b4a80 to 70fbadf Compare September 22, 2020 15:37

mgxd reviewed Sep 22, 2020

View reviewed changes

niworkflows/anat/ants.py Outdated Show resolved Hide resolved

niworkflows/anat/ants.py Outdated Show resolved Hide resolved

niworkflows/anat/ants.py Outdated Show resolved Hide resolved

niworkflows/anat/ants.py Outdated Show resolved Hide resolved

effigies reviewed Sep 22, 2020

View reviewed changes

niworkflows/anat/ants.py Outdated Show resolved Hide resolved

oesteban and others added 2 commits September 22, 2020 10:58

fix: do not run subprocess at import

988242b

Apply suggestions from code review [skip ci]

ffbb03e

Co-authored-by: Mathias Goncalves <goncalves.mathias@gmail.com>

oesteban force-pushed the fix/smriprep-125 branch from 3898340 to ffbb03e Compare September 22, 2020 17:58

oesteban added 2 commits September 22, 2020 11:50

enh: use 32bit precision where @mgxd's suggested

b2f04c0

sty: run black [skip ci]

ed4d06e

oesteban mentioned this pull request Sep 22, 2020

REL: 1.3.1 #570

Merged

oesteban requested a review from bpinsard September 22, 2020 19:23

effigies approved these changes Sep 22, 2020

View reviewed changes

oesteban merged commit ffe8eb1 into nipreps:master Sep 23, 2020

oesteban deleted the fix/smriprep-125 branch September 23, 2020 07:46

dependabot bot mentioned this pull request Sep 28, 2020

Bump smriprep from 0.6.2 to 0.7.0 HALFpipe/HALFpipe#4

Merged

shnizzedy mentioned this pull request Nov 25, 2020

♻️ Refactor niworkflow-ants for skullstripping FCP-INDI/C-PAC#1403

Closed

8 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

FIX: Revision of `antsBrainExtraction`, better handling edge cases #567

FIX: Revision of `antsBrainExtraction`, better handling edge cases #567

oesteban commented Sep 16, 2020

pull-assistant bot commented Sep 16, 2020 •

edited

Loading

codecov bot commented Sep 16, 2020 •

edited

Loading

effigies Sep 17, 2020

oesteban Sep 21, 2020

effigies Sep 22, 2020

oesteban Sep 22, 2020

oesteban commented Sep 19, 2020 •

edited

Loading

oesteban commented Sep 21, 2020

oesteban commented Sep 21, 2020

pep8speaks commented Sep 22, 2020 •

edited

Loading

oesteban commented Sep 22, 2020

oesteban commented Sep 22, 2020

mgxd left a comment

oesteban commented Sep 22, 2020

oesteban commented Sep 22, 2020

effigies left a comment

oesteban commented Sep 23, 2020

FIX: Revision of antsBrainExtraction, better handling edge cases #567

FIX: Revision of antsBrainExtraction, better handling edge cases #567

Conversation

oesteban commented Sep 16, 2020

pull-assistant bot commented Sep 16, 2020 • edited Loading

codecov bot commented Sep 16, 2020 • edited Loading

Codecov Report

effigies Sep 17, 2020

Choose a reason for hiding this comment

oesteban Sep 21, 2020

Choose a reason for hiding this comment

effigies Sep 22, 2020

Choose a reason for hiding this comment

oesteban Sep 22, 2020

Choose a reason for hiding this comment

oesteban commented Sep 19, 2020 • edited Loading

oesteban commented Sep 21, 2020

oesteban commented Sep 21, 2020

pep8speaks commented Sep 22, 2020 • edited Loading

Comment last updated at 2020-09-22 18:51:16 UTC

oesteban commented Sep 22, 2020

ds005

ds051

ds054/sub-100003

ds054/sub-100057

ds054/sub-100042

oesteban commented Sep 22, 2020

mgxd left a comment

Choose a reason for hiding this comment

oesteban commented Sep 22, 2020

oesteban commented Sep 22, 2020

effigies left a comment

Choose a reason for hiding this comment

oesteban commented Sep 23, 2020

FIX: Revision of `antsBrainExtraction`, better handling edge cases #567

FIX: Revision of `antsBrainExtraction`, better handling edge cases #567

pull-assistant bot commented Sep 16, 2020 •

edited

Loading

codecov bot commented Sep 16, 2020 •

edited

Loading

oesteban commented Sep 19, 2020 •

edited

Loading

pep8speaks commented Sep 22, 2020 •

edited

Loading