Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add adaptive mask plot to report #1073

Merged
merged 18 commits into from
Apr 16, 2024
Merged

Add adaptive mask plot to report #1073

merged 18 commits into from
Apr 16, 2024

Conversation

tsalo
Copy link
Member

@tsalo tsalo commented Apr 10, 2024

Closes #1072.

Changes proposed in this pull request:

  • Create a new function named tedana.reporting.static_figures.plot_adaptive_mask, which creates a figure showing the base mask, the denoising mask (adaptive mask >= 1), and the classification mask (adaptive mask >= 3) overlaid on top of the mean optimally combined image.
  • Add the adaptive mask figure to the HTML report.

@tsalo tsalo added reports issues related to boilerplate generation or visual reports enhancement issues describing possible enhancements to the project labels Apr 10, 2024
Copy link

codecov bot commented Apr 10, 2024

Codecov Report

Attention: Patch coverage is 96.55172% with 1 line in your changes missing coverage. Please review.

Project coverage is 89.81%. Comparing base (62e15ab) to head (b5d5a17).
Report is 29 commits behind head on main.

Files Patch % Lines
tedana/workflows/tedana.py 75.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1073      +/-   ##
==========================================
+ Coverage   89.74%   89.81%   +0.07%     
==========================================
  Files          26       26              
  Lines        3509     3536      +27     
  Branches      619      620       +1     
==========================================
+ Hits         3149     3176      +27     
  Misses        211      211              
  Partials      149      149              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Collaborator

@eurunuela eurunuela left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes look good to me. The only thing I would change is I would make the figure bigger with some css like style='height: 500px' or something like that.

@tsalo tsalo requested a review from eurunuela April 11, 2024 18:44
@eurunuela
Copy link
Collaborator

eurunuela commented Apr 11, 2024

I think that's better. I noticed the figure has some blank area above and below the actual figure itself. This means that effectively, the brains are not going to be 500px in height, but the whole thing with the blank area above and below will be. I don't know where this blank area came from. It must be from the function that generates the image.

See below 👇 The blue area shows the whole <img> component, containing the brains plus the blank areas I mentioned.

CleanShot 2024-04-11 at 20 47 43@2x

@tsalo
Copy link
Member Author

tsalo commented Apr 11, 2024

I dropped the brainplot class and things look better. @eurunuela WDYT?

@tsalo
Copy link
Member Author

tsalo commented Apr 12, 2024

I'm working on improving the figure- the 3-echo one was looking weird.

@tsalo
Copy link
Member Author

tsalo commented Apr 12, 2024

Okay it's looking a lot better now.

Screenshot 2024-04-12 at 10 55 37 AM

Copy link
Member

@handwerkerd handwerkerd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code all looks fine. This will be a useful addition.

Just add explanatory documentation and a picture to the docs. The meaning of the three contours for base optimal combination and classification might not be obvious so explain a bit what they mean.
https://github.com/tsalo/tedana/blob/ab3cbdfbdcd4bac3e4dd42788500c8ad50e0e5a3/docs/outputs.rst#L309

@tsalo
Copy link
Member Author

tsalo commented Apr 12, 2024

@tsalo tsalo requested a review from handwerkerd April 12, 2024 20:11
Copy link
Member

@handwerkerd handwerkerd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor, optional change of wording. Just wanted to clarify what 1 & 3 mean.

docs/outputs.rst Outdated Show resolved Hide resolved
tsalo and others added 3 commits April 12, 2024 17:26
Copy link
Member

@handwerkerd handwerkerd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Copy link
Collaborator

@eurunuela eurunuela left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is great. Thank you @tsalo!

@tsalo tsalo merged commit ee714f3 into ME-ICA:main Apr 16, 2024
16 checks passed
@tsalo tsalo deleted the plot-adaptive-mask branch April 16, 2024 12:04
BahmanTahayori pushed a commit to BahmanTahayori/tedana_florey that referenced this pull request Aug 30, 2024
* Limit current adaptive mask method to brain mask (ME-ICA#1060)

* Limit adaptive mask calculation to brain mask.

Limit adaptive mask calculation to brain mask.

Expand on logic of first adaptive mask method.

Update tedana/utils.py

Improve docstring.

Update test.

Add decreasing-signal-based adaptive mask.

Keep removing.

Co-Authored-By: Dan Handwerker <7406227+handwerkerd@users.noreply.github.com>

* Use `compute_epi_mask` in t2smap workflow.

* Try fixing the tests.

* Fix make_adaptive_mask.

* Update test_utils.py

* Update test_utils.py

* Improve docstring.

* Update utils.py

* Update test_utils.py

* Revert "Update test_utils.py"

This reverts commit 259b002.

* Don't take absolute value of echo means.

* Log echo-wise thresholds in adaptive mask.

* Add comment about non-zero voxels.

* Update utils.py

* Update test_utils.py

* Update test_utils.py

* Update test_utils.py

* Log the thresholds again.

* Address review.

* Fix test.

---------

Co-authored-by: Dan Handwerker <7406227+handwerkerd@users.noreply.github.com>

* Update nilearn requirement from <=0.10.3,>=0.7 to >=0.7,<=0.10.4 (ME-ICA#1077)

* Add adaptive mask plot to report (ME-ICA#1073)

* Update scikit-learn requirement (ME-ICA#1075)

Updates the requirements on [scikit-learn](https://github.com/scikit-learn/scikit-learn) to permit the latest version.
- [Release notes](https://github.com/scikit-learn/scikit-learn/releases)
- [Commits](scikit-learn/scikit-learn@0.21.0...1.4.2)

---
updated-dependencies:
- dependency-name: scikit-learn
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Update pandas requirement from <=2.2.1,>=2.0 to >=2.0,<=2.2.2 (ME-ICA#1076)

Updates the requirements on [pandas](https://github.com/pandas-dev/pandas) to permit the latest version.
- [Release notes](https://github.com/pandas-dev/pandas/releases)
- [Commits](pandas-dev/pandas@v2.0.0...v2.2.2)

---
updated-dependencies:
- dependency-name: pandas
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Update bokeh requirement from <=3.4.0,>=1.0.0 to >=1.0.0,<=3.4.1 (ME-ICA#1078)

Updates the requirements on [bokeh](https://github.com/bokeh/bokeh) to permit the latest version.
- [Changelog](https://github.com/bokeh/bokeh/blob/branch-3.5/docs/CHANGELOG)
- [Commits](bokeh/bokeh@1.0.0...3.4.1)

---
updated-dependencies:
- dependency-name: bokeh
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Load user-defined mask as expected by plot_adaptive_mask (ME-ICA#1079)

* DOC desc-optcomDenoised -> desc-denoised (ME-ICA#1080)

* docs: add mvdoc as a contributor for code, bug, and doc (ME-ICA#1082)

* docs: update README.md

* docs: update .all-contributorsrc

---------

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>

* Identify the last good echo in adaptive mask instead of sum of good echoes (ME-ICA#1061)

* Limit adaptive mask calculation to brain mask.

Limit adaptive mask calculation to brain mask.

Expand on logic of first adaptive mask method.

Update tedana/utils.py

Improve docstring.

Update test.

Add decreasing-signal-based adaptive mask.

Keep removing.

Co-Authored-By: Dan Handwerker <7406227+handwerkerd@users.noreply.github.com>

* Use `compute_epi_mask` in t2smap workflow.

* Try fixing the tests.

* Fix make_adaptive_mask.

* Update test_utils.py

* Update test_utils.py

* Improve docstring.

* Identify the last good echo instead of sum.

Improve docstring.

Update test_utils.py

Update test_utils.py

Fix make_adaptive_mask.

Try fixing the tests.

Use `compute_epi_mask` in t2smap workflow.

Limit adaptive mask calculation to brain mask.

Limit adaptive mask calculation to brain mask.

Expand on logic of first adaptive mask method.

Update tedana/utils.py

Improve docstring.

Update test.

Add decreasing-signal-based adaptive mask.

Keep removing.

Co-Authored-By: Dan Handwerker <7406227+handwerkerd@users.noreply.github.com>

* Fix.

* Update utils.py

* Update utils.py

* Try fixing.

* Update utils.py

* Update utils.py

* add checks

* Just loop over voxels.

* Update utils.py

* Update utils.py

* Update test_utils.py

* Revert "Update test_utils.py"

This reverts commit 259b002.

* Update test_utils.py

* Update test_utils.py

* Remove checks.

* Don't take absolute value of echo means.

* Log echo-wise thresholds in adaptive mask.

* Add comment about non-zero voxels.

* Update utils.py

* Update utils.py

* Update test_utils.py

* Update test_utils.py

* Update test_utils.py

* Log the thresholds again.

* Update test_utils.py

* Update test_utils.py

* Update test_utils.py

* Add simulated data to adaptive mask test.

* Clean up the tests.

* Add value that tests the base mask.

* Remove print in test.

* Update tedana/utils.py

Co-authored-by: Dan Handwerker <7406227+handwerkerd@users.noreply.github.com>

* Update tedana/utils.py

Co-authored-by: Dan Handwerker <7406227+handwerkerd@users.noreply.github.com>

---------

Co-authored-by: Dan Handwerker <7406227+handwerkerd@users.noreply.github.com>

* Output RMSE map and time series for decay model fit (ME-ICA#1044)

* Draft function to calculate decay model fit.

* Calculate root mean squared error instead.

* Incorporate metrics.

* Output RMSE results.

* Output results in tedana.

* Hopefully fix things.

* Update decay.py

* Try improving performance.

* Update decay.py

* Fix again.

* Use tqdm.

* Update decay.py

* Update decay.py

* Update decay.py

* Update expected outputs.

* Add figures.

* Update outputs.

* Include global signal in confounds file.

* Update fiu_four_echo_outputs.txt

* Rename function.

* Rename function.

* Update tedana.py

* Update tedana/decay.py

Co-authored-by: Dan Handwerker <7406227+handwerkerd@users.noreply.github.com>

* Update decay.py

* Update decay.py

* Whoops.

* Apply suggestions from code review

Co-authored-by: Dan Handwerker <7406227+handwerkerd@users.noreply.github.com>

* Fix things maybe.

* Fix things.

* Update decay.py

* Remove any files that are built through appending.

* Update outputs.

* Add section on plots to docs.

* Fix the description.

* Update docs/outputs.rst

Co-authored-by: Dan Handwerker <7406227+handwerkerd@users.noreply.github.com>

* Update docs/outputs.rst

* Fix docstring.

---------

Co-authored-by: Dan Handwerker <7406227+handwerkerd@users.noreply.github.com>

* minimum nilearn 0.10.3 (ME-ICA#1094)

* Use nearest-neighbors interpolation in `plot_component` (ME-ICA#1098)

* Use nearest-neighbors interpolation in plot_stat_map.

* Only use NN interp for component maps.

* Update scipy requirement from <=1.13.0,>=1.2.0 to >=1.2.0,<=1.13.1 (ME-ICA#1100)

Updates the requirements on [scipy](https://github.com/scipy/scipy) to permit the latest version.
- [Release notes](https://github.com/scipy/scipy/releases)
- [Commits](scipy/scipy@v1.2.0...v1.13.1)

---
updated-dependencies:
- dependency-name: scipy
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Update scikit-learn requirement from <=1.4.2,>=0.21 to >=0.21,<=1.5.0 (ME-ICA#1101)

Updates the requirements on [scikit-learn](https://github.com/scikit-learn/scikit-learn) to permit the latest version.
- [Release notes](https://github.com/scikit-learn/scikit-learn/releases)
- [Commits](scikit-learn/scikit-learn@0.21.0...1.5.0)

---
updated-dependencies:
- dependency-name: scikit-learn
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Update numpy requirement from <=1.26.4,>=1.16 to >=1.16,<=2.0.0 (ME-ICA#1104)

* Update numpy requirement from <=1.26.4,>=1.16 to >=1.16,<=2.0.0

Updates the requirements on [numpy](https://github.com/numpy/numpy) to permit the latest version.
- [Release notes](https://github.com/numpy/numpy/releases)
- [Changelog](https://github.com/numpy/numpy/blob/main/doc/RELEASE_WALKTHROUGH.rst)
- [Commits](numpy/numpy@v1.16.0...v2.0.0)

---
updated-dependencies:
- dependency-name: numpy
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

* Use np.nan instead of np.NaN

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Taylor Salo <salot@pennmedicine.upenn.edu>

* Filter out non-diagonal affine warning (ME-ICA#1103)

* Filter out non-diagonal affine warning.

* Fix warning capture.

* Update tedana/reporting/static_figures.py

Co-authored-by: Dan Handwerker <7406227+handwerkerd@users.noreply.github.com>

* Update static_figures.py

---------

Co-authored-by: Dan Handwerker <7406227+handwerkerd@users.noreply.github.com>

* Update bokeh requirement from <=3.4.1,>=1.0.0 to <=3.5.0,>=3.5.0 (ME-ICA#1109)

* Update bokeh requirement from <=3.4.1,>=1.0.0 to <=3.5.0,>=3.5.0

Updates the requirements on [bokeh](https://github.com/bokeh/bokeh) to permit the latest version.
- [Changelog](https://github.com/bokeh/bokeh/blob/branch-3.6/docs/CHANGELOG)
- [Commits](bokeh/bokeh@1.0.0...3.5.0)

---
updated-dependencies:
- dependency-name: bokeh
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

* Update pyproject.toml

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Taylor Salo <salot@pennmedicine.upenn.edu>

* Update scikit-learn requirement from <=1.5.0,>=0.21 to <=1.5.1,>=1.5.1 (ME-ICA#1108)

* Update scikit-learn requirement from <=1.5.0,>=0.21 to <=1.5.1,>=1.5.1

Updates the requirements on [scikit-learn](https://github.com/scikit-learn/scikit-learn) to permit the latest version.
- [Release notes](https://github.com/scikit-learn/scikit-learn/releases)
- [Commits](scikit-learn/scikit-learn@0.21.0...1.5.1)

---
updated-dependencies:
- dependency-name: scikit-learn
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

* Update pyproject.toml to restore minimum version of scikit-learn

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Dan Handwerker <7406227+handwerkerd@users.noreply.github.com>

* Update scipy requirement from <=1.13.1,>=1.2.0 to <=1.14.0,>=1.14.0 (ME-ICA#1106)

* Update scipy requirement from <=1.13.1,>=1.2.0 to <=1.14.0,>=1.14.0

Updates the requirements on [scipy](https://github.com/scipy/scipy) to permit the latest version.
- [Release notes](https://github.com/scipy/scipy/releases)
- [Commits](scipy/scipy@v1.2.0...v1.14.0)

---
updated-dependencies:
- dependency-name: scipy
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

* Update pyproject.toml to retain minimum version of scipy

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Dan Handwerker <7406227+handwerkerd@users.noreply.github.com>
Co-authored-by: Eneko Uruñuela <e.urunuela@icloud.com>

* Update numpy requirement from <=2.0.0,>=1.16 to >=1.16,<=2.0.1 (ME-ICA#1112)

Updates the requirements on [numpy](https://github.com/numpy/numpy) to permit the latest version.
- [Release notes](https://github.com/numpy/numpy/releases)
- [Changelog](https://github.com/numpy/numpy/blob/main/doc/RELEASE_WALKTHROUGH.rst)
- [Commits](numpy/numpy@v1.16.0...v2.0.1)

---
updated-dependencies:
- dependency-name: numpy
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Cleaning up installation instructions (ME-ICA#1113)

* install instructions

* Update docs/installation.rst

Co-authored-by: Taylor Salo <tsalo90@gmail.com>

* Update docs/installation.rst

Co-authored-by: Eneko Uruñuela <e.urunuela@icloud.com>

---------

Co-authored-by: Taylor Salo <tsalo90@gmail.com>
Co-authored-by: Eneko Uruñuela <e.urunuela@icloud.com>

* Update bokeh requirement from <=3.5.0,>=1.0.0 to >=1.0.0,<=3.5.1 (ME-ICA#1116)

Updates the requirements on [bokeh](https://github.com/bokeh/bokeh) to permit the latest version.
- [Changelog](https://github.com/bokeh/bokeh/blob/3.5.1/docs/CHANGELOG)
- [Commits](bokeh/bokeh@1.0.0...3.5.1)

---
updated-dependencies:
- dependency-name: bokeh
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Update list of multi-echo datasets (ME-ICA#1115)

* Generate metrics from external regressors using F stats (ME-ICA#1064)

* Get required metrics from decision tree.

* Continue changes.

* More updates.

* Store necessary_metrics as a list.

* Update selection_nodes.py

* Update selection_utils.py

* Update across the package.

* Keep updating.

* Update tedana.py

* Add extra metrics to list.

* Update ica_reclassify.py

* Draft metric-based regressor correlations.

* Fix typo.

* Work on trees.

* Expand regular expressions in trees.

* Fix up the expansion.

* Really fix it though.

* Fix style issue.

* Added external regress integration test

* Got intregration test with external regressors working

* Added F tests and options

* added corr_no_detrend.json

* updated names and reporting

* Run black.

* Address style issues.

* Try fixing test bugs.

* Update test_component_selector.py

* Update component_selector.py

* Use component table directly in selectcomps2use.

* Fix.

* Include generated metrics in necessary metrics.

* Update component_selector.py

* responding to feedback from tsalo

* Update component_selector.py

* Update test_component_selector.py

* fixed some testing failures

* fixed test_check_null_succeeds

* fixed ica_reclassify bug and selector_properties test

* ComponentSelector initialized before loading data

* fixed docstrings

* updated building decision tree docs

* using external regressors and most tests passing

* removed corr added tasks

* fit_model moved to stats

* removed and cleaned up external_regressors_config option

* Added task regressors and some tests. Now alll in decision tree

* cleaning up decision tree json files

* removed mot12_csf.json changed task to signal

* fixed tests with task_keep signal

* Update tedana/metrics/external.py

Co-authored-by: Taylor Salo <salot@pennmedicine.upenn.edu>

* Update tedana/metrics/_utils.py

Co-authored-by: Taylor Salo <salot@pennmedicine.upenn.edu>

* Update tedana/metrics/collect.py

Co-authored-by: Taylor Salo <salot@pennmedicine.upenn.edu>

* Update tedana/metrics/external.py

Co-authored-by: Taylor Salo <salot@pennmedicine.upenn.edu>

* Update tedana/metrics/external.py

Co-authored-by: Taylor Salo <salot@pennmedicine.upenn.edu>

* Responding to review comments

* reworded docstring

* Added type hints to external.py

* fixed external.py type hints

* type hints to _utils collect and component_selector

* type hints and doc improvements in selection_utils

* no expand_node recursion

* removed expand_nodes expand_node expand_dict

* docstring lines break on punctuation

* updating external tests and docs

* moved test data downloading to tests.utils.py and started test for fit_regressors

* fixed bug where task regressors retained in partial models

* matched testing external regressors to included mixing and fixed bugs

* Made single function for detrending regressors

* added tests for external fit_regressors and fix_mixing_to_regressors

* Full tests in test_external_metrics.py

* adding tests

* fixed extern regress validation warnings and added tests

* sorting set values for test outputs

* added to test_metrics

* Added docs to building_decision_trees.rst

* Added motion task decision tree flow chart

* made recommended change to external_regressor_config

* Finished documentation and renamed demo decision trees

* added link to example external regressors tsv file

* Apply suggestions from code review

Fixed nuissance typos

Co-authored-by: Taylor Salo <salot@pennmedicine.upenn.edu>

* Minor documentation edits

---------

Co-authored-by: Taylor Salo <tsalo006@fiu.edu>
Co-authored-by: Taylor Salo <salot@pennmedicine.upenn.edu>
Co-authored-by: Neha Reddy <nreddy@northwestern.edu>

* Link to the open-multi-echo-data website (ME-ICA#1117)

* Update multi-echo.rst

* Update multi-echo.rst

* Refactor `metrics.dependence` module (ME-ICA#1088)

* Add type hints to metric functions.

* Use keyword arguments.

* Update tests.

* Update dependence.py

* Update collect.py

* Fix other stuff.

* documentation and resource updates (ME-ICA#1114)

* documentation and resource updates

* Fixed citation numbering and updated posters

---------

Co-authored-by: Neha Reddy <nreddy@northwestern.edu>

* Adding already requested changes

* fixed failing tests

* updated documentation in faq.rst

* more documentation changes

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Taylor Salo <salot@pennmedicine.upenn.edu>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Matteo Visconti di Oleggio Castello <mvdoc@users.noreply.github.com>
Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>
Co-authored-by: Eneko Uruñuela <e.urunuela@icloud.com>
Co-authored-by: Taylor Salo <tsalo90@gmail.com>
Co-authored-by: Taylor Salo <tsalo006@fiu.edu>
Co-authored-by: Neha Reddy <nreddy@northwestern.edu>
handwerkerd added a commit that referenced this pull request Sep 23, 2024
…sults (#1013)

* Add robustica method

* Incorporation of major comments regarding robustica addition

Manual modification of commit f2cdb4e to remove unwanted file additions.

* Add robustica 0.1.3 to dependency list

Cherry-pick of 41354cb.

* Multiple fixes to RobustICA addition from code review

From: BahmanTahayori#2.

Co-authored-by: Robert E. Smith <robert.smith@florey.edu.au>

* Specify magic number fixed seed of 42 as a constant

Cherry-pick of da1b128 (with modification).

* Updated

* Robustica Updates

* Incorporating the third round of Robert E. Smith's comments

* Enhance the "ica_method" description suggested by D. Handwerker

Co-authored-by: Dan Handwerker <7406227+handwerkerd@users.noreply.github.com>

* Enhancing the "n_robust_runs" description suggested by D. Handwerkerd

Co-authored-by: Dan Handwerker <7406227+handwerkerd@users.noreply.github.com>

* RobustICA: Restructure code loop over robust methods (#4)

* RobustICA: Restructure code loop over robust methods

* Addressing the issue with try/except

---------

Co-authored-by: Bahman <tahayori@gmail.com>

* Applied suggested changes

In this commit, some of the comments from Daniel Handwerker and Robert
Smith were incorporated.

* Incorporating more comments

* Fixing the problem of argument parser for n_robust_runs.

* Removing unnecessary tests from the test_integration. There are 3
  tests for echo as before, but the ica_method is robustica for five and
three echos and fatsica for the four echo test.

* Adding already requested changes

* fixed failing tests

* updated documentation in faq.rst

* more documentation changes

* Update docs/faq.rst

Co-authored-by: Robert Smith <robert.smith@florey.edu.au>

* Update docs/faq.rst

Co-authored-by: Robert Smith <robert.smith@florey.edu.au>

* Aligning robustICA with current Main + (#5)

* Limit current adaptive mask method to brain mask (#1060)

* Limit adaptive mask calculation to brain mask.

Limit adaptive mask calculation to brain mask.

Expand on logic of first adaptive mask method.

Update tedana/utils.py

Improve docstring.

Update test.

Add decreasing-signal-based adaptive mask.

Keep removing.

Co-Authored-By: Dan Handwerker <7406227+handwerkerd@users.noreply.github.com>

* Use `compute_epi_mask` in t2smap workflow.

* Try fixing the tests.

* Fix make_adaptive_mask.

* Update test_utils.py

* Update test_utils.py

* Improve docstring.

* Update utils.py

* Update test_utils.py

* Revert "Update test_utils.py"

This reverts commit 259b002.

* Don't take absolute value of echo means.

* Log echo-wise thresholds in adaptive mask.

* Add comment about non-zero voxels.

* Update utils.py

* Update test_utils.py

* Update test_utils.py

* Update test_utils.py

* Log the thresholds again.

* Address review.

* Fix test.

---------

Co-authored-by: Dan Handwerker <7406227+handwerkerd@users.noreply.github.com>

* Update nilearn requirement from <=0.10.3,>=0.7 to >=0.7,<=0.10.4 (#1077)

* Add adaptive mask plot to report (#1073)

* Update scikit-learn requirement (#1075)

Updates the requirements on [scikit-learn](https://github.com/scikit-learn/scikit-learn) to permit the latest version.
- [Release notes](https://github.com/scikit-learn/scikit-learn/releases)
- [Commits](scikit-learn/scikit-learn@0.21.0...1.4.2)

---
updated-dependencies:
- dependency-name: scikit-learn
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Update pandas requirement from <=2.2.1,>=2.0 to >=2.0,<=2.2.2 (#1076)

Updates the requirements on [pandas](https://github.com/pandas-dev/pandas) to permit the latest version.
- [Release notes](https://github.com/pandas-dev/pandas/releases)
- [Commits](pandas-dev/pandas@v2.0.0...v2.2.2)

---
updated-dependencies:
- dependency-name: pandas
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Update bokeh requirement from <=3.4.0,>=1.0.0 to >=1.0.0,<=3.4.1 (#1078)

Updates the requirements on [bokeh](https://github.com/bokeh/bokeh) to permit the latest version.
- [Changelog](https://github.com/bokeh/bokeh/blob/branch-3.5/docs/CHANGELOG)
- [Commits](bokeh/bokeh@1.0.0...3.4.1)

---
updated-dependencies:
- dependency-name: bokeh
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Load user-defined mask as expected by plot_adaptive_mask (#1079)

* DOC desc-optcomDenoised -> desc-denoised (#1080)

* docs: add mvdoc as a contributor for code, bug, and doc (#1082)

* docs: update README.md

* docs: update .all-contributorsrc

---------

Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>

* Identify the last good echo in adaptive mask instead of sum of good echoes (#1061)

* Limit adaptive mask calculation to brain mask.

Limit adaptive mask calculation to brain mask.

Expand on logic of first adaptive mask method.

Update tedana/utils.py

Improve docstring.

Update test.

Add decreasing-signal-based adaptive mask.

Keep removing.

Co-Authored-By: Dan Handwerker <7406227+handwerkerd@users.noreply.github.com>

* Use `compute_epi_mask` in t2smap workflow.

* Try fixing the tests.

* Fix make_adaptive_mask.

* Update test_utils.py

* Update test_utils.py

* Improve docstring.

* Identify the last good echo instead of sum.

Improve docstring.

Update test_utils.py

Update test_utils.py

Fix make_adaptive_mask.

Try fixing the tests.

Use `compute_epi_mask` in t2smap workflow.

Limit adaptive mask calculation to brain mask.

Limit adaptive mask calculation to brain mask.

Expand on logic of first adaptive mask method.

Update tedana/utils.py

Improve docstring.

Update test.

Add decreasing-signal-based adaptive mask.

Keep removing.

Co-Authored-By: Dan Handwerker <7406227+handwerkerd@users.noreply.github.com>

* Fix.

* Update utils.py

* Update utils.py

* Try fixing.

* Update utils.py

* Update utils.py

* add checks

* Just loop over voxels.

* Update utils.py

* Update utils.py

* Update test_utils.py

* Revert "Update test_utils.py"

This reverts commit 259b002.

* Update test_utils.py

* Update test_utils.py

* Remove checks.

* Don't take absolute value of echo means.

* Log echo-wise thresholds in adaptive mask.

* Add comment about non-zero voxels.

* Update utils.py

* Update utils.py

* Update test_utils.py

* Update test_utils.py

* Update test_utils.py

* Log the thresholds again.

* Update test_utils.py

* Update test_utils.py

* Update test_utils.py

* Add simulated data to adaptive mask test.

* Clean up the tests.

* Add value that tests the base mask.

* Remove print in test.

* Update tedana/utils.py

Co-authored-by: Dan Handwerker <7406227+handwerkerd@users.noreply.github.com>

* Update tedana/utils.py

Co-authored-by: Dan Handwerker <7406227+handwerkerd@users.noreply.github.com>

---------

Co-authored-by: Dan Handwerker <7406227+handwerkerd@users.noreply.github.com>

* Output RMSE map and time series for decay model fit (#1044)

* Draft function to calculate decay model fit.

* Calculate root mean squared error instead.

* Incorporate metrics.

* Output RMSE results.

* Output results in tedana.

* Hopefully fix things.

* Update decay.py

* Try improving performance.

* Update decay.py

* Fix again.

* Use tqdm.

* Update decay.py

* Update decay.py

* Update decay.py

* Update expected outputs.

* Add figures.

* Update outputs.

* Include global signal in confounds file.

* Update fiu_four_echo_outputs.txt

* Rename function.

* Rename function.

* Update tedana.py

* Update tedana/decay.py

Co-authored-by: Dan Handwerker <7406227+handwerkerd@users.noreply.github.com>

* Update decay.py

* Update decay.py

* Whoops.

* Apply suggestions from code review

Co-authored-by: Dan Handwerker <7406227+handwerkerd@users.noreply.github.com>

* Fix things maybe.

* Fix things.

* Update decay.py

* Remove any files that are built through appending.

* Update outputs.

* Add section on plots to docs.

* Fix the description.

* Update docs/outputs.rst

Co-authored-by: Dan Handwerker <7406227+handwerkerd@users.noreply.github.com>

* Update docs/outputs.rst

* Fix docstring.

---------

Co-authored-by: Dan Handwerker <7406227+handwerkerd@users.noreply.github.com>

* minimum nilearn 0.10.3 (#1094)

* Use nearest-neighbors interpolation in `plot_component` (#1098)

* Use nearest-neighbors interpolation in plot_stat_map.

* Only use NN interp for component maps.

* Update scipy requirement from <=1.13.0,>=1.2.0 to >=1.2.0,<=1.13.1 (#1100)

Updates the requirements on [scipy](https://github.com/scipy/scipy) to permit the latest version.
- [Release notes](https://github.com/scipy/scipy/releases)
- [Commits](scipy/scipy@v1.2.0...v1.13.1)

---
updated-dependencies:
- dependency-name: scipy
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Update scikit-learn requirement from <=1.4.2,>=0.21 to >=0.21,<=1.5.0 (#1101)

Updates the requirements on [scikit-learn](https://github.com/scikit-learn/scikit-learn) to permit the latest version.
- [Release notes](https://github.com/scikit-learn/scikit-learn/releases)
- [Commits](scikit-learn/scikit-learn@0.21.0...1.5.0)

---
updated-dependencies:
- dependency-name: scikit-learn
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Update numpy requirement from <=1.26.4,>=1.16 to >=1.16,<=2.0.0 (#1104)

* Update numpy requirement from <=1.26.4,>=1.16 to >=1.16,<=2.0.0

Updates the requirements on [numpy](https://github.com/numpy/numpy) to permit the latest version.
- [Release notes](https://github.com/numpy/numpy/releases)
- [Changelog](https://github.com/numpy/numpy/blob/main/doc/RELEASE_WALKTHROUGH.rst)
- [Commits](numpy/numpy@v1.16.0...v2.0.0)

---
updated-dependencies:
- dependency-name: numpy
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

* Use np.nan instead of np.NaN

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Taylor Salo <salot@pennmedicine.upenn.edu>

* Filter out non-diagonal affine warning (#1103)

* Filter out non-diagonal affine warning.

* Fix warning capture.

* Update tedana/reporting/static_figures.py

Co-authored-by: Dan Handwerker <7406227+handwerkerd@users.noreply.github.com>

* Update static_figures.py

---------

Co-authored-by: Dan Handwerker <7406227+handwerkerd@users.noreply.github.com>

* Update bokeh requirement from <=3.4.1,>=1.0.0 to <=3.5.0,>=3.5.0 (#1109)

* Update bokeh requirement from <=3.4.1,>=1.0.0 to <=3.5.0,>=3.5.0

Updates the requirements on [bokeh](https://github.com/bokeh/bokeh) to permit the latest version.
- [Changelog](https://github.com/bokeh/bokeh/blob/branch-3.6/docs/CHANGELOG)
- [Commits](bokeh/bokeh@1.0.0...3.5.0)

---
updated-dependencies:
- dependency-name: bokeh
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

* Update pyproject.toml

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Taylor Salo <salot@pennmedicine.upenn.edu>

* Update scikit-learn requirement from <=1.5.0,>=0.21 to <=1.5.1,>=1.5.1 (#1108)

* Update scikit-learn requirement from <=1.5.0,>=0.21 to <=1.5.1,>=1.5.1

Updates the requirements on [scikit-learn](https://github.com/scikit-learn/scikit-learn) to permit the latest version.
- [Release notes](https://github.com/scikit-learn/scikit-learn/releases)
- [Commits](scikit-learn/scikit-learn@0.21.0...1.5.1)

---
updated-dependencies:
- dependency-name: scikit-learn
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

* Update pyproject.toml to restore minimum version of scikit-learn

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Dan Handwerker <7406227+handwerkerd@users.noreply.github.com>

* Update scipy requirement from <=1.13.1,>=1.2.0 to <=1.14.0,>=1.14.0 (#1106)

* Update scipy requirement from <=1.13.1,>=1.2.0 to <=1.14.0,>=1.14.0

Updates the requirements on [scipy](https://github.com/scipy/scipy) to permit the latest version.
- [Release notes](https://github.com/scipy/scipy/releases)
- [Commits](scipy/scipy@v1.2.0...v1.14.0)

---
updated-dependencies:
- dependency-name: scipy
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

* Update pyproject.toml to retain minimum version of scipy

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Dan Handwerker <7406227+handwerkerd@users.noreply.github.com>
Co-authored-by: Eneko Uruñuela <e.urunuela@icloud.com>

* Update numpy requirement from <=2.0.0,>=1.16 to >=1.16,<=2.0.1 (#1112)

Updates the requirements on [numpy](https://github.com/numpy/numpy) to permit the latest version.
- [Release notes](https://github.com/numpy/numpy/releases)
- [Changelog](https://github.com/numpy/numpy/blob/main/doc/RELEASE_WALKTHROUGH.rst)
- [Commits](numpy/numpy@v1.16.0...v2.0.1)

---
updated-dependencies:
- dependency-name: numpy
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Cleaning up installation instructions (#1113)

* install instructions

* Update docs/installation.rst

Co-authored-by: Taylor Salo <tsalo90@gmail.com>

* Update docs/installation.rst

Co-authored-by: Eneko Uruñuela <e.urunuela@icloud.com>

---------

Co-authored-by: Taylor Salo <tsalo90@gmail.com>
Co-authored-by: Eneko Uruñuela <e.urunuela@icloud.com>

* Update bokeh requirement from <=3.5.0,>=1.0.0 to >=1.0.0,<=3.5.1 (#1116)

Updates the requirements on [bokeh](https://github.com/bokeh/bokeh) to permit the latest version.
- [Changelog](https://github.com/bokeh/bokeh/blob/3.5.1/docs/CHANGELOG)
- [Commits](bokeh/bokeh@1.0.0...3.5.1)

---
updated-dependencies:
- dependency-name: bokeh
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Update list of multi-echo datasets (#1115)

* Generate metrics from external regressors using F stats (#1064)

* Get required metrics from decision tree.

* Continue changes.

* More updates.

* Store necessary_metrics as a list.

* Update selection_nodes.py

* Update selection_utils.py

* Update across the package.

* Keep updating.

* Update tedana.py

* Add extra metrics to list.

* Update ica_reclassify.py

* Draft metric-based regressor correlations.

* Fix typo.

* Work on trees.

* Expand regular expressions in trees.

* Fix up the expansion.

* Really fix it though.

* Fix style issue.

* Added external regress integration test

* Got intregration test with external regressors working

* Added F tests and options

* added corr_no_detrend.json

* updated names and reporting

* Run black.

* Address style issues.

* Try fixing test bugs.

* Update test_component_selector.py

* Update component_selector.py

* Use component table directly in selectcomps2use.

* Fix.

* Include generated metrics in necessary metrics.

* Update component_selector.py

* responding to feedback from tsalo

* Update component_selector.py

* Update test_component_selector.py

* fixed some testing failures

* fixed test_check_null_succeeds

* fixed ica_reclassify bug and selector_properties test

* ComponentSelector initialized before loading data

* fixed docstrings

* updated building decision tree docs

* using external regressors and most tests passing

* removed corr added tasks

* fit_model moved to stats

* removed and cleaned up external_regressors_config option

* Added task regressors and some tests. Now alll in decision tree

* cleaning up decision tree json files

* removed mot12_csf.json changed task to signal

* fixed tests with task_keep signal

* Update tedana/metrics/external.py

Co-authored-by: Taylor Salo <salot@pennmedicine.upenn.edu>

* Update tedana/metrics/_utils.py

Co-authored-by: Taylor Salo <salot@pennmedicine.upenn.edu>

* Update tedana/metrics/collect.py

Co-authored-by: Taylor Salo <salot@pennmedicine.upenn.edu>

* Update tedana/metrics/external.py

Co-authored-by: Taylor Salo <salot@pennmedicine.upenn.edu>

* Update tedana/metrics/external.py

Co-authored-by: Taylor Salo <salot@pennmedicine.upenn.edu>

* Responding to review comments

* reworded docstring

* Added type hints to external.py

* fixed external.py type hints

* type hints to _utils collect and component_selector

* type hints and doc improvements in selection_utils

* no expand_node recursion

* removed expand_nodes expand_node expand_dict

* docstring lines break on punctuation

* updating external tests and docs

* moved test data downloading to tests.utils.py and started test for fit_regressors

* fixed bug where task regressors retained in partial models

* matched testing external regressors to included mixing and fixed bugs

* Made single function for detrending regressors

* added tests for external fit_regressors and fix_mixing_to_regressors

* Full tests in test_external_metrics.py

* adding tests

* fixed extern regress validation warnings and added tests

* sorting set values for test outputs

* added to test_metrics

* Added docs to building_decision_trees.rst

* Added motion task decision tree flow chart

* made recommended change to external_regressor_config

* Finished documentation and renamed demo decision trees

* added link to example external regressors tsv file

* Apply suggestions from code review

Fixed nuissance typos

Co-authored-by: Taylor Salo <salot@pennmedicine.upenn.edu>

* Minor documentation edits

---------

Co-authored-by: Taylor Salo <tsalo006@fiu.edu>
Co-authored-by: Taylor Salo <salot@pennmedicine.upenn.edu>
Co-authored-by: Neha Reddy <nreddy@northwestern.edu>

* Link to the open-multi-echo-data website (#1117)

* Update multi-echo.rst

* Update multi-echo.rst

* Refactor `metrics.dependence` module (#1088)

* Add type hints to metric functions.

* Use keyword arguments.

* Update tests.

* Update dependence.py

* Update collect.py

* Fix other stuff.

* documentation and resource updates (#1114)

* documentation and resource updates

* Fixed citation numbering and updated posters

---------

Co-authored-by: Neha Reddy <nreddy@northwestern.edu>

* Adding already requested changes

* fixed failing tests

* updated documentation in faq.rst

* more documentation changes

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Taylor Salo <salot@pennmedicine.upenn.edu>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Matteo Visconti di Oleggio Castello <mvdoc@users.noreply.github.com>
Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>
Co-authored-by: Eneko Uruñuela <e.urunuela@icloud.com>
Co-authored-by: Taylor Salo <tsalo90@gmail.com>
Co-authored-by: Taylor Salo <tsalo006@fiu.edu>
Co-authored-by: Neha Reddy <nreddy@northwestern.edu>

* align with main

* fixed ica.py docstring error

* added scikit-learn-extra to pyproject and changed ref name

* increment circleci version keys

* Removing the scikit-learn-extra dependency

* Updating pyproject.toml file

* Minor changes to make the help more readable

* Minor changes

* upgrading to robustica 0.1.4

* Update docs

Co-authored-by: Dan Handwerker <7406227+handwerkerd@users.noreply.github.com>

* updating utils.py, toml file and the docs

* minor change to utils.py

* Incorporating Eneko's comments

Co-authored-by: Eneko Uruñuela <e.urunuela@icloud.com>

* Added a warning when the clustering method is changed

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Robert E. Smith <robert.smith@florey.edu.au>
Co-authored-by: Dan Handwerker <7406227+handwerkerd@users.noreply.github.com>
Co-authored-by: handwerkerd <dan.handwerker@gmail.com>
Co-authored-by: Taylor Salo <salot@pennmedicine.upenn.edu>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Matteo Visconti di Oleggio Castello <mvdoc@users.noreply.github.com>
Co-authored-by: allcontributors[bot] <46447321+allcontributors[bot]@users.noreply.github.com>
Co-authored-by: Eneko Uruñuela <e.urunuela@icloud.com>
Co-authored-by: Taylor Salo <tsalo90@gmail.com>
Co-authored-by: Taylor Salo <tsalo006@fiu.edu>
Co-authored-by: Neha Reddy <nreddy@northwestern.edu>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement issues describing possible enhancements to the project reports issues related to boilerplate generation or visual reports
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add adaptive mask figure to the HTML report
3 participants