Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FIX: Estimate free memory with "available", not "free" #2985

Merged
merged 1 commit into from
Apr 10, 2023

Conversation

effigies
Copy link
Member

Changes proposed in this pull request

According to the psutil docs:

Return statistics about system memory usage as a named tuple including the following fields, expressed in bytes. Main metrics:

  • total: total physical memory (exclusive swap).
  • available: the memory that can be given instantly to processes without the system going into swap. This is calculated by summing different memory values depending on the platform and it is supposed to be used to monitor actual memory usage in a cross platform fashion.

Other metrics:

  • used: memory used, calculated differently depending on the platform and designed for informational purposes only. total - free does not necessarily match used.
  • free: memory not being used at all (zeroed) that is readily available; note that this doesn’t reflect the actual memory available (use available instead). total - used does not necessarily match free.

Running locally, virtual_memory().available matches what I see in top, not virtual_memory().free.

This was found while debugging an issue with unwarp_wf.resample, which selects num_threads based on the memory available.

@effigies effigies merged commit 802c43d into nipreps:master Apr 10, 2023
@effigies effigies deleted the fix/free-mem-estimate branch April 10, 2023 17:35
@effigies effigies added this to the 23.1.0 milestone Apr 20, 2023
effigies added a commit that referenced this pull request Jun 12, 2023
23.1.0 (June 12, 2023)

New feature release in the 23.1.x series.

This release substantially reworks the resampling to fsLR grayordinate space,
better accounting for partial volumes and high variance voxels. If you are
resampling using ``--project-goodvoxels``, we strongly recommend upgrading.

Fieldmap handling is improved, with better preference given to single-band
references in both PEPolar and SyN-SDC schemes. Additionally, fMRIPrep will
no longer estimate fieldmaps that are not intended to be used to correct BOLD
series, reducing unneeded processing.

This release removes ICA-AROMA from the fMRIPrep workflow. To use ICA-AROMA,
set ``MNI152NLin6Asym:res-2`` as a target output space. MELODIC and ICA-AROMA
can be run on the resulting images in a separate pipeline. For further
information on the reasoning behind this change, see
`GitHub issue #2936 <https://github.com/nipreps/fmriprep/issues/2936>`__.

This release increments the versions of ANTs and FSL bundled in the Docker
image.

With thanks to Eilidh MacNicol, Basille Pinsard and Taylor Salo for contributions
in fMRIPrep and SDCflows.

* FIX: Raise RuntimeError at build if echos have mismatched shapes (#3028)
* FIX: Inconsistent fmapless estimation when ignoring fieldmaps (#2994)
* FIX: Dilate BOLD mask by 2 voxels to prevent over-aggressive masking degrading T2* map estimation (#2986)
* FIX: Estimate free memory with "available", not "free" (#2985)
* ENH: Add ``--me-t2s-fit-method`` parameter (#3030)
* ENH: Resample BOLD to fsLR directly, dropping fsaverage intermediate (#3011)
* ENH: Allow SBref+EPI PEPolar fieldmaps to correct BOLD series (#3008)
* ENH: Remove ICA-AROMA from workflow and docs (#2966)
* RF: Filter fieldmaps based on whether they will be used to correct a BOLD series (#3025)
* MNT: Update ANTs pin in Docker image (#3016)
* MNT: Update governance docs (#2992)
* MNT: Refactor Docker build process (#2982)
* MNT: Pin conda environment more strictly (#2853)
* MNT: Require niworkflows ~1.3.6 (#2740)
* CI: Use registry for layer caching (#3012)
* CI: Upgrade docker orb (#2865)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants