Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add alternative memory management strategies #395

Merged
merged 2 commits into from
May 28, 2018
Merged

Add alternative memory management strategies #395

merged 2 commits into from
May 28, 2018

Conversation

wlandau
Copy link
Member

@wlandau wlandau commented May 28, 2018

Summary

Add a pruning_strategy argument to make() and drake_config() so the user can decide how drake keeps non-import dependencies in memory when it builds a target.

If pruning_strategy is "memory", drake removes all targets from memory (i.e. config$envir) except the direct dependencies of the target is about to build. This is suitable for data so large that the optimal strategy is to minimize memory consumption. If pruning_strategy is "speed", drake loads all the dependencies and keeps in memory everything that will eventually be a dependency of a downstream target. This strategy consumes more memory, but does more to minimize the number of times data is read from storage/disk.

Related GitHub issues

Checklist

  • I have read drake's code of conduct, and I agree to follow its rules.
  • I have read the guidelines for contributing.
  • I have listed any substantial changes in the development news.
  • I have added testthat unit tests to tests/testthat to confirm that any new features or functionality work correctly.
  • I have tested this pull request locally with devtools::check()
  • This pull request is ready for review.
  • I think this pull request is ready to merge.

@wlandau wlandau changed the title Fix # Add alternative memory management strategies May 28, 2018
@wlandau wlandau merged commit aefa7a5 into master May 28, 2018
@wlandau wlandau deleted the i385 branch May 28, 2018 12:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants