-
Notifications
You must be signed in to change notification settings - Fork 420
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Setup py improvements #38
Conversation
Prevents accidental overwriting of changes (or unnecessary pulling of more stuff when I explicitly *not* init a subset of the modules).
setup.py
Outdated
# Init & update all submodules if not already (the user might be pinned | ||
# on some particular commit or have working tree changes, don't destroy | ||
# those) | ||
if in_git() and not os.path.exists(".git/modules"): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I like the idea of this -- I have resorted to some pretty hacky hacks to get git to not update submodules (adding a random space or newline to a file and then not committing it being my favorite).
However, I also like that currently when people pull a new build, they don't have to worry about updating submodules. Would adding a --no-update-modules
flag be sufficient (I am about to submit a PR adding a --cmake-args
flag, so I can add it to that)?
CC: @msavva
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I see your point. Yes that would work too, I would just need to remember to always add that flag (which is okay) :)
A more general question from a setup.py
noob -- do setuptools have something similar to cmake's cache? Like, you enable / set up some flags and properties and they get remembered for the next invocation; so you don't have to specify the whole command-line but can do just ./setup.py
and it will pick up the rest from somewhere, which is (like cmake's build dir) not commited to the git repository but private to your working copy? Or am I asking for the impossible?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah... unfortunately setup.py doesn't do that. We could roll our own as pickling out args
and reloading if it exists would work.....
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I also like the idea. I agree with @erikwijmans 's suggestion to make this into a --no-update-modules
flag. It's good to accommodate for the expectation that everything should work through a naive pull-and-build. I'm not aware of a way to make setup.py
cache flags etc. :(
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Made a PR into this PR with a go at caching (it contains more than just that tho).
* Argparse * --cmake-args * Caching of arguments to build_ext and argparse
Merged #40 into this. This should be good to go too! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM! 👍 Tested build and interactive viewer on MacOS.
* setup.py: option to don't run git submodule update if modules already exist. * setup.py: make it possible to *reduce* the amount of jobs. * Setup.py -- Argprase, Cmake Args, and caching (facebookresearch#40) * Argparse * --cmake-args * Caching of arguments to build_ext and argparse
Motivation and Context
While #24 said that
I'm actually one of those crazy multitaskers :) 100% CPU wouldn't be a problem, but when running on all cores, something in the build demands 8 GB of RAM which I usually don't have, so my system comes to a grinding halt. (I still want to investigate what's responsible and if it's possible to tell
ninja
to not overschedule such jobs.)While the default is still all the cores, it now allows to override the setting using the setuptools' builtin
--parallel
option. Together with redirecting it to use a pre-existing CMake build (which was set up manually for #35) dir my command-line looks like this:Apart from this, because I use some system deps, I'm not interested in
setup.py
implicitly pulling submodules that I am not interested in, so it runsgit submodule update
only the very first time.How Has This Been Tested
I'm now able to both have 163 tabs open in my browser and build Habitat :)
Types of changes
dependency upgradebuildsystem updateChecklist