Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve build speed by refactoring Makefile #453

Closed
jkowalski opened this issue Dec 21, 2018 · 4 comments · Fixed by #506
Closed

Improve build speed by refactoring Makefile #453

jkowalski opened this issue Dec 21, 2018 · 4 comments · Fixed by #506
Labels
area/build-tools Development tooling. I.e. pretty much everything in the `build` directory. help wanted We would love help on these issues. Please come help us!
Milestone

Comments

@jkowalski
Copy link
Contributor

There are currently two issues with Makefile which prevent it from being parallelizable:

  1. Some dependencies are missing in the Makefile which prevent it from working with -j N where N > 0.

For example if I do:

$ make -j2 build build-controller-binary push-controller-image

It will likely fail or be incorrect (because build and push will be running in parallel, it will either fail to push or push a previous version). A fix for this is to add a dependency on build to push. There are all kinds of issues like that in the Makefile.

Ideally I should be able to say export MAKEFLAGS="-j 4" in my ~/.bashrc and my builds would be magically faster, but still correct regardless of targets that I specify.

  1. Some targets (e.g. build-agones-sdk-binary) are sequential, and pretty slow (in particular when building on macOS, which has horrible filesystem mount performance). They might benefit from being broken into smaller pieces which can run in paralllel.
@markmandel markmandel added area/build-tools Development tooling. I.e. pretty much everything in the `build` directory. help wanted We would love help on these issues. Please come help us! labels Dec 21, 2018
@jkowalski
Copy link
Contributor Author

I can work on fixing that.

Also I discovered that we're not using any go build caching, which kills interactive (re)build performance. Will send a PR which fixes that...

@markmandel
Copy link
Member

Also I discovered that we're not using any go build caching, which kills interactive (re)build performance. Will send a PR which fixes that...

YEESSSS - I've wanted this for a while, but never got around to it! 🤸‍♂️

@cyriltovena
Copy link
Collaborator

Some targets (e.g. build-agones-sdk-binary) are sequential, and pretty slow (in particular when building on macOS, which has horrible filesystem mount performance). They might benefit from being broken into smaller pieces which can run in paralllel.

YES ! building on macOS is slow.

@wilsonwang371
Copy link

It looks like building agones on Linux still has problems when building with -j. Is there any bug tracking this issue?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/build-tools Development tooling. I.e. pretty much everything in the `build` directory. help wanted We would love help on these issues. Please come help us!
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants