Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CI] Split out pip install and apt-get install into separate build steps. #3585

Merged
merged 2 commits into from
Apr 5, 2024

Conversation

jlebar
Copy link
Collaborator

@jlebar jlebar commented Apr 5, 2024

This lets us see how long they take.

@jlebar jlebar requested a review from ptillet as a code owner April 5, 2024 18:03
@jlebar
Copy link
Collaborator Author

jlebar commented Apr 5, 2024

cc @antiagainst

@jlebar jlebar requested a review from zahimoud April 5, 2024 18:04
@jlebar jlebar enabled auto-merge (squash) April 5, 2024 18:04
@jlebar jlebar marked this pull request as draft April 5, 2024 18:04
auto-merge was automatically disabled April 5, 2024 18:04

Pull request was converted to draft

@jlebar jlebar force-pushed the cache-pip branch 3 times, most recently from 9d7dd10 to 223d44b Compare April 5, 2024 18:22
@jlebar
Copy link
Collaborator Author

jlebar commented Apr 5, 2024

The Docker (apparently) already has Torch and some other pip packages installed, but when I add this new thing, we have to install them all from scratch, which is very slow.

Caching might help, but it's not clear to me that it would actually be an improvement over the current state of affairs. Maybe instead we should update the Docker so it includes more of the packages we're installing via pip, i.e. make more of the packages like Torch.

@jlebar
Copy link
Collaborator Author

jlebar commented Apr 5, 2024

Let me use the approval here for the subset of this PR that just splits pip install and apt-get install into separate steps, so we can see how slow they are.

…d steps.

This lets us see how long they take.
@jlebar jlebar changed the title [CI] Cache pip dependencies in Triton build. [CI] Split out pip install and apt-get install into separate build steps. Apr 5, 2024
@jlebar jlebar marked this pull request as ready for review April 5, 2024 20:04
@jlebar jlebar enabled auto-merge (squash) April 5, 2024 20:04
@jlebar jlebar merged commit 1af93bd into main Apr 5, 2024
5 checks passed
@jlebar jlebar deleted the cache-pip branch April 5, 2024 20:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants