Investigate if caching actually helps & how much #24
Comments
|
Did a little more playing with this and it needs to be removed. There are cases where it causes re-builds to not happen. |
|
Did a little more playing with this and it definitely impacts performance. It basically makes everything 100% re-build on every |
dmikusa
pushed a commit
that referenced
this issue
Apr 18, 2021
- Remove buildpack's layer caching. Caching for the layer could be difficult, so removing it just leaves things up to Cargo which is the most accurate - Creates two folders under the cargo cache layer, `target/` and `home/`. The former is where build files go while the latter is where `cargo` cached downloads, like from crates.io go. - At the moment, a layer cached by the lifecycle does not preserve file mtimes. They are squashed in the name of reproducible builds. To get around this, the buildpack will preserve the mtimes of everything install, after cargo runs & then restore them the next time before cargo runs. This keeps consistent file mtimes, which is necessary for cargo to work properly. - Removes unnecessary directories from cargo's home directory, based on https://doc.rust-lang.org/cargo/guide/cargo-home.html#caching-the-cargo-home-in-ci, which saves space on the cache layer. Resolves: #24 and #25
ForestEckhardt
pushed a commit
that referenced
this issue
Apr 21, 2021
- Remove buildpack's layer caching. Caching for the layer could be difficult, so removing it just leaves things up to Cargo which is the most accurate - Creates two folders under the cargo cache layer, `target/` and `home/`. The former is where build files go while the latter is where `cargo` cached downloads, like from crates.io go. - At the moment, a layer cached by the lifecycle does not preserve file mtimes. They are squashed in the name of reproducible builds. To get around this, the buildpack will preserve the mtimes of everything install, after cargo runs & then restore them the next time before cargo runs. This keeps consistent file mtimes, which is necessary for cargo to work properly. - Removes unnecessary directories from cargo's home directory, based on https://doc.rust-lang.org/cargo/guide/cargo-home.html#caching-the-cargo-home-in-ci, which saves space on the cache layer. Resolves: #24 and #25
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Right now the buildpack will look at the
Cargo.lockfile, take its hash, and store that for future runs. The next time it runs, the buildpack will pull this hash & compare it to the hash of theCargo.lockfile that was pushed. The assumption is that if the two are different, then we need to rebuild. That seems likely to be true, but I'm not sure if that is always true and I think if it will hold true it will depend on thecargo installarguments used. Given that we'll be allowing users to set their own custom arguments in #19, this might become a problem.At the same time, it's not clear that this is really doing anything advantageous. If we are saving state between builds, which we try to do (the
target/directory is placed into its own layer marked with build + cache),cargo installshould be able to run and make its own determination if something needs to be rebuilt. This should still be fast, but should also be accurate.After #19 is merged, we need to investigate if for small/medium/large projects taking out this hash-based caching impacts the build speed & if so, how much. If the impact is minimal, as expected, then we can take out this caching.
The text was updated successfully, but these errors were encountered: