-
Notifications
You must be signed in to change notification settings - Fork 2.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Redefine CARGO_TARGET_DIR
to be only an artifacts directory
#14125
Comments
In terms of incompatibilities, only a few come to my mind:
|
@poliorcetics from rust-lang/rfcs#3664 (comment)
|
Overall, I see this as a solution alternative to #6790 and had recommended we have that conversation there (or on internals).
This is likely going to be the most difficult topic to work through and we'll need to make sure we get wide input on this from #6790 users and others.
imo this is out of scope for this proposal (see #5931) and we should keep this focused so as not to get distracted. |
I see this as a re-framing of the problem, addressing #6790 and rust-lang/rfcs#3371 Instead of us defining a new
Potential names
Cargo script would default its This would need input from
|
This would need an audit of ways we publicly treat the target dir as a working dir, like exposing |
Sharing sources over NFS adds another incompatibility when the ability to move Every modern buildsystem allows to keep sources and build results separated (and users and tools do not have problems with it). I do not think that cargo should go the way back and enforce a fixed |
I'm not suggesting to force it to always be |
We talked about this in today's Cargo team meeting. Our care abouts include
While we acknowledged the potential for user confusion with The general shape of what we proposed in the meeting is... Shiny future
Legacy
Other
Path to Shiny Future
|
Something we overlooked in the above analysis is other "artifacts". In particular, I'm thinking of Ways of solving this
|
Problem
There are a couple of issues with the
CARGO_TARGET_DIR
that are seemingly in conflict with each other:Multiple locations of
target
dirs complicate excluding them from backups and full-disk search, cleanup of the temp files, moving temp files to dedicated partitions, out of slow network drives or container mounts, etc. Users don't like that thetarget
dir is huge, and multiple instances of it add up to lot of disk space. Users would prefer a central location to ease management of the temp files, and also to dedupe/reuse dependencies across many projects.People (and tools) are relying on a relative
./target
directory being present to copy or run built files out of there. Additionally, users may not want to configure a sharedCARGO_TARGET_DIR
due to risk of file name conflicts between projects.However, the dilemma between 1 and 2 exists only because Cargo uses
CARGO_TARGET_DIR
for two different roles:Proposed Solution
So to satisfy both uses, I suggest to change the thinking about what the role of
CARGO_TARGET_DIR
should be. Instead of thinking where to put the same huge all-purpose mixedCARGO_TARGET_DIR
, think how to deduplicate and slimCARGO_TARGET_DIR
, and move everything non-user-facing out of it.Instead of merging or sharding the
CARGO_TARGET_DIR
as-is with all of its current content, and adding--artifact-dir
as a separate place where final products are being copied to — makeCARGO_TARGET_DIR
to be the artifact dir (without copying).As long as the
CARGO_TARGET_DIR
dir is the place for all of the build files, of all crates including all the crates.io and local builds, with all the caches, all the temp junk, then this is going to be a problematic large directory that needs to be managed. But if the purpose of the./target
dir was changed to be only for user-facing files (files that users can name, and would access via./target
path themselves), then this directory would be relatively small, with a good reason to stay workspace-relative.What isn't an intermediate build product? (and should stay in
./target
).a
/.so
, wherelib.crate-type
calls for them. Possibly.rlib
/.rmeta
in the future if there's a stable ABI..d
files for all of the above (so that IDEs and other build systems know when to rebuild the artifacts).OUT_DIR
forbuild.rs
, see better support for generation and access of associated files #13663), then for build scripts belonging to the current workspace it would be inside./target
as well.So generally files that users build intentionally, and may want to access directly (run themselves, or package up for distribution) and files that users may need configure their IDE and debugger to find inside the project.
Crates in
[patch.crates-io]
with apath
are a gray area, an might also have their artifacts included in the./target
dir (but in some way that avoids clobbering workspaces' files).What isn't a final build product, and doesn't belong to
./target
:source = "registry+…"
).fingerprint
andincremental
dir content of all crates. These are implementation details of the compiler, and nobody should be accessing these directly via./target/…
..o
files. Users are not supposed to use them directly either (Rust has static libs for this).All of these should be built in some other shared build cache dir (one that is not inside
CARGO_TARGET_DIR
), configurable by a new option/env var.Registry dependencies would get unique paths derived from rustc version + package IDs + enabled features (so that different crates using different features don't invalidate each others' caches all the time). This would enable sharing built crates.io dependencies across all projects for the same local user, without also causing local workspaces to clobber each others'
CARGO_TARGET_DIR/profile/product
paths. Temp directories for local projects would need some hashed paths in the shared build/temp dir too.Advantages
./target
dirs (forcargo
itself, it makes./target/debug
with binaries and tests take 415MB, instead of 4.2GB). This makes cleanup of all the scatteredtarget
dirs less of a pressing problem../target
keeps relatively few files, and removes high-frequency-churning files out of it, which makes it less of a problem for real-time disk indexing (like search and backups on macOS)../target
stops being critical for build speeds, unlike I/O of the incremental cache and rewrites of thousands of.o
files. It becomes feasible to have project directory on a network drive without overridingCARGO_TARGET_DIR
(network filesystems are used by non-Linux systems where tools like Vagrant and Docker have to run full-fat VMs, and can't cheaply share the file system)../target
contain only workspace-unique files, which makes it justified for every workspace to have one.target/release/exe
etc.--artifact-dir
or.cargo/config
.Notes
No response
The text was updated successfully, but these errors were encountered: