Add `--force` flag to cargo install #2405

gkoz · 2016-02-21T21:18:31Z

Close #2082.

Adding --force (-f) instructs cargo to overwrite existing binaries (updating the metadata accordingly).
This allows updating crates via cargo install -f <crate>.

Installation happens in two stages now: binaries are copied into a temporary subdirectory of the destination first, then moved into destination. This should catch some errors earlier.

In case of installation error cargo will remove new binaries but won't attempt to undo successful overwrites.

rust-highfive · 2016-02-21T21:18:46Z

r? @huonw

(rust_highfive has picked a reviewer for you, use r? to override)

gkoz · 2016-02-21T21:19:01Z

r? @alexcrichton

alexcrichton · 2016-02-22T18:17:37Z

Thanks for the PR @gkoz! There's a few technical aspects I'd tweak here, but I agree that the first thing to do here is probably to decide whether we want to do this in the first place. Let's discuss more on the issue, though, as that's probably the best place to have this discussion.

(I'll write a comment over there shortly)

gkoz · 2016-02-23T11:10:16Z

src/cargo/ops/cargo_install.rs

+            if replace {
+                return Ok(Some(p.clone()))
+            }
+            msg.push_str(&format!(" as part of `{}` (use --replace to override)", p));
        }
        Err(human(msg))


I've realized that it's safer to always list all conflicts when run without --replace thus the error should probably be generated somewhere up the stack.

gkoz · 2016-02-25T19:16:18Z

There doesn't seem to be any opposition to this change.

alexcrichton · 2016-03-11T17:47:50Z

Ok, @wycats and I had a chance to talk about this recently, and this seems like it's desirable as it's pretty orthogonal from the --update flag. Put another way, cargo install --replace is distinct from cargo install --update --replace, so it seems fine to have both!

One thought we had was perhaps this could be renamed -f or --force? That follows the convention of many other tools for saying "just do this". Also, from the discussion on #2082 it sounds like the opinions about --update may have changed in light of our current situation, so if you'd like to implement that as well here feel free!

gkoz · 2016-03-11T18:19:08Z

Yes --force would work. I had avoided it to sidestep a bigger question of whether cargo should have such a flag and consistency considerations (if install gets it other commands might want to as well).

I'll keep --update in mind but wouldn't do it straight away.

You had mentioned some implementation issues. I'm guessing you don't like the replace (or force) flag being passed as a bool from bin/install.rs?

alexcrichton · 2016-03-11T21:27:16Z

Ah I believe what I had in mind was the "transactional" nature of the install. Right now the uninstall happens first, but it probably wants to get delayed until right before the actual new binaries are moved into place (either that or nothing is really uninstalled, just metadata is updated).

bors · 2016-03-12T20:34:47Z

☔ The latest upstream changes (presumably #2421) made this pull request unmergeable. Please resolve the merge conflicts.

alexcrichton · 2016-04-06T16:10:18Z

ping @gkoz, any update on this?

gkoz · 2016-04-07T15:40:23Z

Oh wow, it's been a month? I'll revisit this in a few days.

alexcrichton · 2016-04-08T00:11:32Z

Sounds good to me! Feel free to ping me if you need help pushing over the finish line.

gkoz · 2016-04-12T15:05:04Z

Ah I believe what I had in mind was the "transactional" nature of the install. Right now the uninstall happens first, but it probably wants to get delayed until right before the actual new binaries are moved into place (either that or nothing is really uninstalled, just metadata is updated).

To avoid any misunderstanding... The current patch calls uninstall after building but before copying the binaries. If that is too risky (after uninstalling we might find there's not enough space for the new binaries?) I could perhaps create a temporary subdirectory (TempDir::new_in(&dst)), copy new binaries into it and then overwrite the old ones.

alexcrichton · 2016-04-12T18:24:33Z

Basically what I'm getting at is two properties:

If a normal error happens for whatever reason, we shouldn't tamper with the install root at all
If ctrl-c happens, the "window of where things are corrupted" should be as small as possible

So given that, the error that I'm worried about here is that we successfully remove something but then hit an I/O error copying in a new file, In that case we won't restore the just-removed binaries. Could this be updated to handle that?

gkoz · 2016-04-15T11:46:17Z

I've pushed a version, that copies all binaries into a temp subdirectory of the destination and only after that succeeds tries to replace the existing binaries. If overwriting is partially successful, it doesn't try to undo anything (should it? that could further increase complexity) but still updates the metadata.

Another difference from former behavior is (when --force is absent) listing all of the binaries to be overwritten, not just the first one. However the error message still doesn't make it very clear that without --bin the old packages will be removed completely.

Error handling is not ideal here:

I'm not sure how to make several lines of "binary ... already exists ..." and the "Add --force to overwrite" hint look nicer,
there doesn't seem to be a way to report two unrelated but equally important errors.

Oh and I haven't tested on Windows yet to make sure there's no issue with EXE_SUFFIX.

alexcrichton · 2016-04-15T23:36:07Z

src/cargo/ops/cargo_install.rs

+                }
+            }
+        }
+        // If `--bin` or `--example` weren't specified, uninstall obsolete


This seems a little surprising to me, this is basically saying if package A installed foo and bar, then B installed foo, you'd remove A's bar? I'd probably expect that A's bar stuck around and you could uninstall it with cargo uninstall A

All this time I'd been assuming it's a good idea for cargo install -f foo to completely remove the remnants of the old package by default because that's probably what the user wants. :) If that's not the case I should probably add a hint "run 'cargo uninstall foo:{previous version}' to remove remaining binaries."

Hm yeah I guess it depends on how you look at it. On one hand you could have installed a package with a ton of binaries, some of which you never used, and then you overwrite one of the never used ones. On the other hand this could also hamstring a package to be useless if it's the one binary that counts.

To me though -f just means "do this anyway" which doesn't necessarily imply "do the extra work of removing everything else", and I like the idea of adding a note that other binaries from the package are lying around and may wish to be removed.

alexcrichton · 2016-04-15T23:37:45Z

Thanks @gkoz! Can you also write some tests for some of the corner cases here? Just to make sure the metadata is managed properly and everything works as expected.

gkoz · 2016-04-26T13:34:00Z

Still working on adding tests. Wondering how to imitate failure modes.

Want to get this right first, and then add the hint about outdated binaries, possibly in another PR.

alexcrichton · 2016-04-27T06:37:47Z

It's ok to not necessarily exhaustively test them, it's fine to just smoke test a few simple ones like something already exists, a build failure when overwriting, etc.

Close #2082 Adding `--force` (`-f`) instructs cargo to overwrite existing binaries (updating the metadata accordingly). This allows updating crates via `cargo install -f <crate>`. Installation happens in two stages now: binaries are copied into a temporary subdirectory of the destination first, then moved into destination. This should catch some errors earlier. In case of installation error cargo will remove new binaries but won't attempt to undo successful overwrites.

Try moving the binaries (and fall back to copying) if the build directory is a temporary one (source isn't a local path).

gkoz · 2016-05-01T00:07:25Z

Added more tests and an optional optimization in a separate commit.

alexcrichton · 2016-05-02T16:54:55Z

@bors: r+ 9f0fa24

Thanks!

bors · 2016-05-02T16:54:57Z

⌛ Testing commit 9f0fa24 with merge 401209f...

Add `--force` flag to cargo install Close #2082. Adding `--force` (`-f`) instructs cargo to overwrite existing binaries (updating the metadata accordingly). This allows updating crates via `cargo install -f <crate>`. Installation happens in two stages now: binaries are copied into a temporary subdirectory of the destination first, then moved into destination. This should catch some errors earlier. In case of installation error cargo will remove new binaries but won't attempt to undo successful overwrites.

bors · 2016-05-02T17:21:13Z

💔 Test failed - cargo-win-msvc-32

alexcrichton · 2016-05-02T17:25:52Z

@bors: retry

On Mon, May 2, 2016 at 10:21 AM, bors notifications@github.com wrote:

[image: 💔] Test failed - cargo-win-msvc-32
http://buildbot.rust-lang.org/builders/cargo-win-msvc-32/builds/391

—
You are receiving this because you were assigned.
Reply to this email directly or view it on GitHub
#2405 (comment)

bors · 2016-05-02T17:26:15Z

⚡ Previous build results for cargo-linux-32, cargo-linux-64, cargo-mac-32, cargo-mac-64, cargo-win-gnu-64, cargo-win-msvc-64 are reusable. Rebuilding only cargo-cross-linux, cargo-win-gnu-32, cargo-win-msvc-32...

bors · 2016-05-02T17:40:22Z

☀️ Test successful - cargo-cross-linux, cargo-linux-32, cargo-linux-64, cargo-mac-32, cargo-mac-64, cargo-win-gnu-32, cargo-win-gnu-64, cargo-win-msvc-32, cargo-win-msvc-64

rust-highfive assigned huonw Feb 21, 2016

rust-highfive assigned alexcrichton and unassigned huonw Feb 21, 2016

alexcrichton mentioned this pull request Feb 22, 2016

cargo install --reinstall? #2082

Closed

gkoz reviewed Feb 23, 2016
View reviewed changes

gkoz changed the title ~~Add --replace flag to cargo install~~ Add --force flag to cargo install Apr 15, 2016

alexcrichton reviewed Apr 15, 2016
View reviewed changes

alexcrichton mentioned this pull request Apr 21, 2016

option for removing existing binary #2600

Closed

gkoz added 2 commits May 1, 2016 02:40

cargo-install: move binaries from the build dir if possible

9f0fa24

Try moving the binaries (and fall back to copying) if the build directory is a temporary one (source isn't a local path).

bors merged commit 9f0fa24 into rust-lang:master May 2, 2016

iddm mentioned this pull request Jan 26, 2018

Add --reinstall argument to the install option #4981

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add `--force` flag to cargo install #2405

Add `--force` flag to cargo install #2405

gkoz commented Feb 21, 2016 •

edited

Loading

rust-highfive commented Feb 21, 2016

gkoz commented Feb 21, 2016

alexcrichton commented Feb 22, 2016

gkoz Feb 23, 2016

gkoz commented Feb 25, 2016

alexcrichton commented Mar 11, 2016

gkoz commented Mar 11, 2016

alexcrichton commented Mar 11, 2016

bors commented Mar 12, 2016

alexcrichton commented Apr 6, 2016

gkoz commented Apr 7, 2016

alexcrichton commented Apr 8, 2016

gkoz commented Apr 12, 2016

alexcrichton commented Apr 12, 2016

gkoz commented Apr 15, 2016

alexcrichton Apr 15, 2016

gkoz Apr 16, 2016

alexcrichton Apr 16, 2016

alexcrichton commented Apr 15, 2016

gkoz commented Apr 26, 2016 •

edited

Loading

alexcrichton commented Apr 27, 2016

gkoz commented May 1, 2016

alexcrichton commented May 2, 2016

bors commented May 2, 2016

bors commented May 2, 2016

alexcrichton commented May 2, 2016

bors commented May 2, 2016

bors commented May 2, 2016

Add --force flag to cargo install #2405

Add --force flag to cargo install #2405

Conversation

gkoz commented Feb 21, 2016 • edited Loading

rust-highfive commented Feb 21, 2016

gkoz commented Feb 21, 2016

alexcrichton commented Feb 22, 2016

gkoz Feb 23, 2016

Choose a reason for hiding this comment

gkoz commented Feb 25, 2016

alexcrichton commented Mar 11, 2016

gkoz commented Mar 11, 2016

alexcrichton commented Mar 11, 2016

bors commented Mar 12, 2016

alexcrichton commented Apr 6, 2016

gkoz commented Apr 7, 2016

alexcrichton commented Apr 8, 2016

gkoz commented Apr 12, 2016

alexcrichton commented Apr 12, 2016

gkoz commented Apr 15, 2016

alexcrichton Apr 15, 2016

Choose a reason for hiding this comment

gkoz Apr 16, 2016

Choose a reason for hiding this comment

alexcrichton Apr 16, 2016

Choose a reason for hiding this comment

alexcrichton commented Apr 15, 2016

gkoz commented Apr 26, 2016 • edited Loading

alexcrichton commented Apr 27, 2016

gkoz commented May 1, 2016

alexcrichton commented May 2, 2016

bors commented May 2, 2016

bors commented May 2, 2016

alexcrichton commented May 2, 2016

bors commented May 2, 2016

bors commented May 2, 2016

Add `--force` flag to cargo install #2405

Add `--force` flag to cargo install #2405

gkoz commented Feb 21, 2016 •

edited

Loading

gkoz commented Apr 26, 2016 •

edited

Loading