Merge cargo-add into cargo

[epage]

This is more a scratchpad for myself to iterate on thoughts without pinging everybody in rust-lang/cargo#8062 (comment)

[epage]

Draft of the proposal to merge

This is a proposal to merge cargo-add into cargo

Motivation

In addition to users being used to this as they migrate from other ecosystems (see prior art), this provides guard rails to help the user avoid errors and even guide them to best practices, including

• Version constraints over *
• Using version and path for dependencies but path only for dev-dependencies (see Handle package cycles crate-ci/cargo-release#288 which led to cargo add should "Do the Right Thing" for workspaces #480)
• --help is able to provide a targeted subset of the Manifest format documentation

Drawbacks

This is another area of consideration for new RFCs like rust-lang/rfcs#3143 or rust-lang/rfcs#2906

This is a high UX feature that will draw a lot of attention (ie Issue influx), e.g.

• "cargo add" should use dotted keys if already used in Cargo.toml #521
• cargo add: include only MAJOR.MINOR #126
• cargo add: use predominant string style #217

Behavior

Help output

$ cargo-add 0.8.0       
cargo-add 0.8.0
Add dependency to a Cargo.toml manifest file

USAGE:
    cargo add [OPTIONS] <CRATE>...

ARGS:
    <CRATE>...    Crates to be added

OPTIONS:
        --allow-prerelease        Include prerelease versions when fetching from crates.io (e.g. '0.6.0-alpha')
    -B, --build                   Add crate as build dependency
        --branch <BRANCH>         Specify a git branch to download the crate from
    -D, --dev                     Add crate as development dependency
        --features <FEATURES>     Space-separated list of features to add. For an alternative approach to enabling features, consider installing the `cargo-feature` utility
        --git <URI>               Specify a git repository to download the crate from
    -h, --help                    Print help information
        --manifest-path <PATH>    Path to the manifest to add a dependency to
        --no-default-features     Set `default-features = false` for the added dependency
        --offline                 Run without accessing the network
        --optional                Add as an optional dependency (for use in features)
    -p, --package <PKGID>         Package id of the crate to add this dependency to
        --path <PATH>             Specify the path the crate should be loaded from
        --quiet                   Do not print any output in case of success
    -r, --rename <RENAME>         Rename a dependency in Cargo.toml, https://doc.rust-lang.org/cargo/reference/specifying-dependencies.html#renaming-dependencies-in-cargotoml. Only works when specifying a single dependency
        --registry <REGISTRY>     Registry to use
        --rev <REV>               Specify a git branch to download the crate from
    -s, --sort                    Sort dependencies even if currently unsorted
        --tag <TAG>               Specify a git branch to download the crate from
        --target <TARGET>         Add as dependency to the given target platform
        --upgrade <METHOD>        Choose method of semantic version upgrade.  Must be one of "none" (exact version, `=` modifier), "patch" (`~` modifier), "minor" (`^` modifier), "all" (`>=`), or "default" (no modifier) [default:
                                  default] [possible values: none, patch, minor, all, default]
    -V, --version                 Print version information
        --vers <URI>              Specify the version to grab from the registry(crates.io). You can also specify version as part of name, e.g `cargo add bitflags@0.3.2`

This command allows you to add a dependency to a Cargo.toml manifest file. If <crate> is a github or gitlab repository URL, or a local path, `cargo add` will try to automatically get the crate name and set the appropriate `--git` or
`--path` value.

Please note that Cargo treats versions like '1.2.3' as '^1.2.3' (and that '^1.2.3' is specified as '>=1.2.3 and <2.0.0'). By default, `cargo add` will use this format, as it is the one that the crates.io registry suggests. One goal of
`cargo add` is to prevent you from using wildcard dependencies (version set to '*').

Example commands

cargo add regex
cargo add regex serde
cargo add regex@1
cargo add regex@~1.0
cargo add regex --vers 1.0
cargo add regex --upgrade minor  # for some reason this is the policy name for `^`
cargo add https://github.com/rust-lang/regex
cargo add regex --git https://github.com/rust-lang/regex
cargo add ../dependency
cargo add dep --path ../dependency

For an exhaustive set of examples, see test .toml specs

Particular points

• Effectively there are two modes
  • Fill in any relevant field for one package
  • Add multiple packages, erroring for fields that are package-specific (--vers, --features, etc)
• We infer if the dependencies table is sorted and preserve that sorting when adding a new dependency
• Adding a workspace dependency
  • dev-dependencies always use path
  • all other dependencies use version + path
• Re-add to modify an existing entry
  • No --rename will remove a prior --rename
  • --optional is never removed (cargo-edit#298)
  • Only explicit --features removes a prior one
  • --vers, --path, and --git overwrite each other

Alternatives

Prior Art

• (Python) poetry add
  • git+ is needed for inferring git dependencies
  • Has dry-run, which we lack
  • Doesn't have separate --vers and --git flags
  • git branch is specified via a URL fragment, instead of a --branch
  • No --upgrade <policy> argument to control the version requirement operator
  • No --sort to force sorting of dependencies
• (Javascript) yarn add
  • name@data where data can be version, git (with fragment for branch), etc
  • -E / --exact, -T / --tilde, -C / --caret to control version requirement operator instead of --upgrade <policy> (also controlled through defaultSemverRangePrefix in config)
  • --cached for using the lock file (cargo add: Be clever and infer version from Cargo.lock #41)
  • In addition to --dev, it has --prefer-dev which will only add the dependency if it doesn't already exist in dependencies as well as dev-dependencies
  • --mode update-lockfile will ensure the lock file gets updated as well
• (Javascript) pnpm-add
• (Javascript) npm doesn't have a native solution
  • Specify version with @<version>
  • Also overloads <name>[@<version>] with path and repo
    • Supports a git host-specific protocol for shorthand, like github:user/repo
    • Uses fragment for git ref, seems to have some kind of special semver syntax for tags?
  • Only supports --save-exact / -E for operators outside of the default
• (Go) go get
  • Specify version with @<version>
  • Remove dependency with @none
• (Haskell) stack doesn't seem to have a native solution
• (Julia) pkg Add
• (Ruby) bundle add
  • Uses --version / -v instead of --vers (we use --vers because of --version / -V)
  • --source instead of path (path correlates to manifest field)
  • Uses --git / --branch like cargo-add
• (Dart) pub add
  • Uses --git-url instead of --git
  • Uses --git-ref instead of --branch, --tag, --rev

Future Possibilities

• Add a dry-run like some from Prior Art
• Update lock file accordingly
• Exploring the idea of a --local flag
• Take the MSRV into account when automatically creating version req (Take MSRV into account when auto-selecting a versio #587)
• Integrate rustsec to report advisories on new dependencies (Check rustsec on cargo add #512)
• Integrate with licensing to report license, block add, etc (e.g. Option to include license comment with crate? #386)
• Pull version from lock file (cargo add: Be clever and infer version from Cargo.lock #41)
• Exploring if any vendoring integration would be beneficial (currently errors)
• Upstream cargo-rm, cargo-upgrade, and cargo-set-version (in that order of priority)
• Update crates.io with cargo add snippets in addition to or replacing the manifest snippets

For more, see https://github.com/killercup/cargo-edit/issues?q=is%3Aissue+is%3Aopen+label%3Acargo-add

Questions

• Can we define a policy around generated TOML to keep things well scoped?
  • The goal being to limit the burden to maintainers in dealing with an influx of formatting issues and to keep the UI from being so over complicated that it would be better to edit the file directly
  • Proposal: (1) Focus on the encouraged path being easy (Pit of Success) and (2) Be opinionated in the style but not in communication
    • Inspired by rustfmt making things consistent in the little syntax details but preserving higher level organization like newlines and ordering
    • Use consistent TOML style from run to run (string style, table style), regardless of the existing manifest's style
    • Preserving sorting where its used; this falls under the "communication" side
    • However, do not re-format existing code, that belongs more to a "rustfmt for manifests"
      • This means we'd remove the --sort flag
    • If the community and the manifest's design evolve where we might want to migrate styles, we either make this configurable in .cargo/config.toml or in the "rustfmt for manifests"
• --upgrade <policy> isn't quite right
  • ^ being called minor, should be semver
  • Policy names don't map well to what Rust developers are used to (operator characters)
  • Proposal: remove
    • Redundant with --vers <req> and <name>@<req> except when "take latest" which people are unlikely to do if they also care about the version requirement operator
    • Most people use default policy anyways
• Several issues around the sourcing / constraining
  • --vers is redundant with @ and is an awkward name
  • --git and --path are redundant with accepting a path / git url directly
  • URL detection can be brittle and limits evolution
  • Proposal
    • As positional, accept zero or more <name>[:<version-req>] or <path>
      • Use : like pkgids
      • Don't accept git url's to keep detection logic simple
    • Accept --git <url>[#<type>=<ref>] where type is one of branch, tag, or rev
      • If possible, we auto detect the type from the remote and allow #<ref> (maybe only do this?)
      • Use a fragment since its client side, it won't conflict with any requirements from the server
      • Focus is on simplicity, rather than technical completeness, since the UX isn't intended to be exhaustive
      • Will get name from remote if needed
      • Alternative, replace --git <url> with git+<url> positional like Poetry
    • Remove --branch, --vers
• Is our story around pre-release complete enough to have --allow-prelease?
  • See the above issue as one example
  • Proposal: remove this flag
    • Pre-releases are uncommon enough that requiring people to explicitly set the version rather than "pick latest" seems sufficient
    • Pre-releases can break from one to the next, so people probably should be explicit about what they are intended to use
• Should we change the default operator when adding pre-release versions?
  • See Changing Cargo semver compatibility for pre-releases
  • Proposal: yes and changing this is not considered a breaking change (if cargo removes the need for this)
    • Despite the above about --allow-prerelease, this helps encourage a best practice that prevents users from being broken