Implementation of shallow libgit2 fetches behind an unstable flag

[connorworley]

This is my first contribution, so guidance is appreciated.

Fixes #1171 by moving the shallow-index and shallow-deps aspects of -Zgitoxide to a new -Zgit unstable flag.

The only change in interaction with libgit2 happens in src/cargo/sources/git/utils.rs, where we set the depth fetch option if applicable.

Shallow fetch tests for gitoxide continue to pass, but libgit2 is harder to test as it silently ignores the depth option for local fetches. I would love any ideas on how to test it in a lightweight way or whether it's OK to wait for an upstream fix.

[rustbot]

Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @weihanglo (or someone else) soon.

Please see the contribution instructions for more information. Namely, in order to ensure the minimum review times lag, PR authors and assigned reviewers should ensure that the review label (S-waiting-on-review and S-waiting-on-author) stays updated, invoking these commands when appropriate:

• @rustbot author: the review is finished, PR author should check the comments and take action accordingly
• @rustbot review: the author is ready for a review, this PR will be queued again in the reviewer's queue

[weihanglo]

Thanks for the PR. Looks pretty neat!

Shallow fetch tests for gitoxide continue to pass, but libgit2 is harder to test as it silently ignores the depth option for local fetches. I would love any ideas on how to test it in a lightweight way or whether it's OK to wait for an upstream fix.

Do you have the issue link so we can track it?

[connorworley]

Thanks for the PR. Looks pretty neat!

Shallow fetch tests for gitoxide continue to pass, but libgit2 is harder to test as it silently ignores the depth option for local fetches. I would love any ideas on how to test it in a lightweight way or whether it's OK to wait for an upstream fix.

Do you have the issue link so we can track it?

libgit2/libgit2#6634

[weihanglo]

I've tested it manually, libgit2 shallow clone seems to work for both registries and git dependencies 👍🏾.

Just one concern, maybe not really relevant. When testing -Zgitoxide I used to run git rev-list --all to check what is inside a shallow repo for multiple subsequent clones. For gitoxide it outputs the two commits we've fetched, but with libgit2 we've an error.

# Cargo.toml
[package]
name = "foo"
edition = "2021"

[dependencies]
tokio = { git = "https://github.com/tokio-rs/tokio.git", rev = "3275cfb638fe6cac6b0bbb1f60ee59eb499f6c2a" }
tokio2 = { package = "tokio", git = "https://github.com/tokio-rs/tokio.git", rev = "84c5674c601dfc36ab417ff0ec01763c2dd30a5c" }

$ cd ~/.cargo/git/db/tokio-377c595163f99a10-shallow/
$ git rev-list -all

# gitoxide
84c5674c601dfc36ab417ff0ec01763c2dd30a5c
3275cfb638fe6cac6b0bbb1f60ee59eb499f6c2a

# git2
84c5674c601dfc36ab417ff0ec01763c2dd30a5c
error: Could not read 9780bf491f0a69986f9f35b9c6a81ac951356aff
fatal: Failed to traverse parents of commit 3275cfb638fe6cac6b0bbb1f60ee59eb499f6c2a

Funniest, I've tried Git CLI git fetch --depth 1. It retrieved nothing without an error.

$ git init --bare foo && foo
$ git fetch https://github.com/tokio-rs/tokio.git --depth 1 3275cfb638fe6cac6b0bbb1f60ee59eb499f6c2a
$ git fetch https://github.com/tokio-rs/tokio.git --depth 1 84c5674c601dfc36ab417ff0ec01763c2dd30a5c
$ git rev-list --all
# nothing

Besides, git CLI, libgit2, and gitoxide all got different packfiles and indexes for the subsequent fetch on 84c5674c601dfc36ab417ff0ec01763c2dd30a5c. I don't really know if it is a bug or just an implementation detail.

That said, this looks good to me. We could setup some minimal tests with docker or something lighweight in follow-ups.

[weihanglo]

Although there is a CLI option change for an unstable feature in this PR, it should pretty easy to adapt to the new interface. Hence, I decide not to do a FCP and just merge it as-is.

@bors r+

---

If you used -Zgitoxide previously, here is how you can migrate to the new -Zgit + -Zgitoxide.

• -Zgitoxide → -Zgitoxide -Zgit
• -Zgitoxide=fetch,shallow-deps → -Zgitoxide=fetch -Zgit=shallow-deps
• -Zgitoxide=fetch,shallow-index → -Zgitoxide=fetch -Zgit=shallow-index
• No gitoxide at all but libgit2 shallow clones→ -Zgit

[bors]

📌 Commit 426b9e3 has been approved by weihanglo

It is now in the queue for this repository.

[bors]

⌛ Testing commit 426b9e3 with merge bf27f90...

[bors]

☀️ Test successful - checks-actions
Approved by: weihanglo
Pushing bf27f90 to master...