Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: normalize relative git submodule urls with ssh:// #12411

Merged
merged 1 commit into from
Jul 30, 2023

Conversation

weihanglo
Copy link
Member

What does this PR try to resolve?

Git only assumes a URL is a relative path if it starts with ./ or ../.
See https://git-scm.com/docs/git-submodule. To fetch the current repo,
we need to construct the complete submodule URL.

At this moment it comes with some limitations:

  • GitHub doesn't accept non-normalized URLs wth relative paths.
    (ssh://git@github.com/rust-lang/cargo.git/relative/.. is invalid)
  • url crate cannot parse SCP-like URLs.
    (git@github.com:rust-lang/cargo.git is not a valid WHATWG URL)

To overcome these, this patch always tries url first to normalize the
path. If it couldn't, append the relative path as the last resort and
pray the remote git service supports non-normalized URLs.

How should we test and review this PR?

I did testing by hand. Not sure how to construct a proper test around it.

cc @ehuss if you can help with this. I am a bit lost.

Additional information

See #12404 and #12295.

Fixes #12404.

I'd like to nominate this as a beta backport (again 😂).

@rustbot
Copy link
Collaborator

rustbot commented Jul 29, 2023

r? @ehuss

(rustbot has picked a reviewer for you, use r? to override)

@rustbot rustbot added A-git Area: anything dealing with git S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jul 29, 2023
@ehuss
Copy link
Contributor

ehuss commented Jul 29, 2023

Thanks for taking a look at this! Can you add a unittest to validate the behavior of the path joining code? That is, move all the code that computes child_remote_url into a free function, and add a unittest that has a few asserts for the various permutations that would be possible? Like, what if it starts with ./ or ../ or does or does not end with / or contains multiple .. or is a relative path, or if the child is a full url, etc. That should also help keep the update_submodule function a little shorter to help make it a little easier to follow.

@weihanglo
Copy link
Member Author

Updated. Thank you for the advice. I almost forgot we have unit tests 🤣.

Git only assumes a submodule URL is a relative path if it starts with `./`
or `../` [^1]. To fetch the correct repo, we need to construct an aboslute
submodule URL.

At this moment it comes with some limitations:

* GitHub doesn't accept non-normalized URLs wth relative paths.
  (`ssh://git@github.com/rust-lang/cargo.git/relative/..` is invalid)
* `url` crate cannot parse SCP-like URLs.
  (`git@github.com:rust-lang/cargo.git` is not a valid WHATWG URL)

To overcome these, this patch always tries `Url::parse` first to normalize
the path. If it couldn't, append the relative path as the last resort and
pray the remote git service supports non-normalized URLs.

See also rust-lang#12404 and rust-lang#12295.

[^1]: <https://git-scm.com/docs/git-submodule>
@ehuss
Copy link
Contributor

ehuss commented Jul 30, 2023

Thanks!

I pushed a change to fix some typos in the comments.

@bors r+

@bors
Copy link
Contributor

bors commented Jul 30, 2023

📌 Commit 8a4d044 has been approved by ehuss

It is now in the queue for this repository.

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jul 30, 2023
@bors
Copy link
Contributor

bors commented Jul 30, 2023

⌛ Testing commit 8a4d044 with merge 9fe58ac...

@bors
Copy link
Contributor

bors commented Jul 30, 2023

☀️ Test successful - checks-actions
Approved by: ehuss
Pushing 9fe58ac to master...

@bors bors merged commit 9fe58ac into rust-lang:master Jul 30, 2023
18 checks passed
@weihanglo weihanglo deleted the issue/12404 branch July 30, 2023 18:11
bors added a commit that referenced this pull request Jul 30, 2023
[beta-1.72] backport #12411

Beta backports:

- <#12411>

In order to make CI pass, the following PRs are also cherry-picked:

-
bors added a commit to rust-lang-ci/rust that referenced this pull request Jul 31, 2023
Update cargo

14 commits in 7ac9416d82cd4fc5e707c9ec3574d22dff6466e5..c91a693e7977e33a1064b63a5daf5fb689f01651
2023-07-24 14:29:38 +0000 to 2023-07-31 00:26:46 +0000
- fix: align `cargo --help` text (rust-lang/cargo#12418)
- fix: normalize relative git submodule urls with `ssh://` (rust-lang/cargo#12411)
- test: relax help text assertion (rust-lang/cargo#12416)
- test: relax assertions of panic handler message format (rust-lang/cargo#12413)
- fix(package): Recognize and normalize `cargo.toml` (rust-lang/cargo#12399)
- Clarify `lto` setting passing `-Clinker-plugin-lto` (rust-lang/cargo#12407)
- doc: add missing reference to `CARGO_PROFILE_&lt;name&gt;_STRIP` in environment variables docs (rust-lang/cargo#12408)
- Update curl-sys to pull in curl 8.2.1 (rust-lang/cargo#12406)
- docs: raise awareness of resolver used inside workspace (rust-lang/cargo#12388)
- chore: update `home` to 0.5.7 (rust-lang/cargo#12401)
- Update curl-sys to pull in curl 8.2.0 (rust-lang/cargo#12400)
- test(cli): Track --help output (rust-lang/cargo#11912)
- refactor(test): Move cargo-config into a dir (rust-lang/cargo#12398)
- refactor(tests): Name init ui tests more consistently (rust-lang/cargo#12397)

r? `@ghost`
bors added a commit to rust-lang-ci/rust that referenced this pull request Aug 1, 2023
…nglo

[beta-1.72] Update cargo

1 commits in dd6536b8ed28f73c0e82089c72ef39a03bc634be..11ffe0e500346b26e3de1ba115482b4da586dfac
2023-07-18 14:02:13 +0000 to 2023-07-30 20:44:11 +0000
- [beta-1.72] backport rust-lang/cargo#12411 (rust-lang/cargo#12417)

r? `@ghost`
@ehuss ehuss added this to the 1.73.0 milestone Aug 22, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-git Area: anything dealing with git S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Cargo does not normalize relative urls for submodules in ssh git dependencies
4 participants