Check out two branches?

[emiltin]

Hi,
I have a repo with the folder docs/ publish to Github pages, and a docs/dev/ folder with YARD docs. I'm trying to setup a github action for updating the docs when there's a push to master. The idea is to check out the gh-pages branch, then pull in changes to the docs/ folder from the master branch master with:

# assuming we're in gh-pages
git checkout master --no-overlay -- docs

Afterwards I would run yardoc to update the documenation.

But I need both master and gh-pages checked out. Is there a way to check out two branches?

Thanks!

[jakub-g]

I have the same requirement, I want to fetch two branches and do some merging between them; but the current docs only tell about extreme use cases "one branch" or "all branches and tags".
Currently I use the GH action for first branch, and then manually calling git command for the second branch.

[runspired]

Same, ability to have the head commit of master alongside the commit for the PR without needing to fetch the world would be amazing

[polarathene]

But I need both master and gh-pages checked out. Is there a way to check out two branches?

@emiltin

You only have one branch checked out in git at a time. So to have both checked out, you would need to use two separate directories as shown in the "side-by-side" and "nested" examples.

Otherwise, if you instead just need two branches fetched, then you can fetch the master branch before running your checkout?

- name: 'Checkout docs branch'
  uses: actions/checkout@v3
    with:
      ref: gh-pages
- name: 'Fetch master branch and checkout/copy over only the docs folder to gh-pages'
  run: |
    git fetch origin master --depth 1
    git checkout origin/master --no-overlay -- docs

That should accomplish what you seemed to want? The gh-pages branch remains checked out, it just has the contents of the docs/ folder from master branch now.

---

ability to have the head commit of master alongside the commit for the PR without needing to fetch the world

@runspired

The default merge commit ref with a fetch-depth of 2 will provide you with the merge commit and two parents, one from the head (PR) and base (master) refs (branches in this case).

That's enough for some operations, such as getting a diff of changed files. In this state the ref for the master branch related commit is missing until you perform a git fetch for it, which other operations may require.

I describe some examples for doing such here.

[ryan-williams]

I've struggled with this for years, often with a workflow like:

1. checkout a branch (using actions/checkout)
2. do some work, commit the results
3. push the new commit to another branch (to trigger another GitHub Action)

git fetch --unshallow origin <branch>

git fetch --unshallow origin <branch to push to> seems like the best way to do this (h/t #520 (comment)) (example).

From what I can tell, it doesn't also fetch extraneous history.

---

Bugs, and things that don't work

It seems that many commonsense fetch/pull/push workflows don't work in "shallow" clones (created by actions/checkout, by default).

Plain fetch/push

Often, a "fast-forward" push fails with a misleading/inaccurate error message like:

To https://github.com/ryan-williams/gha-test
 ! [rejected]        HEAD -> main (non-fast-forward)
error: failed to push some refs to 'https://github.com/ryan-williams/gha-test'
hint: Updates were rejected because a pushed branch tip is behind its remote
hint: counterpart. Check out this branch and integrate the remote changes
hint: (e.g. 'git pull ...') before pushing again.
hint: See the 'Note about fast-forwards' in 'git push --help' for details.
Error: Process completed with exit code 1.

Here's an example: actions/checkout cloned and checked out a branch test that was one commit ahead of main, then I ran:

git fetch origin main
git push origin HEAD:main

and got the error above. Note that (non-fast-forward) and a pushed branch tip is behind its remote are incorrect:

• I am attempting a 1-commit, fast-forward push
• I've already fetched the commit I'm trying to push (with depth 1, via the initial actions/checkout step)
• I've also fetched the branch being pushed to (which points at the parent of the initially fetched commit).

Git should have enough info to realize that this is in fact a fast-forward commit, but it does not, and the error message asserts a falsehood.

fetch --update-shallow, fetch --update-exclude=<branch>

Adding --update-shallow to the fetch doesn't help (example), and fetch --update-exclude=main origin fails with the mysterious fatal: error processing shallow info: 4 (example).

Hack: fetch-depth: 0

I usually give up and pass fetch-depth: 0 to actions/checkout (cloning the entire history of every branch of the repo), but that's very wasteful (prohibitively so, in some larger repositories).

[polarathene]

@ryan-williams make sure you check the commit hash is what you're expecting (matches the commit you think it is).

It's been a while since I commented here and similar issues, but I recall that the default wasn't necessarily what you'd have expected 😅 (brief glance over this comment of mine, and it might be a merge commit that is fetched by default by the checkout action)

EDIT: I actually point this out in my comment prior to yours:

The default merge commit ref with a fetch-depth of 2 will provide you with the merge commit and two parents, one from the head (PR) and base (master) refs (branches in this case).

That's enough for some operations, such as getting a diff of changed files. In this state the ref for the master branch related commit is missing until you perform a git fetch for it, which other operations may require.

Although if you're not triggering on a PR, where a merge commit would be relevant perhaps that's not the case for you. Could still be if it's triggered from a merge occurring.

I'd still verify that the commit history is what you'd expect, or that the refs are correct?

---

You can see my example here where I use the CI context to fetch enough commits to find a common ancestor between two branches:

• Fetches all the commits + 1 of the branch
• Checks the date of the oldest commit which should be the fork point (or older)
• Does a shallow clone of the other branch (eg: main) pulling in commits since that date which should get you that same commit.
• Then does what I wanted to do, get a diff from the merge base specified.

There is a much simpler alternative to the above, but as it mentions it's not deterministic if the action is re-run on older history. So I wouldn't rely on that.

If you know the commit of the other branch that you want to fetch, you can specify it, and it'll fetch N commits until that is found in the fetched history (I don't recall specifics, but I think it doubles the amount of commits to fetch each time by default?)

---

From what I can tell, it doesn't also fetch extraneous history.

I cloned a single branch of a repo which was a PR with 1 commit:

git clone --single-branch --shallow-exclude master --branch my-branch https://github.com/org/repo
# Updates the branch with full history, be that my-branch or it's ancestor (main)
git fetch --unshallow origin main

No tags or anything like that sure, but the commit history goes all the way back to 2016 and I'm still left with a single branch, there is none of the newer commits from main either. I probably misunderstood you though.

As for checking out a branch, doing some work and pushing, it's not exactly what you want, but here's a workflow snippet that may be relevant to you. We're checking out two branches, the main branch is to run the workflow script, while the gh-pages branch gets commits and push of these changes.

push the new commit to another branch (to trigger another GitHub Action)

I can't recall if that works? IIRC at least some triggers won't be valid when they're caused by a Github Action itself. Instead you need to use workflow_run or similar (the workflow snippet I linked belongs to a repo that does this for doc PR previews).

Here's an example

You linked to some action run logs but they're not viewable to public (assuming the repo still exists and is private). Just a heads-up that no one can see them 😅

[ryan-williams]

Thanks, I made my example repo public, and made a fresh example run, here, of the misleading non-fast-forward error message:

The test0 branch is at 95b6b3, one commit ahead of main, and the action looks like:

name: Test fetch/pull/push
on:
  push:
    branches:
      - test0
jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - run: git fetch origin main
      - run: git push origin HEAD:main

The push fails, because actions/checkout performs a shallow clone (with depth 1), by default, and git fetch origin main is not enough for Git to realize that it is a fast-forward push.

I've seen other answers around the internet where people compute the number of commits in a PR, and pass a depth to actions/checkout that is one greater than that number. Even that feels indirect, and easy to mess up (e.g. in the presence of merge commits). fetch --unshallow is apparently a useful flag, and there are other ways to accomplish this, but they're all pretty subtle and easy to get wrong. I'm not sure what the right answer is, mostly wanted to document this thing I've found frustrating when trying to do something slightly more complicated with actions/checkout (clone one branch, push it to another).

One idea might be to let actions/checkout take a range input (e.g. range: branch1..branch2), which would clone just enough history so that you could perform git push origin branch2:branch1, or see which files changed between branch1 and branch2.

[polarathene]

The push fails, because actions/checkout performs a shallow clone (with depth 1), by default, and git fetch origin main is not enough for Git to realize that it is a fast-forward push.

Did you try to reproduce locally?

Take a look at this, it's roughly what the workflow run log is doing:

$ mkdir /tmp/example && cd /tmp/example

# Run actions/checkout@v2
$ git init
$ git remote add origin https://github.com/ryan-williams/gha-test


# Fetches a single commit from the test0 remote branch via the refspec:
$ git fetch --no-tags --prune --progress --no-recurse-submodules --depth=1 origin +95b6b3e9ed41c8263973db747396cc477cef005a:refs/remotes/origin/test0

 * [new ref]         95b6b3e9ed41c8263973db747396cc477cef005a -> origin/test0


# Checkout the branch, local branch is created tracking the remote one:
$ git switch -c test0 refs/remotes/origin/test0

branch 'test0' set up to track 'origin/test0'.
Switched to a new branch 'test0'


# Let's take a peek at the current commit history (you can run this, even without the git switch/checkout):
$ git log --oneline --graph --all

* 95b6b3e (grafted, HEAD -> test0, origin/test0) `git fetch main` / `git push origin main` fails


# Fetch main branch from remote:
$ git fetch origin main

 * branch            main       -> FETCH_HEAD
 * [new branch]      main       -> origin/main


# Look at what happened to the commit history, it pulled all of it in!:
$ git log --oneline --graph --all

* 95b6b3e (grafted, HEAD -> test0, origin/test0) `git fetch main` / `git push origin main` fails
* dae1edb (origin/main) --unshallow
* d1f42f4 rm --update-shallow
* bbccddf --update-shallow, fetch+push
* 817f726 try --shallow-exclude=main
* c41b96b try --update-shallow
* f50f717 Test fetch/pull/push
* e4ea485 test default input values
* e991fe8 optional env
*   cf1c54a Merge remote-tracking branch 'u/main'
|\  
| * 471168e add test2.txt
| * 203f8cb add test.txt
* | 8f877cf dispatch.yml
* | 511aba6 cat release
* | 5dc195a simplify
* | 4f651f2 add wait
* | fc27dea test concurrent commands
* | cb06238 add test2.txt
* | 673c428 add test.txt
|/  
* e423aa0 initial gha

Pay attention to those top two commits in the graph, it's not as obvious at a glance and may seem like origin/test0 is sharing that commit history with origin/main (which would be it's correct history too in this case), but see that "grafted"?:

# test0 branch:
$ git log --oneline --graph origin/test0

* 95b6b3e (grafted, HEAD -> test0, origin/test0) `git fetch main` / `git push origin main` fails


# Main branch:
$ git log --oneline --graph origin/main

* dae1edb (origin/main) --unshallow
* d1f42f4 rm --update-shallow
* bbccddf --update-shallow, fetch+push
* 817f726 try --shallow-exclude=main
* c41b96b try --update-shallow
* f50f717 Test fetch/pull/push
* e4ea485 test default input values
* e991fe8 optional env
*   cf1c54a Merge remote-tracking branch 'u/main'
|\  
| * 471168e add test2.txt
| * 203f8cb add test.txt
* | 8f877cf dispatch.yml
* | 511aba6 cat release
* | 5dc195a simplify
* | 4f651f2 add wait
* | fc27dea test concurrent commands
* | cb06238 add test2.txt
* | 673c428 add test.txt
|/  
* e423aa0 initial gha

They're not connected, test0 is a single commit on it's own branch. I don't think your fast-forward will work because there is no common ancestor commit linking the two branches?

# Make another copy where our current local repo will be our pretend remote to push to:
$ git clone "localhost:$(pwd)" ../example-local
$ cd ../example-local

# We've already got the main branch history here from the full clone above, so just push it:
$ git push origin HEAD:main

 ! [rejected]        HEAD -> main (non-fast-forward)
error: failed to push some refs to 'localhost:/tmp/example'
hint: Updates were rejected because a pushed branch tip is behind its remote
hint: counterpart. Check out this branch and integrate the remote changes
hint: (e.g. 'git pull ...') before pushing again.
hint: See the 'Note about fast-forwards' in 'git push --help' for details.

See...? Same problem you encountered in your workflow. Now, easy fix is to ensure you have that extra commit and is what my advice I've linked you to demonstrates in examples. We'll keep it simple for CLI here, but in your GH action you'd get the proper commit depth via GH context.

# Effectively the same as above (without refspec targeting a specific commit):
$ git clone --single-branch --branch test0 --depth 2 https://github.com/ryan-williams/gha-test /tmp/gha-test
$ cd /tmp/gha-test


# Now there's the extra commit from main branch which is the grafted commit:
$ git log --oneline --graph --all

* 95b6b3e (HEAD -> test0, origin/test0) `git fetch main` / `git push origin main` fails
* dae1edb (grafted) --unshallow


# Fetch origin main, but note that this time since we already have that common commit, it doesn't fetch all history:
$ git fetch origin main
$ git log --oneline --graph --all

* 95b6b3e (HEAD -> test0, origin/test0) `git fetch main` / `git push origin main` fails
* dae1edb (grafted) --unshallow


# Another copy to try push:
$ git clone "localhost:/tmp/gha-test" /tmp/gha-test-local
$ cd /tmp/gha-test-local
$ git push origin HEAD:main


# Success! The "remote" now has the test0 commit belonging to it's main branch:
$ cd /tmp/gha-test
$ git log --oneline --graph --all

* 95b6b3e (HEAD -> test0, origin/test0, main) `git fetch main` / `git push origin main` fails
* dae1edb (grafted) --unshallow

That simple! :)

Hope that explains for you why it didn't work out and how we resolved it effectively. All you should have to do is ensure you fetch enough commits.

---

easy to mess up (e.g. in the presence of merge commits). fetch --unshallow is apparently a useful flag, and there are other ways to accomplish this, but they're all pretty subtle and easy to get wrong.

Read my answers. They are fairly detailed about approach and where things could go wrong. --unshallow brings in unnecessary commits.

I'm not sure what the right answer is, mostly wanted to document this thing I've found frustrating when trying to do something slightly more complicated with actions/checkout (clone one branch, push it to another)

I documented / shared my solution for similar reasons.

which would clone just enough history so that you could perform git push origin branch2:branch1, or see which files changed between branch1 and branch2.

That's effectively what my linked comments were about.

I would have PRs and want to bring in all the commits of that branch (from the point it branched off at), then perform an operation like a file name diff against the target branch (main/master). I spent a fair amount of time looking into that.

Look at this use of --shallow-since: #520 (comment)

IIRC, I mention in one of the comments that if you've got a lot of merge commits they're counted as a commit by the github context commit count, but don't count towards fetch count, so you can end up fetching a little excess history via --depth, but that's far better than --unshallow. A common ancestor commit is then derived between the two branches, and the commit date give to --shallow-since to fetch the target branches newer commits since branching off.