Ability to automatically push to remote mirror repository

[yanjiew1]

I like the feature of mirroring the repositories by pulling the remote repositories.

Is it possible to make gitea to push local repositories to remote mirror repositories?

It's nice if gitea is able to automatically push local repositories to remote mirror repositories such as github, gitlab or other git services.

[mxmehl]

Good idea, I have been looking for such a option already.

IMHO this would increase adoption of Gitea. People would be more inclined to using Gitea if they knew that they could easily mirror stuff to GitLab/Github without having to fiddle with hooks.

[ghost]

I'm also pretty interested in this, and wouldn't mind having a go at implementing it if someone could point me in the right direction!

[thehowl]

Right now I am doing this on a project of mine. Since I am the admin of the instance, I have access to the git hooks of the repos, and I can add this as the post-receive hook:

#!/bin/bash
git push --mirror --quiet https://user:token@github.com/osuripple/hanayo.git &> /dev/null
echo 'osuripple/hanayo updated'

Of course, this is just a hack, and a proper mechanism would be gladly appreciated.

[ulm0]

this would be a really nice feature to have in Gitea, please!

[ghost]

Yeah this would be an amazingly useful feature!

[ghost]

Is this really just a limitation of other systems disguised to look like a feature in Gitea? If Webhooks can do it why run the risk and adding complexity trying to make Gitea manage features absent from other products?

Now, if we could have an SVN push mirror that would be extremely useful use case for the ad hoc nature of the umptysquillion poorly managed WordPress SVN repos in existence today who would gladly being porting over go Gitea to simplify their workflow and reduce premature aging.

[snth]

@thehowl Thanks. I made a slight modification so I don't have to wait for the push to the mirror to complete (which can be slow).

#!/bin/bash
git push --mirror --quiet https://user:token@github.com/osuripple/hanayo.git &> /dev/null &
echo 'osuripple/hanayo update initiated.'

Of course now you no longer have a confirmation that the push succeeded but a main motivation for my use case of gitea was to avoid the long lags of updates to slow remote hosts.

[pklapperich]

The git hook trick won't work for mirror repos. And a lot of services don't respond to inbound webhooks, for example gitea itself doesn't, a simple ssh server acting as a git remote, etc. Having this built in would be useful.

[lunny]

You have to store the token or private key on gitea if we implement that.

[chotaire]

This is a very interesting feature!

[belliash]

+1

[HarvsG]

This would be a very useful feature. It would be great to have a system where users could develop a project on our hosted Gitea instance and then 'publish' it to a public git service by setting it up as a mirrored repo.

[millergarym]

You have to store the token or private key on gitea if we implement that.

No, ssh auth forwarding could be used.

[petrac-daniel]

After I used the hack from above, I've written an post receive script to update the remote repo with all changes from gitea. All changes made in gitea will be automatically pushed to the "mirror repo" but not the other way around. Updating the gitea repo is still manually.

It is unfortunately still necessary to update the config file of the repo and the git authentication on the maschine running gitea:

[remote "origin"]
	url = <remote_url_here>
	fetch = +refs/heads/*:refs/remotes/origin/*

#!/usr/bin/env bash

z40=0000000000000000000000000000000000000000

while read local_ref local_sha remote_ref remote_sha
do
    if [[ $remote_ref =~ refs\/heads\/(.+) ]]; then
        branch_name=${BASH_REMATCH[1]}
        
        if [ "$local_sha" = $z40 ]
        then
            # Handle delete
            echo HANDLE DELETE BRANCH ON <remote_url_here>
            echo ----------
            git push --delete origin $branch_name
            echo ----------
        else
            echo HANDLE PUSH BRANCH ON ON <remote_url_here>
            echo ----------
            git push --set-upstream origin $branch_name
            echo ----------
        fi
    else
        echo "unable to parse remote branch $branch_name"
    fi
done

[PeterHindes]

Any progress on adding this to the gitea source? It seems like all we have are workarounds, Fairly clever ones at that, but some actual progress would be a good sign.

Gitlab has this feature built in already Might be some useable code there.

[WattsC-90]

I am interested in this feature primarily because our current infrastructure uses mirrored servers globally for developers to use to improve speeds, there is a node next to (virtually) the build agent/cluster to speed up build checkouts etc. too. This includes failover too, if the primary node goes down, then we can turn one of the mirrors into primary. (I must note, we are using SVN, so the use of master/slave nodes may be redundant, but I want to pitch in with why I was looking and my use case to help, if possible, improve the end result.) I am loving the feel of this project and shall be looking to pull our SVN system over to GIT if corporate decision makers allow... just needs the right feature set!

Another use case we have discussed previously is to commit to one server for code reviews, and then once the code review is done, push it to the master. Whenever you checkout, you get it from the master. Similar to the diagram here (only the last two stages with "push to gerrit" and "submit". Mirroring functionality would allow such a feature to be offered.

[referenced from: https://www.bsi-software.com/fileadmin/daten/processed/5/7/csm_Gerrit_workflow_33e6332529.png]

[lafriks]

It's not ideal but drone ci could be used to push changes further to other git server

[WattsC-90]

@lafriks I have TeamCity already in place and working well for the business, so dont really want to change CI product, just keen to have a commit gating process like the one shown.

[StrangeWill]

@WattsC-90 I mean it could be done in TeamCity too, it's an interesting suggestion and I'm considering it -- we have an internal CI server that we build everything on, then sometimes our customers have their own repos/servers/etc that right now we manually push to, would be nice to stop doing that..

I could just make part of TeamCity's build process to push out to the external repo on successful build (a good-ish idea), and even if TeamCity was on the other "side" of the push, I could just make a build that does nothing but sync on change.

Not the best workaround, but even with Gitlab we're worried that repo mirroring and security won't be the best, so it is kind of a "if nothing else works, make CI do it".

[WattsC-90]

@StrangeWill I found this in my downtime, might be an interesting reference to how it could be done... https://github.com/takezoe/gitmesh just wonder if by having a topology similar to that shown in the github repo linked, the logic could be put in the "controller" level (see link), to push to "golden" repo (or potentially any other task..)

[poperigby]

I wonder if it would also be possible to mirror the issue tracker with a bot.

[keks24]

@thehowl Thanks. I made a slight modification so I don't have to wait for the push to the mirror to complete (which can be slow).

#!/bin/bash
git push --mirror --quiet https://user:token@github.com/osuripple/hanayo.git &> /dev/null &
echo 'osuripple/hanayo update initiated.'

Of course now you no longer have a confirmation that the push succeeded but a main motivation for my use case of gitea was to avoid the long lags of updates to slow remote hosts.

I took your adaptation a bit further:
https://github.com/keks24/git-hooks/blob/master/home/gitea/.local/share/gitea/hooks/post-receive.d/post-receive

Be aware, that git push --force will fail on the mirrors, if branches are protected.

[lunny]

If we want gitea to do that, it depends on #12065

[neumantm]

For anyone wanting to update a github mirror with a deploy key rather than access token:

#!/bin/bash

# Setup ssh config
grep -qxF 'Include config.d/*' ~/.ssh/config || echo 'Include config.d/*' >> ~/.ssh/config
mkdir --parents ~/.ssh/config.d/
cat << EOF > ~/.ssh/config.d/github-mirror-setup
Host github.com
CheckHostIP no
EOF
# Need to add github to known hosts, because cannot press yes on first connect.
grep -qF 'github.com' ~/.ssh/known_hosts || echo 'github.com ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAq2A7hRGmdnm9tUDbO9IDSwBK6TbQa+PXYPCPy6rbTrTtw7PHkccKrpp0yVhp5HdEIcKr6pLlVDBfOLX9QUsyCOV0wzfjIJNlGEYsdlLJizHhbn2mUjvSAHQqZETYP81eFzLQNnPHt4EVVUh7VfDESU84KezmD5QlWpXLmvU31/yMf+Se8xhHTvKSCZIFImWwoG6mbUoWf9nzpIoaSjB+weqqUUmpaaasXVal72J+UX2B+2RPW3RcT0eOzQgqlJL3RKrTJvdsjE3JEAvGq3lGHSZXy28G3skua2SmVi/w4yCE6gbODqnTWlg7+wC604ydGXA8VJiS5ap43JXiUFFAaQ==' >> ~/.ssh/known_hosts

# Setup ssh-agent
eval `ssh-agent`
cat << EOF | ssh-add -
-----BEGIN OPENSSH PRIVATE KEY-----
<your private key here>
-----END OPENSSH PRIVATE KEY-----
EOF

ssh -T git@github.com
git push --mirror git@github.com:<username>/<reponame>.git
echo 'github mirror updated.'

[rexzhang]

This is a very nice feature!

[kdumontnu]

Bounty added:

If you'd like to promote this, please add

[![Bountysource](https://api.bountysource.com/badge/issue?issue_id=54827645)](https://www.bountysource.com/issues/54827645-ability-to-automatically-push-to-remote-mirror-repository)

To the Issue summary

Edit: @techknowlogick can this get the bounty badge?

[KN4CK3R]

I've started some days ago working on this.

[6543]

@KN4CK3R dont forget to claim the bounty :)