Github-as-code

[lemeurherve]

Manage @jenkins-infra "as code", in order to:

• streamline repositories access management, protections and settings (including their archived state)
• create and manage github apps
• manage teams membership
• manage labels
• be able to exclude certain repositories if needed
• be able to review repositories operations (renaming, protections, manage access)
• get an history of these operations
• avoid manual errors
• tag repositories (with their services for example)
• block all PR merges on release days

[lemeurherve]

In order to manage this, I'll have first to make a complete backup of all repositories (cloning them all, shouldn't be difficult), but also their settings, which I've never done and will have to search how to.

Does anyone has already done something similar? @timja maybe?

[lemeurherve]

Studying this atm: https://docs.github.com/en/enterprise-server@3.3/admin/user-management/migrating-data-to-and-from-your-enterprise/exporting-migration-data-from-githubcom

(from https://docs.github.com/en/repositories/archiving-a-github-repository/backing-up-a-repository)

[timja]

No I haven't but also wouldn't worry too much. Shouldn't be anything too special, it can always be re-created.

[lemeurherve]

FTR, backup of a repo, with an `admin:org` PAT:

# Start the migration (one repo, without lock, for testing purpose, should be locked for a real backup!), you'll get the migration id in return
curl -H "Authorization: token $GITHUB_ACCESS_TOKEN" -X POST -H "Accept: application/vnd.github.v3+json" https://api.github.com/orgs/jenkins-infra/migrations -d '{"repositories":["kubernetes-management"]}'

# Check the state of the migration
curl -H "Authorization: token $GITHUB_ACCESS_TOKEN" -H "Accept: application/vnd.github.v3+json" https://api.github.com/orgs/jenkins-infra/migrations/<migration-id>

# When the migration state is "exported", retrive the backup with this command returning a download link
curl -H "Authorization: token $GITHUB_ACCESS_TOKEN"-H "Accept: application/vnd.github.v3+json" https://api.github.com/orgs/jenkins-infra/migrations/<migration-id>/archive 

# Wait 7 days for automatic deletion or delete the archive directly
curl -H "Authorization: token $GITHUB_ACCESS_TOKEN" -H "Accept: application/vnd.github.v3+json" https://api.github.com/orgs/jenkins-infra/migrations/<migration-id>/archive -X DELETE

Result for this repo:

./attachments/52eeab2e-2475-4e72-8b0c-9efc58246991/image.png
./attachments_000001.json
./issue_comments_000001.json
./issue_events_000001.json
./issues_000001.json
./organizations_000001.json
./repositories/jenkins-infra/helpdesk.git/HEAD
./repositories/jenkins-infra/helpdesk.git/branches
./repositories/jenkins-infra/helpdesk.git/config
./repositories/jenkins-infra/helpdesk.git/description
./repositories/jenkins-infra/helpdesk.git/hooks/applypatch-msg.sample
./repositories/jenkins-infra/helpdesk.git/hooks/commit-msg.sample
./repositories/jenkins-infra/helpdesk.git/hooks/fsmonitor-watchman.sample
./repositories/jenkins-infra/helpdesk.git/hooks/post-update.sample
./repositories/jenkins-infra/helpdesk.git/hooks/pre-applypatch.sample
./repositories/jenkins-infra/helpdesk.git/hooks/pre-commit.sample
./repositories/jenkins-infra/helpdesk.git/hooks/pre-merge-commit.sample
./repositories/jenkins-infra/helpdesk.git/hooks/pre-push.sample
./repositories/jenkins-infra/helpdesk.git/hooks/pre-rebase.sample
./repositories/jenkins-infra/helpdesk.git/hooks/pre-receive.sample
./repositories/jenkins-infra/helpdesk.git/hooks/prepare-commit-msg.sample
./repositories/jenkins-infra/helpdesk.git/hooks/push-to-checkout.sample
./repositories/jenkins-infra/helpdesk.git/hooks/update.sample
./repositories/jenkins-infra/helpdesk.git/info/exclude
./repositories/jenkins-infra/helpdesk.git/logs/HEAD
./repositories/jenkins-infra/helpdesk.git/objects/info
./repositories/jenkins-infra/helpdesk.git/objects/pack/pack-0594aac5377047a9198e387bb7ef9dafe0ddb2d6.idx
./repositories/jenkins-infra/helpdesk.git/objects/pack/pack-0594aac5377047a9198e387bb7ef9dafe0ddb2d6.pack
./repositories/jenkins-infra/helpdesk.git/packed-refs
./repositories/jenkins-infra/helpdesk.git/refs/heads
./repositories/jenkins-infra/helpdesk.git/refs/tags
./repositories_000001.json
./repository_files/7823979/1.pcap.zip
./repository_files_000001.json
./schema.json
./teams_000001.json
./users_000001.json

[jetersen]

Have you seen: https://github.com/github/safe-settings ?

[lemeurherve]

Have you seen: github/safe-settings ?

Interesting, thanks @jetersen !

Found this quick summary:

safe-settings Key Features
In safe-settings all the settings are stored centrally in an admin repo within the organization. This is important. Unlike Settings Probot, the settings files cannot be in individual repositories.

There are 3 levels at which the settings could be managed: Org-level settings are defined in .github/settings.yml Suborg level settings. A suborg is an arbitrary collection of repos belonging to projects, business units, or teams. The suborgsettings reside in a yaml file for each suborg in the .github/suborgsfolder. Repo level settings. They reside in a repo specific yaml in .github/reposfolder

It is recommended to break the settings into org-level, suborg-level, and repo-level units. This will allow different teams to be define and manage policies for their specific projects or business units.With CODEOWNERS, this will allow different people to be responsible for approving changes in different projects.

I was thinking about using another official github tool/integration, https://github.com/integrations/terraform-provider-github, which has much more articles written about (or so I found after a quick search)

I'll test the two of them on another staging org to see and compare what can be done with each of them.

[alecharp]

Just to mention it here, there is also https://github.com/probot/settings which could help having common set of labels across repositories. This would help the release-drafter configuration as well.

[jetersen]

github safe settings also supports this @alecharp :)