Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[INFRA-2697] Unfork repos #2272

Open
jenkins-infra-bot opened this issue Aug 3, 2020 · 42 comments
Open

[INFRA-2697] Unfork repos #2272

jenkins-infra-bot opened this issue Aug 3, 2020 · 42 comments
Labels
github imported-jira-issue Issues imported from the Jira INFRA project, see #9 jira-component:github jira-type:github

Comments

@jenkins-infra-bot
Copy link

As noted in cli/cli#1467 there are various repositories in @jenkinsci which are still formally considered forks of long-dormant personal repositories, even though they are in fact official. We should remove the linkage. jenkins-test-harness-htmlunit and plugin-pom I have noticed, but I am sure you could find others with a search.


Originally reported by jglick, imported from: Unfork repos
  • assignee: danielbeck
  • status: In Progress
  • priority: Major
  • resolution: Unresolved
  • imported: 2022/01/10
@jenkins-infra-bot
Copy link
Author

danielbeck:

This is a work in progress I've recently started with jenkins-infra/infra-reports#24 and jenkins-infra/jenkins.io#3585. I plan to send a proposal to the dev list this week.

In another issue tracker this is known as JENSEC-921.

@jenkins-infra-bot
Copy link
Author

@jenkins-infra-bot
Copy link
Author

hlemeur:

Daniel Beck I see your PRs have been merged, is there still something to do for this issue or can it be closed?

@jenkins-infra-bot
Copy link
Author

danielbeck:

Still open.

@offa
Copy link

offa commented Aug 5, 2022

@timja
Copy link
Member

timja commented Aug 5, 2022

cc @MarkEWaite, @jmMeessen, @res0nance ^^

@basil
Copy link
Collaborator

basil commented Aug 5, 2022

I've got one of these too - https://github.com/jenkinsci/jakarta-mail-api-plugin - but I have no idea how I would go about unforking it.

@NotMyFault
Copy link
Member

NotMyFault commented Aug 5, 2022

If nothing changed over the past couple of years, this is still done manually by submitting a request to GitHub through its bot assistent.
Looks like you need to be an org or repo admin for that.

@MarkEWaite
Copy link

Based on https://stackoverflow.com/questions/38831301/how-to-un-fork-the-github-repository/66470086#66470086 I was able to use their virtual assistant to submit a request to unfork the elastic axis plugin and the git plugin. I hit the rate limit when I tried to unfork the node label parameter plugin

@timja
Copy link
Member

timja commented Aug 5, 2022

I used the virtual assistant recently for one, plugin-pom I think. It was done fairly quickly

@MarkEWaite
Copy link

I used the virtual assistant recently for one, plugin-pom I think. It was done fairly quickly

All four of my requests are now complete. The request for git plugin is resolved.

@NotMyFault
Copy link
Member

Btw, is there a policy for the forking process, because https://github.com/jenkinsci/ecu-test-execution-plugin was just forked ~10h ago and may be considered as unforking candidate sooner or later 🤔

@timja
Copy link
Member

timja commented Aug 5, 2022

see jenkins-infra/repository-permissions-updater#2679 (comment)

(wording could probably be updated as there's now an official process from GitHub side on it)

@basil
Copy link
Collaborator

basil commented Aug 5, 2022

I successfully used the virtual assistant to unfork https://github.com/jenkinsci/jakarta-mail-api-plugin - thanks!

@NotMyFault
Copy link
Member

NotMyFault commented Aug 5, 2022

I submited a request for repositories I can manage 👍🏻

Edit: Btw, basil, where did you get this list from? If I query forks via https://github.com/orgs/jenkinsci/repositories?q=&type=fork&language=&sort= I receive 776 results 👀

@basil
Copy link
Collaborator

basil commented Aug 5, 2022

Just browsing around a few popular plugins.

@timja
Copy link
Member

timja commented Aug 5, 2022

There's a report here that has all plugins in this situation:
https://reports.jenkins.io/github-jenkinsci-fork-report.json

@timja
Copy link
Member

timja commented Aug 5, 2022

I've created a request for the snakeyaml one

@NotMyFault
Copy link
Member

There's a report here that has all plugins in this situation: https://reports.jenkins.io/github-jenkinsci-fork-report.json

If we want to continue detaching forks, I'd recommend that one person submits all eligible repositories on behalf of the jenkinsci umbrella organization.
I don't think we necessarily need to create one ticket per person, if we can and want to get all done in one row.

@timja
Copy link
Member

timja commented Aug 5, 2022

One of the questions is 'have you committed to the repository'

I don't think one person will meet that criteria, although possibly they would let an organisation owner do it.

@NotMyFault
Copy link
Member

One of the questions is 'have you committed to the repository'

That was no blocker for a repo I detached an hour ago.

although possibly they would let an organisation owner do it.

I just asked them on my ticket, 'cause I'd rather not want to submit 770 tickets.
Gonna add it up when they respond.

@NotMyFault
Copy link
Member

Since high volume requests consume a non-trivial amount of resources (and our support tooling isn't equipped to handle more than a few forks at a time), this has been escalated to another team for further review.

While we don't yet have an ETA, we'll be sure to be in touch with next steps, or to keep you apprised of any news.

Still ongoing.

@NotMyFault
Copy link
Member

NotMyFault commented Aug 16, 2022

GH evaluated the feasibility internally, but given unforking is a manual resource and time intensive process, GH is unable to unfork the vast amount of forks we host.

However, they offered us to change all fork network owners to the jenkinsci organization, if desired.
The latter requires an approval by a GH org administrator on my ticket.
GH also advised us to write something down about the network change on our community forums, they (and we) can point people to, in case someone reaches out for them asking about the change.

@timja
Copy link
Member

timja commented Aug 16, 2022

Should be fine to do that, cc @daniel-beck

@NotMyFault
Copy link
Member

NotMyFault commented Aug 16, 2022

I sent you the email to authorize the process, Tim.

I could create a forum post later on, if we want to proceed with this.

@daniel-beck
Copy link

However, they offered us to change all fork network owners to the jenkinsci organization, if desired.

This would be wrong for some repos, so we would provide either a list of repos to apply this to, or a list of repos to not apply this to. Otherwise https://github.com/jenkinsci/jgit https://github.com/jenkinsci/rubywm https://github.com/jenkinsci/plexus-utils https://github.com/jenkinsci/jna https://github.com/jenkinsci/jmdns-fork would probably upset some folks.

@daniel-beck
Copy link

@timja
Copy link
Member

timja commented Aug 17, 2022

We could delete those forks? I don't think any of them are used now

@daniel-beck
Copy link

daniel-beck commented Aug 17, 2022

We could delete those forks? I don't think any of them are used now

These are a handful of examples I found when browing through the repos list. Is that something you can confidently say about every fork that should be a fork?

We'll need this list of forks to change the root of and forks to keep as forks anyway, to be able to delete the latter, so we do not need to provide a list to GH.

@NotMyFault
Copy link
Member

NotMyFault commented Aug 17, 2022

However, they offered us to change all fork network owners to the jenkinsci organization, if desired.

This would be wrong for some repos, so we would provide either a list of repos to apply this to, or a list of repos to not apply this to. Otherwise https://github.com/jenkinsci/jgit https://github.com/jenkinsci/rubywm https://github.com/jenkinsci/plexus-utils https://github.com/jenkinsci/jna https://github.com/jenkinsci/jmdns-fork would probably upset some folks.

I started writing down a non exhaustive list of forks that should remain as is, given those are legit forks outside the plugin hosting process: https://gist.github.com/NotMyFault/dfec71b094419f45145d2c187636b6b1
Though this may not be considered given GH offered us to process all forks.

@daniel-beck
Copy link

non exhaustive list of forks

Can you do this as a list of all forks we have, and label them "actual fork" and "hosting fork"? (FWIW I'd use a 2 column Google Doc for this.) That way it's easy to confirm all repos were looked at, and an explicit decision to unfork/root (or not) was made. Especially interesting also for weird edge cases (we also have some non-plugin forks of vaguely Jenkins related repos of contributors…)

@timja
Copy link
Member

timja commented Aug 17, 2022

GitHub have not offered to process a list from what I understand they would do all of them or small numbers.

If we don’t want something unforked and we want to do this process we need to delete the repo, which could mean moving it to a temp account and back

@daniel-beck
Copy link

If we don’t want something unforked and we want to do this process we need to delete the repo, which could mean moving it to a temp account and back

Right, but that still means we should have a complete list to minimize the risk that we have forked another project and suddenly become root of their network.

@NotMyFault
Copy link
Member

GitHub have not offered to process a list from what I understand they would do all of them or small numbers.

Right, GH's offer covers all forks.

have a complete list

Based on the fork report, I generated https://docs.google.com/spreadsheets/d/197A5yzVCfBsXkRF3Uh9JbSUKD2kU7q-z6O8rNxLdhNE/edit#gid=0, which highlights repositories that should remain as is and we would need to move to a temp account (or possibly delete, if no commits have been made and the repository brings in nothing of use and and value).

@daniel-beck
Copy link

highlights repositories that should remain as is and we would need to move to a temp account

Right, and now explicitly mark those that should have their network root in jenkinsci. TBH I'd be pretty surprised if that's trivially decidable for everything not currently marked to remain a fork.

@timja
Copy link
Member

timja commented Aug 17, 2022

I've reviewed all of the repositories not containing -plugin and added a couple more.

Right, and now explicitly mark those that should have their network root in jenkinsci. TBH I'd be pretty surprised if that's trivially decidable for everything not currently marked to remain a fork.

I don't quite understand. Shouldn't all the plugins have their network root as jenkinsci?

@daniel-beck
Copy link

daniel-beck commented Aug 17, 2022

I don't quite understand. Shouldn't all the plugins have their network root as jenkinsci?

All all repos not marked to remain a fork plugins? No, plenty of counter-examples.

Are all -plugin Jenkins plugins? No, see the already marked Maven plugin.

So I suggest you go through the list and mark every entry with "Should remain fork", or "Is Jenkins plugin", as applicable. See whether anything remains unmarked.

@NotMyFault
Copy link
Member

I seperated the list across different sheets on the spreadsheet, but I also didn't find anything new reading over it.

Given that changing fork networks is a simple procedure on GH's end and we discover a fork we want to restore in the future, we can always go ahead and do so and prepare the few forks giving GH the go.

@daniel-beck
Copy link

daniel-beck commented Aug 17, 2022

The point of my recent messages is that we need to be able to distinguish "action should be taken" and "nobody has looked at this so we don't know yet". Yes, looking at 700+ repos to see whether they're Jenkins plugins, Jenkins related, or something else, is annoying, but it's IMO irresponsible to ask for batch action without any confidence that this won't mess with other folks' projects.

I know of at least one repo currently marked for rooting in this doc that shouldn't be because it's an actively maintained open source project unrelated to Jenkins that we have a fork of. This alone tells me that the "Jenkins plugins" tab was not populated carefully (even assuming you intended it to be wider than Jenkins plugins).

@hashar
Copy link

hashar commented Aug 18, 2022

Based on the fork report, I generated https://docs.google.com/spreadsheets/d/197A5yzVCfBsXkRF3Uh9JbSUKD2kU7q-z6O8rNxLdhNE/edit#gid=0, which highlights repositories that should remain as is and we would need to move to a temp account (or possibly delete, if no commits have been made and the repository brings in nothing of use and and value).

As the maintainer of https://github.com/jenkinsci/gearman-plugin.git I would very welcome it to become its own fork network (in case you are seeking feedback).

The plugin development was originally done on OpenStack Gerrit instance with a mirror set to https://github.com/openstack-infra/gearman-plugin . I think I forked it under the jenkinsci organization for convenience but there is really no need to keep that fork relation since the parent was merely a mirror, it had no issue tracker and the PR were automatically rejected. When proposing a PR to jenkinsci/gearman-plugin GitHub web UI defaults to fill the PR against the obsolete parent openstack-infra/gearman-plugin which is a bit confusing.

Thanks for leading that effort with GitHub!

@daniel-beck
Copy link

@hashar Note that GH said they cannot disconnect our stuff in batch (#2272 (comment)), but instead offered to make the jenkinsci fork the new root of the network.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
github imported-jira-issue Issues imported from the Jira INFRA project, see #9 jira-component:github jira-type:github
Projects
None yet
Development

No branches or pull requests

9 participants