forked from sagemath/sage
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
sagemathgh-36686:
.ci/merge-fixes.sh
: Obtain patches via URL, make …
…customizable by repository variable <!-- ^^^^^ Please provide a concise, informative and self-explanatory title. Don't put issue numbers in there, do this in the PR body below. For example, instead of "Fixes sagemath#1234" use "Introduce new method to calculate 1+1" --> <!-- Describe your changes here in detail --> Instead of using `gh pr checkout`, we obtain the CI fixes via their patch URLs. - This is faster because typically we do not have to unshallow the repo to apply the patches (seconds instead of ~2 minutes) - Fewer surprises when applied to a PR based on an older release - Conjecturally uses fewer API queries, helping avoid sagemath#36685 When a repository variable `SAGE_CI_FIXES_FROM_REPOSITORIES` is set in a fork, it is used instead of the hardcoded sagemath/sage as the source(s) of the CI fixes; this gives better control in decentralized development. When set to "none", this also makes it possible to see the "ground truth", addressing a concern raised in sagemath#36349 (comment). See comments at the top of `.ci/merge-fixes.sh` for details. Also improving the display of details of the merged PRs, as requested by @tornaria in sagemath#36696 (comment) Example runs: - https://github.com/sagemath/sage/actions/runs/6854931832/job/186389018 59?pr=36686 (here it recovers gracefully from failed merges due to the 10.2.rc1/10.2.rc2 tagging confusions) - https://github.com/mkoeppe/sage/actions/runs/6855410957/job/1864038039 8#step:3:8 (I have set `SAGE_CI_FIXES_FROM_REPOSITORIES=none` in my fork -- for testing this PR) <!-- Why is this change required? What problem does it solve? --> <!-- If this PR resolves an open issue, please link to it here. For example "Fixes sagemath#12345". --> <!-- If your change requires a documentation PR, please link it appropriately. --> ### 📝 Checklist <!-- Put an `x` in all the boxes that apply. --> <!-- If your change requires a documentation PR, please link it appropriately --> <!-- If you're unsure about any of these, don't hesitate to ask. We're here to help! --> <!-- Feel free to remove irrelevant items. --> - [x] The title is concise, informative, and self-explanatory. - [x] The description explains in detail what this PR is about. - [x] I have linked a relevant issue or discussion. - [ ] I have created tests covering the changes. - [ ] I have updated the documentation accordingly. ### ⌛ Dependencies <!-- List all open PRs that this PR logically depends on - sagemath#12345: short description why this is a dependency - sagemath#34567: ... --> <!-- If you're unsure about any of these, don't hesitate to ask. We're here to help! --> URL: sagemath#36686 Reported by: Matthias Köppe Reviewer(s): Gonzalo Tornaría, Kwankyu Lee, Matthias Köppe
- Loading branch information
Showing
8 changed files
with
87 additions
and
33 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,36 +1,82 @@ | ||
#!/bin/sh | ||
# Merge open PRs from sagemath/sage labeled "blocker". | ||
REPO="sagemath/sage" | ||
GH="gh -R $REPO" | ||
PRs="$($GH pr list --label "p: blocker / 1" --json number --jq '.[].number')" | ||
if [ -z "$PRs" ]; then | ||
echo 'Nothing to do: Found no open PRs with "blocker" status.' | ||
else | ||
echo "Found PRs: " $PRs | ||
export GIT_AUTHOR_NAME="ci-sage workflow" | ||
export GIT_AUTHOR_EMAIL="ci-sage@example.com" | ||
export GIT_COMMITTER_NAME="$GIT_AUTHOR_NAME" | ||
export GIT_COMMITTER_EMAIL="$GIT_AUTHOR_EMAIL" | ||
git tag -f test_base | ||
git commit -q -m "Uncommitted changes" --no-allow-empty -a | ||
for a in $PRs; do | ||
git fetch --unshallow --all > /dev/null 2>&1 && echo "Unshallowed." | ||
echo "::group::Merging PR https://github.com/$REPO/pull/$a" | ||
git tag -f test_head | ||
$GH pr checkout -b pr-$a $a | ||
git checkout -q test_head | ||
if git merge --no-edit --squash -q pr-$a; then | ||
echo "::endgroup::" | ||
if git commit -q -m "Merge https://github.com/$REPO/pull/$a" -a --no-allow-empty; then | ||
echo "Merged #$a" | ||
# Apply open PRs labeled "blocker" from sagemath/sage as patches. | ||
# This script is invoked by various workflows in .github/workflows | ||
# | ||
# The repository variable SAGE_CI_FIXES_FROM_REPOS can be set | ||
# to customize this for CI runs in forks: | ||
# | ||
# - If set to a whitespace-separated list of repositories, use them instead of sagemath/sage. | ||
# - If set to "none", do not apply any PRs. | ||
# | ||
# https://docs.github.com/en/actions/learn-github-actions/variables#creating-configuration-variables-for-a-repository | ||
export GIT_AUTHOR_NAME="ci-sage workflow" | ||
export GIT_AUTHOR_EMAIL="ci-sage@example.com" | ||
export GIT_COMMITTER_NAME="$GIT_AUTHOR_NAME" | ||
export GIT_COMMITTER_EMAIL="$GIT_AUTHOR_EMAIL" | ||
mkdir -p upstream | ||
for REPO in ${SAGE_CI_FIXES_FROM_REPOSITORIES:-sagemath/sage}; do | ||
case $REPO in | ||
none) | ||
echo "Nothing to do for 'none' in SAGE_CI_FIXES_FROM_REPOSITORIES" | ||
;; | ||
*/*) | ||
echo "Getting open PRs with 'blocker' status from https://github.com/$REPO/pulls?q=is%3Aopen+label%3A%22p%3A+blocker+%2F+1%22" | ||
GH="gh -R $REPO" | ||
REPO_FILE="upstream/ci-fixes-${REPO%%/*}-${REPO##*/}" | ||
PRs="$($GH pr list --label "p: blocker / 1" --json number --jq '.[].number' | tee $REPO_FILE)" | ||
date -u +"%Y-%m-%dT%H:%M:%SZ" > $REPO_FILE.date # Record the date, for future reference | ||
if [ -z "$PRs" ]; then | ||
echo "Nothing to do: Found no open PRs with 'blocker' status in $REPO." | ||
else | ||
echo "Empty, skipped" | ||
echo "Found open PRs with 'blocker' status in $REPO: $(echo $PRs)" | ||
git tag -f test_base | ||
git commit -q -m "Uncommitted changes" --no-allow-empty -a | ||
for a in $PRs; do | ||
# We used to pull the branch and merge it (after unshallowing), but when run on PRs that are | ||
# based on older releases, it has the side effect of updating to this release, | ||
# which may be confusing. | ||
# | ||
# Here, we obtain the "git am"-able patch of the PR branch. | ||
# This also makes it unnecessary to unshallow the repository. | ||
# | ||
# Considered alternative: Use https://github.com/$REPO/pull/$a.diff, | ||
# which squashes everything into one diff without commit metadata. | ||
PULL_URL="https://github.com/$REPO/pull/$a" | ||
PULL_FILE="$REPO_FILE-$a" | ||
PATH=build/bin:$PATH build/bin/sage-download-file --quiet "$PULL_URL.patch" $PULL_FILE.patch | ||
date -u +"%Y-%m-%dT%H:%M:%SZ" > $PULL_FILE.date # Record the date, for future reference | ||
LAST_SHA=$(sed -n -E '/^From [0-9a-f]{40}/s/^From ([0-9a-f]{40}).*/\1/p' $PULL_FILE.patch | tail -n 1) | ||
COMMITS_URL="https://github.com/$REPO/commits/$LAST_SHA" | ||
echo "::group::Applying PR $PULL_URL @ $COMMITS_URL as a patch" | ||
export GIT_COMMITTER_NAME="$GIT_AUTHOR_NAME applying $PULL_URL @ $COMMITS_URL" | ||
if git am --signoff --empty=keep < $PULL_FILE.patch; then | ||
echo "---- Applied patch --------------------------------------------------------------------------------" | ||
cat $PULL_FILE.patch | ||
echo "--------------------------------------------------------------------8<-----------------------------" | ||
echo "::endgroup::" | ||
elif git am --abort \ | ||
&& if git fetch --unshallow --all > /dev/null 2>&1; then echo "Unshallowed"; fi \ | ||
&& echo "Retrying with 3-way merge" \ | ||
&& git am --empty=keep --3way < $PULL_FILE.patch; then | ||
echo "---- Applied patch --------------------------------------------------------------------------------" | ||
cat $PULL_FILE.patch | ||
echo "--------------------------------------------------------------------8<-----------------------------" | ||
echo "::endgroup::" | ||
else | ||
echo "---- Failure applying patch -----------------------------------------------------------------------" | ||
git am --signoff --show-current-patch=diff | ||
echo "--------------------------------------------------------------------8<-----------------------------" | ||
echo "::endgroup::" | ||
echo "Failure applying $PULL_URL as a patch, resetting" | ||
git am --signoff --abort | ||
fi | ||
done | ||
fi | ||
else | ||
echo "::endgroup::" | ||
echo "Failure merging #$a, resetting" | ||
git reset --hard | ||
fi | ||
done | ||
git log test_base..HEAD | ||
fi | ||
;; | ||
*) | ||
echo "Repository variable SAGE_CI_FIXES_FROM_REPOSITORIES, if set, should be a whitespace-separated list of repositories or 'none'" | ||
echo "Got: $SAGE_CI_FIXES_FROM_REPOSITORIES" | ||
exit 1 | ||
;; | ||
esac | ||
done |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters