gh-101100: Only show GitHub check annotations on changed doc paragraphs

[CAM-Gerlach]

PR #102513 implemented support for displaying Sphinx warnings on changed files inline in the GitHub UI, which can be a useful aid to authors that they've fixed any defects on the lines they've touched and not introduced new ones.

However, annotations are shown on an entire file, regardless of whether the line the warning was issued on was within or even near those actually modified by the PR. While this has helped motivate large-scale cleanup of many of these warnings, as I commented when this was first deployed it has naturally led to a serious amount of noise, annoyance and confusion for PR authors seeing warnings on lines they had nothing to do with, which in turn leads to the understandable temptation for authors to ignore valid warnings or just silencing them, hiding rather than fixing the underlying defect.

The solution is, naturally, to only show warnings on modified lines, which is somewhat complicated by the fact that warning lines are only necessarily accurate at the paragraph level, requiring some additional processing. After much delay and burnout, I've finally gotten around to implementing it myself with this PR.

As a bonus, the script now works equally as well locally as remotely; just pass the commits/branches/tags to compare with --check-and-annotate-changes <base> <branch>, or use the default (comparing the changes on the current HEAD with the merge base in the local main). Handy!

• Issue: Fix all Sphinx reference warnings in the documentation #101100

---

📚 Documentation preview 📚: https://cpython-previews--108065.org.readthedocs.build/

[AA-Turner]

Thank you for this work! Some initial comments:

[hugovk]

Thank you very much for this!

A few initial comments.

[CAM-Gerlach]

(Just to note, I made another commit to continue to test that the annotations appear (and don't appear) as expected on added files, and on added, fixed and modified warnings in modified files. It deliberately fails the new-warnings check, and I also marked it DO-NOT-MERGE just to be safe; once this is approved and otherwise ready to merge, I'll drop that commit and then queue it for automerge.)

[AA-Turner]

Some further comments:

[CAM-Gerlach]

Since the final changes were just to a comment and a __future__ import, I've dropped the various test commits to confirm various common and edge cases, so this should be ready to merge pending any final comments/suggestions (which hopefully aren't substantial or I'll have to temporarily add them back, heh).

[AA-Turner]

Thanks again for all the work on this!

A

[hugovk]

Thanks! Do we need to backport this?

[CAM-Gerlach]

Do we need to backport this?

I was actually just about to comment on this now as I remembered that I'd forgotten to do so yesterday. The same problems this fixes (warning spam, as well as line numbers being off, etc) will be present in the Files Changed views on the 3.12 branch, where the script is present (its not on 3.11).

We will need to backport PR #106460 first though as it makes comprehensive changes to these same files; its backport was deferred to after the reusable docs changes were merged, which they now are, so it seems we should be able to just go ahead with that backport first, and then this should backport cleanly.

(In case you didn't see it, I posted this reply after the relevant comment chain had already been resolved, which you might find interesting)

[CAM-Gerlach]

I had to manually backport #106460 due to some merge conflicts in the .nitignore, but it should be good to merge as soon as Thomas gets the chance to review it, and then we can merge and backport this (hopefully cleanly).

[CAM-Gerlach]

Alrighty, looks like that backport got in, so I guess we're gogo on this one at last!

[miss-islington]

Thanks @CAM-Gerlach for the PR 🌮🎉.. I'm working now to backport this PR to: 3.12.
🐍🍒⛏🤖

[bedevere-bot]

GH-108127 is a backport of this pull request to the 3.12 branch.