Change how we create our release notes

[robertlaurin]

While going through the release notes for our 0.17.0 some good observations were made about how how some of the breaking change notes were not exactly breaking.

For example farday was marked as having a breaking change, but the PR that this message was derived from actually only changed the tests for Faraday.

Another example is a fairly significant breaking breaking change to the API was missing entirely.

While the automated release note process has saved us a lot of effort along the way, I think the project is at a point where it makes sense for these release notes to be more closely curated by humans.

I'm proposing we do away with the automated changelog generation and instead introduce something like an unreleased changelog section that is populated at the same time as the PRs that introduce the changes. That way we can make sure that PR that changes multiple gems populates the appropriate notes for each gem, some gems may receive breaking changes, and some may not.

In addition to that I strongly believe our release notes should link back to the PRs that introduce the changes.

[fbogsany]

I think the automation is helpful as a starting point, but it would be nice if the suggested changelog additions happened earlier, in the PRs making the changes, rather than just in the release PR.

Perhaps some kind of bot would be suitable for this? Alternatively, a PR template with a checklist that included "update changelog".

[robertlaurin]

I do think a PR template would be good regardless so the expectation is clearly signalled to anyone contributing (or reminding us). I agree about the automation, it would be nice if happened with each PR. Not sure what the would look like exactly, a CI check of sorts, or something that a maintainer would trigger before merging the PR?

[robbkidd]

The changelog generation I've experienced before would use pull request titles for the one-liner entries in the log. That provided maintainers an opportunity to edit the entry as a part of review and prior to merging when the title and a #nnnn reference to the PR would become part of the generated CHANGELOG. Generally, PRs that were backwards breaking would include in their descriptions the ways in which users would adjust to the backwards-breaking behavior. At release time, a maintainer would collect those migration instructions to manually (😭 ) add to a RELEASE_NOTES for a particular version.

Most of the changelog automation I know of is only capable of working with git commit summaries or PR titles, so there's no convenance of "and include these notes, too, please."

A PR template could help that human-in-the-loop migration instruction gathering. Maybe a section for:

"So, you've put a ! in your conventional commit. What do people need to do to adjust to this change?"