Single CHANGELOG.md for the whole project

[jawoznia]

Motivations

In current shape it is possible to generate/update CHANGELOG.md per crate or to specify which crate is "special" and generate CHANGELOG.md only for it.

I would appreciate if there was an option to aggregate the changes to some root level CHANGELOG.md.
Let's say there is a workspace repo with 2 crates

Cargo.toml
project/
project-derive/
CHANGELOG.md

I don't want separate CHANGELOG.md files per crate and using the single-changelog I need to specify for which crate the changes will be considered to fill the CHANGELOG.md

What I want is for release-plz to allow me to specify the CHANGELOG.md in the root or maybe even have it set as default and update it with all the changes in the git project.

• Would you like to implement this feature? n

Solution

It could be done either as:

[workspace]
changelog_path = "./CHANGELOG.md"

which would override the default settings and save the git changes to specified file

or maybe to allow specyfing multiple projects to use the same CHANGELOG.md

[workspace]
changelog_update = false

[[package]]
name = "project"
changelog_update = true
changelog_path = "./CHANGELOG.md"

[[package]]
name = "project-derive"
changelog_update = true
changelog_path = "./CHANGELOG.md"

Neither of those work in the current state unfortunately

[MarcoIeni]

How would the resulting changelog look like?

Does it include commits for both the projects? Or does it distinguish the commits between the two projects?

Is this what you want?

[jawoznia]

Yes this seems about right. I'm thinking about mixing commits from both crates.
It could consider all the commits that match the conventional commits in the git repo and apply them in CHANGELOG.md.
The other solution in which user would define which crates changes should be added to CHANGELOG is also fine.

[MarcoIeni]

It could consider all the commits that match the conventional commits in the git repo

For this you would need to configure git-cliff accordingly. I'm not sure it's possible. Let me know.
By default, release-plz puts the non conventional commits under the "Other" section.

[XAMPPRocky]

How would the resulting changelog look like?

Does it include commits for both the projects? Or does it distinguish the commits between the two projects?

For what it's worth, I would like to see the former. My project https://github.com/XAMPPRocky/rasn has one main crate, and a bunch of sub-crates that use the main crate to implement a standard. It would be great if it could any changes that affected a sub-crate as a separate heading in the CHANGELOG. Eg.

# 0.7.0

- Update Dependencies (#100) # Changes both `rasn` and `rasn-pkix`
- Fixed codec bug (#150) # Changes `rasn`

## rasn-pkix
- Update Dependencies (#100)
- Fixed CRL field tag (#200) # Changes `rasn-pkix`

[MarcoIeni]

This is not supported by git-cliff, the library we use to generate the changelog.
So we would need to replace it with our own changelog generation mechanism.

[XAMPPRocky]

So we would need to replace it with our own changelog generation mechanism.

It's funny because I just added it to the project to see how it would look, and what the bot outputs to GitHub is basically what I want, so I assume that you use the same logic you're using for the bot, in the tool directly and have it output to one file rather than to each file.

[MarcoIeni]

when generating it for the first time, it's easy and your approach works.
The issue is updating it: we need to parse the markdown in the format specified by the user config.

[XAMPPRocky]

The issue is updating it: we need to parse the markdown in the format specified by the user config.

Have you considered allowing users to specify a template for a release? That would mean you don't need to parse it and instead you could expose the abstract JSON to a template engine like Tera, and let the users generate the markdown, and then you just need parse enough to push it to the top of the CHANGELOG.

[MarcoIeni]

Isn't this what we already do?
Docs here.
We use git-cliff for changelog generation, which uses tera internally. 🤔

[XAMPPRocky]

Isn't this what we already do?

Ah I guess, but what I'm more saying is to move the formatting generation inside release-plz, and then in tera you just provide the different projects as an array or map. Also in that documentation, it doesn't actually specify what context object's schema is (or show the default template as an example), so it's hard to know what fields are available.

[MarcoIeni]

then in tera you just provide the different projects as an array or map.

Does this require that release-plz scans the whole git history or similar?
Up until now the changelog mechanism was "append only".

Also in that documentation, it doesn't actually specify what context object's schema is (or show the default template as an example), so it's hard to know what fields are available.

100% agree. At the moment I link to the git-cliff config, but the git-cliff docs are huge. Most of the git-cliff docs are not relevant for release-plz. It would be nice to hide git-cliff to the user, managing the changelog configuration in release-plz.toml 👍

[XAMPPRocky]

Does this require that release-plz scans the whole git history or similar?
Up until now the changelog mechanism was "append only".

I'm not entirely sure what you mean, what about providing the different projects in an array/map would require git history? I assumed that the changelog format would be for a new changelog entry, so the array/map would only contain projects who have new changes to be added to the changelog.

[MarcoIeni]

Oh, I see! Maybe you want:

# Changelog

## crate1 [0.4.5] - 2023-06-09

### Added
- new awesome features

## crate2 [0.1.0] - 2023-06-09

### Added
- new fix

## crate1 [0.4.4] - 2023-06-09

### Added
- new features

I thought you wanted:

# Changelog

## crate1

### [0.4.5] - 2023-06-09

#### Added
- new awesome features

### [0.4.4] - 2023-06-09

#### Added
- new features

## crate2

### [0.1.0] - 2023-06-09

#### Added
- new fix

[MarcoIeni]

version 1 only changes the top of the changelog, so it's feasible 👍

[XAMPPRocky]

@MarcoIeni Yes exactly, that the new crates/changes are always appended to the changelog regardless of their actual version, because if I were to release a new crate in the same workspace at 0.1.0 I would still want it at the top of the changelog so people can see it, where as if I added a new crate to an existing workspace and it tried to match the position with the version position of older crates, then it would mean that recent changes would be buried under old changes.

[frol]

We need it as well. Otherwise we get this instead of this (the second is manually constructed)

[MarcoIeni]

Maybe the changes of the second PR got lost?

[frol]

@MarcoIeni My point is that we want to release a single package borsh, and when changes are only made to borsh-derive we still want to have the CHANGELOG to reflect those. We cannot simply pick one package to reflect the whole project changelog.

As you can see, there are only changes in borsh-derive crate since 1.0.0-alpha.1: near/borsh-rs@borsh-v1.0.0-alpha.1...master, which is why release-plz does not pick up anything useful into the automatically generated changelog.

[MarcoIeni]

would this fix your issue? Can you make an example of the changelog/changelogs you would like release-plz to generate?

[frol]

@MarcoIeni That will probably work for borsh-rs, but it sounds like a more complex solution than just collecting all changes for the whole repo into a single CHANGELOG file. Do you have concerns about this approach? (This wouldn't work well for monorepos with several semi-independent crates, but will work great for borsh-rs, serde, tokio like repos)

[MarcoIeni]

Do you have concerns about this approach?

Yes, because as you said it doesn't work for everyone.

Not sure if the default behavior should be the one you described instead of the current one. Anyway, I think that the include option indicated in this issue should fix many issues. I'll try to find some time to work on it. In the meantime, if you want to see it sooner, PR is welcome!
Also, thanks a lot for your feedback, it helps me improving this project :)

[frol]

@MarcoIeni I did not suggest to turn it by default (though, I would say that most often it is the desired behavior, and yet, I would not make this opinionated breaking change now)

[MarcoIeni]

I have implemented the changelog_include mechanism here. I released the new release-plz binary, but I haven't updated the GitHub action because one check is failing due to how release-plz uses cargo. I will investigate 👍

[MarcoIeni]

The changelog_include config option is now in release-plz action, too! (v0.5.17). 🥳
I'm closing this issue because this option seems to work for both frol and jawoznia. 👍
I wrote the feature request from XAMPPRocky in #916 🙏

[MarcoIeni]

Fixed, thanks!

[frol]

@MarcoIeni You are the best! release-plz seems to be working as expected for borsh-rs now! near/borsh-rs#197