Improve generate-release workflow involving editing and merging not…

…es and guides (#1650)

Co-authored-by: Alice Cecile <alice.i.cecile@gmail.com>
Co-authored-by: BD103 <59022059+BD103@users.noreply.github.com>
Co-authored-by: Asier Illarramendi <asier@illarra.com>

content/learn/contribute/project-information/release-process.md

@@ -43,17 +43,19 @@ When making a release, the Maintainers follow these checklists:
 1. Check regressions tag.
 2. Check appropriate milestone and close it.
 3. Check GitHub Projects page for staleness.
-4. Update change log.
-5. Create migration guide.
-6. Write blog post.
-7. Update book.
-8. Bump version number for all crates, using the "Release" workflow.
+4. Run the [`generate-release`](https://github.com/bevyengine/bevy-website/tree/main/generate-release) tool.
+    1. Create migration guide.
+    2. Write blog post.
+    3. Generate contributors list
+    4. Generate change log.
+5. Update book.
+6. Bump version number for all crates, using the "Release" workflow.
    1. Change the commit message to be nicer.
-9. Create tag on GitHub.
-10. Edit GitHub Release. Add links to the `Release announcement` and `Migration Guide`.
-11. Bump `latest` tag to most recent release.
-12. Run the [`update-screenshots` workflow] to update screenshots. *This will block blog post releases (and take ~40 minutes) so do it early*.
-13. Run the [`build-wasm-examples` workflow] to update Wasm examples.
+7. Create tag on GitHub.
+8. Edit GitHub Release. Add links to the `Release announcement` and `Migration Guide`.
+9. Bump `latest` tag to most recent release.
+10. Run the [`update-screenshots` workflow] to update screenshots. *This will block blog post releases (and take ~40 minutes) so do it early*.
+11. Run the [`build-wasm-examples` workflow] to update Wasm examples.
 
 #### Minor Release
 

generate-release/Cargo.toml

@@ -15,6 +15,7 @@ dotenvy = "0.15.6"
 serde_json = "1.0.91"
 rayon = "1.10.0"
 thiserror = "1.0.61"
+toml = "0.8.19"
 
 [lints]
 workspace = true

generate-release/README.md

@@ -4,12 +4,14 @@ This CLI tool is used to generate all the skeleton files required to create a ne
 
 For a bit more background see this issue: <https://github.com/bevyengine/bevy-website/issues/1163>
 
-All commands assume they are ran in the `/generate-release` folder.
+The commands can be run from anywhere inside the workspace folder. If you have a `.env` file, this will only work if it is located at the root of the workspace.
 
 Each command will generate files in the `/release-content/{release-version}` folder. The `release-version` is an argument to all commands.
 
 Each command have a `--from` and `--to` argument. You can pass it a Git branch, tag, or commit.
 
+To create issues for the `release-notes` subcommand, you need to pass the `--create-issues` flag, otherwise it performs a dry-run that does not have lasting consequences. This should probably only be done for the initial run, after a regular dry-run has been done to confirm the tool is working as expected.
+
 Before running the command, you'll need to generate a GitHub API token at <https://github.com/settings/tokens>. It's easier to use classic tokens.
 The token must have `repo` permissions to be able to open issues (and PRs) on your behalf.
 
@@ -22,16 +24,18 @@ GITHUB_TOKEN=token_string_copied_from_github
 Here's an example for the commands used to generate the `0.14` release:
 
 ```shell
-cargo run -- --from v0.13.0 --to main --release-version 0.14 migration-guides
-cargo run -- --from v0.13.0 --to main --release-version 0.14 release-notes
-cargo run -- --from v0.13.0 --to main --release-version 0.14 changelog
-cargo run -- --from v0.13.0 --to main --release-version 0.14 contributors
+cargo run -p generate-release -- --from v0.13.0 --to main --release-version 0.14 migration-guides
+cargo run -p generate-release -- --from v0.13.0 --to main --release-version 0.14 release-notes
+cargo run -p generate-release -- --from v0.13.0 --to main --release-version 0.14 changelog
+cargo run -p generate-release -- --from v0.13.0 --to main --release-version 0.14 contributors
 ```
 
 ## Generating a release
 
 To generate a release from scratch, run all these commands then add the new migration guide and blog post to their respective `/content` folder. When doing so, it's important to use the `public_draft` feature to hide those pages until the day of the release. For the `public_draft` feature, you'll need to provide it a GitHub issue number, it's recommended to point it to an issue tracker for the current release being worked on. The issue needs to be on the `bevy-website` repo.
 
+When you're merging or editing notes and guides, keep in mind that this tool will not regenerate notes or guides that still have a PR number in any note or guide's metadata, contained in the `_<release_notes|guides>.toml`. This means to merge multiple PRs into one note or guide you simply remove one `[[release_notes]]` or `[[guides]]` entry, and move its PR number to the merged entry that is the sum of all the merged PRs. For editing, this means the other metadata will also not be regenerated if the PR number still exists in the metadata.
+
 The following sections go in more details on each parts of the process.
 
 ### Migration Guides
@@ -45,15 +49,22 @@ Inside that file, you should have something that looks like this:
 
 ```markdown
 +++
-title = "0.13 to 0.14"
+# Let Xa be the old major version, and ya the old minor version,
+# and Xb be the new major version, and yb the new minor version.
+# 
+# Change the Bevy versions below to match these!
+title = "Xa.ya to Xb.yb"
 insert_anchor_links = "right"
 [extra]
-weight = 9
-long_title = "Migration Guide: 0.13 to 0.14"
+# Let N be the weight of the prior / last migration guide, plus one.
+weight = N
+long_title = "Migration Guide: Xa.ya to Xb.yb"
+# GitHub issue number for tracking this release's
+# migration guides or news post.
 public_draft = _release tracking issue number_
 +++
 
-{{ migration_guides(version="0.14") }}
+{{ migration_guides(version="Xb.yb") }}
 ```
 
 The most important part of this is the `migrations_guides` shortcode. It will get the list of guides from the `_guides.toml` file and combine all the separate file and generate appropriate markup for it.
@@ -74,25 +85,34 @@ Once all those files are generated you'll need to create a new blog post in `/co
 
 ```markdown
 +++
-title = "Bevy 0.14"
-date = 2024-05-17
+# Let X be the major version, and y the minor version.
+# Change the Bevy release versions below to match this one!
+title = "Bevy X.y"
+# Insert a date in the year, month, day format.
+# This should be the date that the post will be posted.
+date = YYYY-MM-DD
 [extra]
+# GitHub issue number for tracking this release's
+# news post.
 public_draft = _release tracking issue number_
 +++
 
 <!-- TODO Intro -->
 
 <!-- more -->
 
-{{ release_notes(version="0.14") }}
+{{ release_notes(version="X.y") }}
 
 ## What's Next?
 
 <!-- TODO What's next -->
 
 {{ support_bevy() }}
-{{ contributors(version="0.14") }}
-{{ changelog(version="0.14")}}
+{{ contributors(version="X.y") }}
+{{ changelog(version="X.y")}}
 ```
 
 The most important part of this is the `release_notes`, `changelog`, and `contributors` shortcodes. `release_notes` will get the list of release notes from the `_release_notes.toml` file and combine all the separate file and add them to this file. `contributors()` will load the `contributors.toml` file and generate the necessary markup. `changelog()` will load the `changelog.toml` file and generate the necessary markup.
+
+> [!NOTE]
+> The `contributors` field in `_release_notes.toml` is for all non-PR-author contributors to the PR; they should be added to the `authors` field on a case-by-case basis depending on level of involvement.

generate-release/src/migration_guides.rs

@@ -1,14 +1,34 @@
 use anyhow::Context;
+use serde::Deserialize;
 
 use crate::{
     github_client::{GithubClient, GithubIssuesResponse},
     helpers::{get_merged_prs, get_pr_area},
     markdown::write_markdown_section,
 };
-use std::{collections::BTreeMap, io::Write as IoWrite, path::PathBuf};
+use std::{
+    collections::BTreeMap,
+    fs::{self, OpenOptions},
+    io::Write as IoWrite,
+    path::PathBuf,
+};
 
 type PrsByAreaBTreeMap = BTreeMap<Vec<String>, Vec<(String, GithubIssuesResponse)>>;
 
+#[derive(Deserialize, Clone)]
+struct MigrationGuides {
+    guides: Vec<MigrationGuide>,
+}
+
+#[allow(dead_code)]
+#[derive(Deserialize, Clone)]
+struct MigrationGuide {
+    title: String,
+    prs: Vec<u64>,
+    areas: Vec<String>,
+    file_name: String,
+}
+
 pub fn generate_migration_guides(
     from: &str,
     to: &str,
@@ -25,6 +45,23 @@ pub fn generate_migration_guides(
     // We'll write the file once at the end when all the metdaata is generated
     let mut guides_metadata = Vec::new();
 
+    // If there is metadata that already exists,
+    // and would contain info such as which PR already
+    // has an entry, then get it and use it for that.
+    let preexisting_metadata_file = fs::read_to_string(path.join("_guides.toml")).ok();
+    // Deserializes the file inside the option into the `MigrationGuides` struct,
+    // and then transposes / swaps the internal result of that operation to external,
+    // and returns the error of that result if there is one,
+    // else we have our preexisting metadata, ready to use.
+    let preexisting_metadata: Option<MigrationGuides> = preexisting_metadata_file
+        .as_deref()
+        .map(toml::from_str)
+        .transpose()?;
+
+    eprintln!("metadata exists? {}", preexisting_metadata.is_some());
+
+    let mut new_prs = false;
+
     // Write all the separate migration guide files
     for (area, prs) in areas {
         let mut prs = prs;
@@ -34,6 +71,33 @@ pub fn generate_migration_guides(
         prs.sort_by_key(|k| k.1.closed_at);
 
         for (title, pr) in prs {
+            // If a PR is already included in the migration guides,
+            // then do not generate anything for this PR.
+            //
+            // If overwrite_existing is true, then ignore
+            // if the PRs may have already been generated.
+            if preexisting_metadata.is_some() && !overwrite_existing {
+                let preexisting_metadata = preexisting_metadata.clone().expect(
+                    "that preexisting metadata exists at the _guides.toml for this release version",
+                );
+                let mut pr_already_generated = false;
+
+                for migration_guide in preexisting_metadata.guides {
+                    if migration_guide.prs.contains(&pr.number) {
+                        pr_already_generated = true;
+                    }
+                }
+
+                if pr_already_generated {
+                    eprintln!("PR #{} already exists", pr.number);
+                    continue;
+                }
+            }
+
+            // If the code has reached this point then that means
+            // there is new PRs to be recorded.
+            new_prs = true;
+
             // Slugify the title
             let title_slug = title
                 .replace(' ', "_")
@@ -53,10 +117,7 @@ pub fn generate_migration_guides(
             let metadata_block = generate_metadata_block(&title, &file_name, &area, pr.number);
 
             let file_path = path.join(format!("{file_name}.md"));
-            if file_path.exists() && !overwrite_existing {
-                // Skip existing files because we don't want to overwrite changes when regenerating
-                continue;
-            }
+
             if write_migration_file(
                 &file_path,
                 pr.body.as_ref().context("PR has no body")?,
@@ -67,9 +128,27 @@ pub fn generate_migration_guides(
         }
     }
 
-    // Write the metadata file
-    let mut guides_toml = std::fs::File::create(path.join("_guides.toml"))
-        .context("Failed to create _guides.toml")?;
+    if !new_prs {
+        return Ok(());
+    }
+
+    let mut guides_toml = if overwrite_existing {
+        // Replace and overwrite file.
+        OpenOptions::new()
+            .write(true)
+            .truncate(true)
+            .create(true)
+            .open(path.join("_guides.toml"))
+            .context("Failed to create _guides.toml")?
+    } else {
+        // Append to the metadata file,
+        // creating it if necessary.
+        OpenOptions::new()
+            .append(true)
+            .create(true)
+            .open(path.join("_guides.toml"))
+            .context("Failed to create _guides.toml")?
+    };
     for metadata in guides_metadata {
         writeln!(&mut guides_toml, "{metadata}")?;
     }
@@ -98,7 +177,7 @@ fn get_prs_by_areas(
         let has_breaking_label = pr
             .labels
             .iter()
-            .any(|l| l.name.contains("C-Breaking-Change"));
+            .any(|l| l.name.contains("M-Needs-Migration-Guide"));
 
         // We want to check for PRs with the breaking label but without the guide section
         // to make it easier to track down missing guides
@@ -127,7 +206,7 @@ fn generate_metadata_block(
     format!(
         r#"[[guides]]
 title = "{title}"
-url = "https://github.com/bevyengine/bevy/pull/{pr_number}"
+prs = [{pr_number}]
 areas = [{areas}]
 file_name = "{file_name}.md"
 "#,