Release documentation is lacking key steps

[valeriupredoi]

The release process for ESMValTool has become quite involved, and needs performing a number of steps that are not currently documented, I will try and document those steps, mainly guiding myself by the flow in #2881 - obv not detailing some "sensitive" aspects to privacy (eg VM) but making sure the next RM is not in the thick fog. This is needed even if the RTW will come online by the time of the next release. I will need past RMs like @sloosvel and DKRZ seasoned tenants like @remi-kazeroni and @schlunma to have a look at the PR, please. Also, we need to do something about the data on Levante, that is currently spread in various dirs, something @bouweandela and myself were not all too happy to see 👍

[schlunma]

Awesome, thanks for tackling this V! 🚀

Also, we need to do something about the data on Levante, that is currently spread in various dirs, something @bouweandela and myself were not all too happy to see +1

This is the only thing I disagree with. We have basically two locations where the data is stored - one is the directory maintained by DKRZ and the other one is our "global" download directory where we store data that is not provided "natively" on Levante. I don't think we need to change something here.

This is what we are using for CMIP6:

CMIP6: [
    /work/ik1017/CMIP6/data/CMIP6,  # "official" DKRZ dir
    /work/bd0854/DATA/ESMValTool2/download/CMIP6,  # our "global" download dir
]

What we could/should change though, is the DRS DKRZ so it includes the facet product (like ESGF), see here and/or make DRS configurable by path (ESMValGroup/ESMValCore#129).

[valeriupredoi]

cheers Manu! Well, I had to use multiple dirs for CMIP5/6 last week - how about we give it a short discussion at our next Techy call? I'll add this to the points to discuss list, if that's OK with you 🍺

[bouweandela]

Another step that is missing is that you need to merge the release branch back into the main branch without squashing after making the release off of it, else the version number in the main branch will be wrong after the release. See ESMValGroup/ESMValCore#1773.

[valeriupredoi]

Another step that is missing is that you need to merge the release branch back into the main branch without squashing after making the release off of it, else the version number in the main branch will be wrong after the release. See ESMValGroup/ESMValCore#1773.

yes, the absolute last step 🌜

[schlunma]

I think an alternative approach would be to not push to the release branch directly (e.g., by increasing the version number in a PR to main before branching off the release branch). This has been done for v2.5 and v2.6 and didn't mess up the version numbers as far is I know.

[valeriupredoi]

Yeh, but that's high risk since the actual release might be late - we'd start getting questions from power users about version mismatches. Which we never get anyway 😆

[bouweandela]

If we follow the procedure that any changes first need to be merged into main and if they are considered critical for the release they can be cherry-picked into the release branch, as described in our making a release documentation, this should also work fine without merging back the release branch into the main branch. Unless changes to main would cause a merge conflict in the release branch of course.