Doc release update process

The ACRN release process starts when the code is frozen on the candidate release branch, such as release_3.1. Documentation changes continue on the master branch while developers are validating and testing the release candidate. Once the release candidate branch is declared ready, we need to propagate all the documentation changes that were done on the master branch back into this release candidate branch as well.

In the past, doc changes were cherry-picked one-by-one from the master branch to the release candidate branch, but we've simplified this process (and reduced the chance of missing a documentation change) by essentially merging all the documentation changes made on the master branch since the release candidate branch was made. We do this by making a working directory (outside the local repo) with content from the master branch, checking out the release candidate branch, and then copying that working directory content onto the checked-out release candidate to create a single pull-request with all the changes. The important working assumption here is that all the doc changes done on master after the release branch was created are intended for master and the release branch. Any doc changes not intended for both can be removed before this PR is merged onto the release branch (by restoring the original doc from the release branch).

Assumptions:

• follow the Doc Build instructions to set up required directories and tools
• the cloned acrn-hypervisor repo is in ~/acrn-project/acrn-hypervisor
• origin points to a cloned personal git repo
• upstream points to the projectacrn/acrn-hypervisor.git repo

~/projectacrn/acrn-hypervisor/doc$ git remote -v
origin  https://github.com/dbkinder/acrn-hypervisor.git (fetch)
origin  https://github.com/dbkinder/acrn-hypervisor.git (push)
upstream        https://github.com/projectacrn/acrn-hypervisor.git (fetch)
upstream        https://github.com/projectacrn/acrn-hypervisor.git (push)

Here's the process:

# define the release number and branch name for the PR (normally they'd be the same value,
# but if you run these steps more than once for a release, you'll want a new branch name
# for subsequent runs, so change that relbranch name here  (e.g., 3.1a for the second run)
rel=3.1
relbranch=3.1

# create a "master" working directory in your home folder
rm -rf ~/acrn-master && mkdir ~/acrn-master
cd ~/projectacrn/acrn-hypervisor/doc

# get the latest upstream/master branch content 
git checkout master
git fetch upstream
git merge upstream/master
git push origin master

# clean out doc build artifacts
make clean

# copy all the doc directory content to our master working directory
cp -r . ~/acrn-master/doc

# copy any docs in the hypervisor root folder (e.g., CONTRIBUTING.rst)
cp ../*.rst ~/acrn-master

# and copy docs from the misc directory too
cd ../misc
mkdir ~/acrn-master/misc

# the --parents option maintains the directory structure in the destination directory
# Assumption is there are only .rst files of interest, no images in the misc folders
cp --parents *.rst ~/acrn-master/misc
cp --parents */*.rst ~/acrn-master/misc
cp --parents */*/*.rst ~/acrn-master/misc
cp --parents */*/*/*.rst ~/acrn-master/misc
cp --parents */*/*/*/*.rst ~/acrn-master/misc

# until that last copy fails.
 
# The schema files contain configuration option documentation
# Starting with v2.7, we no longer copy the .xsd files in this process.  Instead
# any changes made after the release branch is created must be duplicated on master
# and the release branch.  (Left here as a comment in case we change our mind.)
# cp --parents config_tools/schema/*.xsd ~/acrn-master/misc

cd ..

# get the release candidate doc contents and make a branch for updating
git checkout upstream/release_$rel
git checkout -b update-docs-$relbranch

# copy all the master working directory doc contents over the release candidate contents
cp -r ~/acrn-master/* .

#-------PAUSE HERE--------------------------------------
# All the above steps work reliably and can't really cause any issues

# check for files that were deleted in the master branch but are still on the release_$rel branch
# (typically orphaned images from docs that were deleted)

diff -rq doc ~/acrn-master/doc | grep "Only in doc"
# use this to actually delete them:     
rm -rf `diff -rq doc ~/acrn-master/doc | tr -d "\:" | grep "Only in doc" | awk '{print $3 "/" $4}'`

# something similar for the misc content, but we only want to look at .rst files (no .xsd since v2.7)
# diff -rq misc ~/acrn-master/misc | egrep "(\.rst|\.xsd)" | grep "Only in misc"
diff -rq misc ~/acrn-master/misc | egrep "\.rst" | grep "Only in misc"

# there shouldn't be any files listed here. If there are, look carefully at why

# do a "make html" now to verify CI doc build will work...
# we can't publish these until after we merge these changes to the release branch
# so we'll get the right last-modified date
cd doc
make clean
make DOC_TAG=release RELEASE=$rel html 

# add changed staged files, commit, and do the PR
cd ..
git add .
git status 
# verify all staged files are added (from the /doc and /misc directories)

git commit -s
# As of release 2.7, we no longer copy over .xsd file changes as part of the doc build, so this Tracked-On is no longer needed.
# in the commit message add "Tracked-On: #5692" to allow the .xsd file changes to get through CI checks
# otherwise we'll get a CI error because there are non-doc changes. 

git push origin update-docs-$relbranch

From here, head over to https://github.com/projectacrn/acrn-hypervisor and create a PR, changing the base to be the release_$rel branch (not master). You should see only the changed documents listed as changed files, and there should be no merge conflicts. It's a good idea to walk through all the changes (or at least the changed files) to verify that no unintended doc changes snuck in. You might find a doc change on master got included that wasn't intended for the release branch.

(You can restore a doc to what was in the release_$rel branch using git checkout -- <filename>, then do the usual git add -a, git commit --amend, and git push -f origin update-docs-$relbranch to remove the changes done on the master branch.)

Submit the PR and wait for the CI system to pass before merging. Once merged, we can pull the updated release_$rel branch and publish the release documentation:

git checkout upstream/release_$rel
cd doc
make clean
make DOC_TAG=release RELEASE=$rel html 
make DOC_TAG=release RELEASE=$rel publish

Now we can head back to the master branch, and make the changes to incorporate the 2.7 release docs into the latest doc version status by editing the conf.py file, as documented in Document Versioning.

cd ~/projectacrn/acrn-hypervisor/doc

git checkout master
git checkout -b add-v$rel-menu

# edit the conf.py to add the new release docs to the menu choice after /latest/

git add .
git commit -s
git push origin add-v$rel-menu

# then build and publish the /latest documents (if the edit didn't cause an error)
make html
make publish

# head over to github and submit the PR to update the conf.py

And your done merging, building, and publishing the new release documents, and rebuilding and publishing /latest to include a link to the new release docs in the version menu.