[Meta] Further deisrel enhancements

[vdice]

Top priority items

To reduce current release toolset down from 2 (deis-workflow-group and deisrel) to 1 (deisrel), in priority order:

• deisrel docker retag: [Meta] Cover step 4 (Docker image tagging and pushing) #58, PR: feat(main.go,actions): create a docker image tagging command #89
• deisrel create (release) branches: add create-branches feature #99
• deisrel create releases: add create-releases feature #98
• get list of repos/components from file (Make deisrel take a list of components & repos #115)
• retrieve canonical generate_params.toml files: Retrieve canonical generate_params.toml via GH API #11
• templatize chart metadata: Templatize "metadata" files in helm charts #103

Next in priority as bandwidth allows:

• jenkins job for component release build/deploy: create component release job jenkins-jobs#110
• deisrel move to server proposal: Proposal: move deisrel to the server #83

These tickets/enhancements would feed into the below proposed release process, and the documentation should be updated as new commands get implemented.

Release Steps Using deisrel

1. create all release branches for all repos (add create-branches feature #99)
   • note: need a --ref flag for this, so we can do unscheduled/patch releases from a previous tag and not master
   • this would kick off the jenkins job detailed above (create component release job jenkins-jobs#110), building/pushing docker images to the quay.io/deisci org.
2. copy helm chart from workflow-dev and update critical files (deisrel helm-stage)
3. change release branch on deis/charts repo to have new generate_params.toml, README, etc... (created by step 2)
4. e2e & manual testing on chart created in step 3
   • if we find an issue in a component, fix in the a branch on the component's repo, PR it, merge it to master, wait for CI to build a new image, then change the docker tag SHA in the deis/charts repo
   • PROBLEM: we don't have a code freeze. if person A commits an unrelated change to master, then person B commits the fix, we would have to do 1 of the following:
     • cherry pick the fix into the release PR - this is a problem b/c the docker image tagged with the master SHA has more than what we cherry picked
     • rebase everything in master up to person B's commit back into the release PR - this is a problem because the release PR has more than just the fix in it
   • SOLUTION: do cherry pick, and have a pipeline that builds, docker-builds and docker-pushes artifacts from release branches
   • Quick Fix: after the cherry pick, manually kick off the appropriate job (https://ci.deis.io/job/builder for example) for the repo that the cherry pick happened on, overriding the sha1 job parameter to match the commit on the release branch
   • Better Fix: implement create component release job jenkins-jobs#110 above that looks at all branches across all repos, and runs a build/docker-build/docker-push pipeline on commits to all branches where branch name matches release-*
5. Merge the deis/charts release PR
   • At this point, the workflow artifact is released. The remaining steps are to release the individual component artifacts (repo tags & docker images)
6. Create releases for all release branches (add create-releases feature #98)
   • implicitly creates changelog & tags
   • not merging any of the release branches back into master (except for the deis/charts one, which was already merged)
   • output from this command is aggregated (global) changelog that should be copied and sent to Jason, marketing, etc... (in future, would be ideal to have deisrel automatically create gist and/or post to an agreed-upon spot)
   • Long-term: move functionality to server (Proposal: move deisrel to the server #83)
7. Re-tag all docker images to the pretty tag ([Meta] Cover step 4 (Docker image tagging and pushing) #58)
   • for example, quay.io/deisci/builder:git-12345 (tag SHA) --> quay.io/deis/builder:v2.1.0 (tag name)
   • Quick fix: re-purpose deisrel docker retag to do this, instead of using the webhook server for now (in progress at feat(main.go,actions): create a docker image tagging command #89)
   • Better fix: move functionality to server (Proposal: move deisrel to the server #83)

[arschles]

This issue is on hold until we finalize our release process discussion

[vdice]

This ticket is no longer valid/relevant now that we are releasing components individually.