Update Release Automation

[ckunki]

Currently, the ai-lab uses release-droid for releases which is not maintained anymore as mid of 2024 the Java developers switched release automation with project-keeper.

The current ticket therefore requests to update the release process and documentation.

Current Release Process

For releasing the AI-Lab the developers currently call release-droid (RD), see How to Release the AI-Lab in the Developer Guide.

RD then calls GitHub workflow release_droid_upload_github_release_assets.yml, which then calls

poetry run python3 -m exasol.ds.sandbox.main start-release-build \
  --upload-url "${{ github.event.inputs.upload_url }}" \
  --branch "$GITHUB_REF"

cli/commands/start_release_build.py calls lib/release_build/run_release_build.py which

• Extracts the numeric RELEASE_ID (datatype int) from the upload URL provided by RD with regular expression ^https://uploads.github.com/repos/([a-zA-Z0-9\-_/]+)/releases/([\d]+)/assets.
• Triggers an AWS CodeBuild.

The AWS CodeBuild template aws-code-build/ci/buildspec_release.yaml calls

poetry run python3 -m exasol.ds.sandbox.main update-release \
  --release-id "$RELEASE_ID" \
  --asset-id "$ASSET_ID"

lib/update_release/run_update_release.py finally

• From AWS retrieves the URLs of all assets created by the CodeBuild
• Writes an inventory linking to the assets into file artifacts.md
• Uploads this inventory to the GitHub release as asset and
• Adds the text content of the inventory to the comment/message/notes of the GitHub release.

Proposed Changes

• Enhance function create_release() in lib/github_release_access.py
  • Add parameter tag
  • Add parameter message-file for reading the release comment/message from specified file
• Add a CLI cli/commands/create_github_release.py command for
  • calling function create_release()
  • Accepting parameters --tag, --title, --message, --comment-file
  • Printing the RELEASE_ID for usage in a GitHub workflow
• cli/commands/start_release_build.py:
  • Add CLI parameter --release-id <int>
  • Forward RELEASE_ID to lib/release_build/run_release_build.py
• Calling _execute_release_build() using the RELEASE_ID directly without parsing it from the URL.
• Add a new GitHub workflow release.yml
  • Triggered on: push: tags: - '**'
  • For experimental purposes also by workflow_dispatch
  • Creating the release by calling the new CLI command create-github-release (see below)
  • And calling start-release-build with the obtained RELEASE_ID.

CLI command call

poetry run python3 -m exasol.ds.sandbox.main create-github-release \
  --tag ${GITHUB_REF_NAME} \
  --title ${GITHUB_REF_NAME} \
  --message-file doc/changes/changes_${GITHUB_REF_NAME}.md \

Proposed New Release Process

• The Developer calls PTB feature prepare-release, see instructions in the Developer Guide of SAPIPY
• The Developer creates and pushes a Git Tag which will trigger the new GitHub workflow release.yml:

TAG="${1}"
git tag "${TAG}"
git push origin "${TAG}"

[ckunki]

See also issues #295, #283

[ckunki]

Proposal from @tkilias : I mostly agree.

Except: create_github_release can be integrated into run_release_build as we don't need two separate CLIs.
This also removed the need in between parsing the ReleaseID from the output in order to pass it to the second call.