Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for LTS versions [DI-258] #808

Merged
merged 13 commits into from
Oct 17, 2024
Merged

Conversation

ldziedziul
Copy link
Contributor

@ldziedziul ldziedziul commented Oct 15, 2024

Fixes: https://hazelcast.atlassian.net/browse/DI-258

The previous LTS approach had an issue that it wouldn't work when we automatically rebuild previous LTS releases (due to CVE fixes etc). It would result with two concurrent LTS builds (e.g. 5.5 and 6.5) both overwriting latest-lts tag.

This PR changes the logic that before overwriting the tag we make sure that the current version is the newest LTS. It's done by looking for the latest released version containing .github/lts file.

Whenever a .github/lts file exists the workflow will check if we're building a latest LTS version and if so it will add latest-lts tags and its variants (e.g. latest-lts-jdk21,latest-lts-slim, latest-lts-slim-jdk21, etc)

For manual runs you need to use IS_LTS input. It will overwrite checking the existence of the file.

Additionally added DRY_RUN flag to manual runs to be able test the tag_image_push.yml workflow in more safer way, instead of actual push you'll see something like:

DRY RUN: Build and push for platforms linux/arm64,linux/amd64,linux/s390x,linux/ppc64le and tags: 5.5.1 5.5.1-jdk21 5.5 5.5-jdk21 5 5-jdk21 latest latest-jdk21 latest-lts latest-lts-jdk21

Test run: https://github.com/hazelcast/hazelcast-docker/actions/runs/11361921888/job/31602670125

@ldziedziul ldziedziul self-assigned this Oct 15, 2024
Adds support for latest-lts tags.

Whenever a `.github/lts` file exists the workflow will check if it is a latest version and if so will push `latest-lts` tags and its variants (e.g. `latest-lts-jdk21`,
`latest-lts-slim`, `latest-lts-slim-jdk21`, etc)

For manual runs you need to use `IS_LTS` input instead.
@ldziedziul ldziedziul marked this pull request as ready for review October 16, 2024 08:52
@ldziedziul ldziedziul requested a review from a team as a code owner October 16, 2024 08:52
@JackPGreen
Copy link
Collaborator

🙏 test coverage

@ldziedziul
Copy link
Contributor Author

🙏 test coverage

I've added tests for bash scripts, for workflow files themselves it's very hard to create automated tests

.github/scripts/version.functions.sh Outdated Show resolved Hide resolved
.github/scripts/version.functions.sh Show resolved Hide resolved
.github/scripts/version.functions.sh Outdated Show resolved Hide resolved
.github/scripts/version.functions.sh Show resolved Hide resolved
.github/scripts/version.functions.sh Outdated Show resolved Hide resolved
.github/workflows/tag_image_push.yml Show resolved Hide resolved
.github/workflows/tag_image_push.yml Outdated Show resolved Hide resolved
.github/workflows/tag_image_push_rhel.yml Outdated Show resolved Hide resolved
.github/workflows/tag_image_push.yml Outdated Show resolved Hide resolved
.github/workflows/tag_image_push_rhel.yml Outdated Show resolved Hide resolved
@nishaatr
Copy link
Contributor

nishaatr commented Oct 16, 2024

Just thought I mention the below as came to my mind and not saying to do it

I think different way to get current lts version could be to call
curl https://hub.docker.com/v2/repositories/hazelcast/hazelcast-enterprise/tags/?page_size=2000 -H 'Content-Type: application/json'

Then extract digest for latest-lts and find the release with the same digest. Personally this feels simpler and exacting than checking tags

That said, the current latest-lts digest is not the same as 5.5.1 and I would have thought would be. May be we ran the build afterwards for latest-lts or may be I am reading this wrong

Anyways, not too fussed here

PS: perhaps a different way to do this is for Release Pipeline to actually trigger the Action and say if its LTS or not - rather than triggering via tags and relying on files (including release_type). May be something for the future!

@ldziedziul
Copy link
Contributor Author

I think different way to get current lts version could be to call curl https://hub.docker.com/v2/repositories/hazelcast/hazelcast-enterprise/tags/?page_size=2000 -H 'Content-Type: application/json'

Then extract digest for latest-lts and find the release with the same digest. Personally this feels simpler and exacting than checking tags

It was my first idea for the solution, but it has few drawbacks:

  • we need to maintain API calls, check if API call structure hasn't changed
  • we need to support paging with curl calls as we can have more tags then page_size
  • using external system makes it slower
  • it's harder to test
  • rate-limiting can fail the build

That said, the current latest-lts digest is not the same as 5.5.1 and I would have thought would be. May be we ran the build afterwards for latest-lts or may be I am reading this wrong

Actually good finding. I've fixed it here: 6d42824

PS: perhaps a different way to do this is for Release Pipeline to actually trigger the Action and say if its LTS or not - rather than triggering via tags and relying on files (including release_type). May be something for the future!

yes, maybe.

BTW I think you mean say if it's latest-LTS or not but how would you auto-rebuild the image in case of system updates? I.e. how would you know that the new image has to be marked as latest-lts or not?

@nishaatr
Copy link
Contributor

It was my first idea for the solution, but it has few drawbacks:

Good points. not sure I agree with all of them but still you make a good case. Actually didn't know there was rate limit.

Actually good finding. I've fixed it here: 6d42824

great.

BTW I think you mean say if it's latest-LTS or not but how would you auto-rebuild the image in case of system updates? I.e. how would you know that the new image has to be marked as latest-lts or not?

Yes that's what I mean. Didn't give it much thought tbh. But you ask a good question. I was thinking that Release Pipeline will say if its LTS so that's easy. For system updates you then use REST API and no need for the file. But looks like REST API has its issues so we can stick with current file implementation.

@ldziedziul ldziedziul enabled auto-merge (squash) October 17, 2024 12:10
@ldziedziul ldziedziul merged commit b307ed2 into master Oct 17, 2024
35 checks passed
@ldziedziul ldziedziul deleted the add-lts-tag-support branch October 17, 2024 12:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants