Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BUG: Spark Infrastructure chart uses a hard-coded aiSSEMBLE version. #373

Closed
ewilkins-csi opened this issue Sep 26, 2024 · 2 comments
Closed
Assignees
Labels
bug Something isn't working slacktask Small ticket done in down time
Milestone

Comments

@ewilkins-csi
Copy link
Contributor

ewilkins-csi commented Sep 26, 2024

Description

The spark infrastructure chart was updated to use the aissemble-spark image, but the tag is hard-coded. We should apply the same technique we're using in other charts to set the image tag from the Maven project directly so that the two values can't fall out of sync as easily.

Steps to Reproduce

  1. Check out aissemble
  2. Add a version tag to _extensions/extensions-helm/extensions-helm-spark-infrastrucutre/pom.xml with some version number different than the current version of aiSSEMBLE. E.g.:
    <version>2.5.0</version>
  3. Build the spark-infrastructure helm charts
    mvn clean install -pl :extensions-helm-spark-infrastrucutre -amd
  4. Check the value of deployment.image.tag in the values.yaml files at the following locations:
    • extensions/extensions-helm/extensions-helm-spark-infrastructure/aissemble-spark-history-chart/values.yaml
    • extensions/extensions-helm/extensions-helm-spark-infrastructure/aissemble-thrift-server-chart/values.yaml

Expected Result

The tag is set to the same version as the POM.

Actual Result:

The tag is hard-coded to 1.10.0-SNAPSHOT.

Test Steps

  1. Build the charts from source:
    mvn clean install -pl :aissemble-thrift-server-chart,:aissemble-spark-history-chart,:aissemble-quarkus-chart -Dmaven.build.cache.skipCache
  2. Use the helm template command on the built charts and ensure spec.template.spec.containers[0].image uses the 1.10.0-SNAPSHOT tag
    helm template ./extensions/extensions-helm/extensions-helm-spark-infrastructure/aissemble-thrift-server-chart/target/helm/repo/aissemble-thrift-server-chart-1.10.0-SNAPSHOT.tgz -s templates/deployment.yaml
    helm template ./extensions/extensions-helm/extensions-helm-spark-infrastructure/aissemble-spark-history-chart/target/helm/repo/aissemble-spark-history-chart-1.10.0-SNAPSHOT.tgz -s templates/deployment.yaml
    helm template ./extensions/extensions-helm/aissemble-quarkus-chart/target/helm/repo/aissemble-quarkus-chart-1.10.0-SNAPSHOT.tgz -s templates/deployment.yaml
  3. Update each of the following POM files to add <version>1.11.0</version>:
    • extensions/extensions-helm/extensions-helm-spark-infrastructure/aissemble-thrift-server-chart/pom.xml
    • extensions/extensions-helm/extensions-helm-spark-infrastructure/aissemble-spark-history-chart/pom.xml
    • extensions/extensions-helm/aissemble-quarkus-chart/pom.xml
  4. Build the charts
    mvn clean install -pl :aissemble-thrift-server-chart,:aissemble-spark-history-chart,:aissemble-quarkus-chart -Dmaven.build.cache.skipCache
  5. Use the helm template command on the built charts and ensure the container images are set to the 1.11.0 tag
    helm template ./extensions/extensions-helm/extensions-helm-spark-infrastructure/aissemble-thrift-server-chart/target/helm/repo/aissemble-thrift-server-chart-1.11.0.tgz -s templates/deployment.yaml
    helm template ./extensions/extensions-helm/extensions-helm-spark-infrastructure/aissemble-spark-history-chart/target/helm/repo/aissemble-spark-history-chart-1.11.0.tgz -s templates/deployment.yaml
    helm template ./extensions/extensions-helm/aissemble-quarkus-chart/target/helm/repo/aissemble-quarkus-chart-1.11.0.tgz -s templates/deployment.yaml
@ewilkins-csi ewilkins-csi added the bug Something isn't working label Sep 26, 2024
@ewilkins-csi ewilkins-csi changed the title BUG: Spark Infrastructure chart uses a different version of Spark than Spark Operator/Spark Worker BUG: Spark Infrastructure chart uses a hard-coded aiSSEMBLE version. Oct 15, 2024
@ewilkins-csi ewilkins-csi added the slacktask Small ticket done in down time label Nov 4, 2024
@ewilkins-csi ewilkins-csi self-assigned this Nov 4, 2024
@ewilkins-csi ewilkins-csi added this to the 1.10.0 milestone Nov 4, 2024
@ewilkins-csi
Copy link
Contributor Author

DoD/OTS with @carter-cundiff

ewilkins-csi added a commit that referenced this issue Nov 5, 2024
[#373] use .Chart.AppVersion for image tags in custom charts
ewilkins-csi added a commit that referenced this issue Nov 5, 2024
Using the `.Chart.AppVersion` for image tags exposed a bug that was
introduced when we moved to opensource.  Previously we were overwriting
path dependencies in helm charts during the release process via the
`release` profile. After open-sourcing our release profile name changed
from `release` to `ossrh-release`, but we didn't update it in the
`extensions-helm` POM.  However, it was incorrect to only rewrite
dependencies during release anyway, as the chart dependencies should be
rewritten _any_ time we're deploying (including snapshots). So instead
we moved the functionality to the `ci` profile (which is also active
during release.)

Also found an issue with release-prepare introduced by #451 preventing
the version update process due to unresolved property.
ewilkins-csi added a commit that referenced this issue Nov 5, 2024
@carter-cundiff
Copy link
Contributor

Testing passed:
Original version:
image
image
image

New version:
image
image
image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working slacktask Small ticket done in down time
Projects
None yet
Development

No branches or pull requests

2 participants