Feature: 1.9.3 Patch Release

[ewilkins-csi]

Description

A clear and concise description of the new feature. It can include what the problem is and what you want to happen, and is preferably focused on the business context or how this impacts downstream users.

DOD

Issues to potentially patch:

• Required patches:
  • Thrift server connection is unstable (found via Data Access)
  • The SparkOperator ArgoCD Application should have the ServerSideApply option by default. We mentioned this as a manual workaround for 1.8, and adjusted the above migration to account for it, but we never added it to the actual template so new 1.9 projects would still be broken. (See https://github.com/boozallen/aissemble/pull/416/files#r1804830533)
• Nice to have:
  • Spark Infrastructure charts use .Release.Name instead of .Values.app.name like our other charts (See ca7b34f)
    • If we need to drop this, make sure we add upgrade info around updating the Tiltfile helm arguments to include release name
  • argocd-template-sync-policy-configuration-migration adds migration instructions to files that don't need it when the entire sync policy section is in the template wholesale - e.g. after the migration runs a second time.
    • This may be specific to the project that raised the issue, and not worth fixing broadly if the logic proves difficult
  • Instructions for updating custom Spark Application values files (See https://github.com/boozallen/aissemble/pull/409/files#diff-5f8520547da4ccaf2b5d54fe85ec2567b904ee5cf79af0318a600614765cc013)

Patch release DOD:

• Release artifacts published
• Release notes published
• Main 1.9 release notes updated to patch version
• Docs published

Test Strategy/Script

1. Create a new project from 1.9.3 (SNAPSHOT for OTS)

   mvn archetype:generate -DarchetypeGroupId=com.boozallen.aissemble \
                          -DarchetypeArtifactId=foundation-archetype \
                          -DarchetypeVersion=1.9.3 \
                          -DgroupId=org.test \
                          -Dpackage=org.test \
                          -DprojectGitUrl=test.org/issue-419.git \
                          -DprojectName="Issue 419 Test" \
                          -DartifactId=issue-419 \
   && cd issue-419

2. Add the attached SparkPipeline.json to issue-419-pipeline-models/src/main/resources/pipelines
3. Fully generate the project until no more manual actions are printed
4. Verify that issue-419-deploy/src/main/resources/templates/spark-operator.yaml has the value ServerSideApply under spec.syncPolicy.syncOptions
5. Open any other YAML file under issue-419-deploy/src/main/resources/templates and verify there is no WARNING comment block with manual migration instructions at the end of the file
6. Build the project one more time without the cache and follow the last manual action

   mvn clean install -Dmaven.build.cache.skipCache

7. Deploy the project

   tilt up; tilt down

8. Verify there are resources in Tilt named spark-history and thrift-server
9. Wait for the thrift-server resource to start. (Note that it will fail until hive-metastore-service is running which will in turn fail until hive-metastore-db is running.)
10. Copy the pod name for the spark-operator resource in Tilt and exec into the pod

    kubectl exec -it <POD-ID> -- bash

11. Run the beeline tool

    ../bin/beeline

12. Verify you can connect to the thrift server. Use a blank username/password by simply hitting Repeat the connection command 8+ times to ensure consistent connection every time.

    !connect jdbc:hive2://thrift-server:10001/default;transportMode=http;httpPath=cliservice;user=;password=

[ewilkins-csi]

DoD @csun-cpointe and @carter-cundiff

[ewilkins-csi]

OTS of patch changes completed with @jacksondelametter

[csun-cpointe]

Final test passed!
**Verify ServerSideApply under spec.syncPolicy.syncOptions

Verify can repeatedly connect to the thrift server