add manual e2e testing procedure

.github/pull_request_template.md

@@ -0,0 +1,7 @@
+### What this PR does / why we need it
+
+
+### Checklist
+
+- [ ] Update changelog in CHANGELOG.md.
+- [ ] Follow deployment test procedure in the tests/manual_e2e directory and have a working branch.

CHANGELOG.md

@@ -7,6 +7,11 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+### Added
+
+- Add "manual e2e" testing procedure.
+- Add PR message template referring to the manual testing procedure.
+
 ## [0.23.0] - 2024-09-10
 
 ### Added

tests/manual_e2e/README.md

@@ -0,0 +1,17 @@
+# Manual e2e testing
+
+As of now, the [apptest-framework](https://github.com/giantswarm/apptest-framework) used for automated e2e testiong doesn't support MC-only apps. Hence the manual procedure described here in order to ensure that the app works as expected in a Giant Swarm environment.
+
+## Procedure
+
+Before proceeding to any kind of test, you'll first have to deploy your custom branch app's version into a testing installation. Don't forget to suspend flux reconciliation for this app during the whole testing process. See [here](https://intranet.giantswarm.io/docs/dev-and-releng/flux/suspending-flux/#how-to-be-more-granular--subtle-with-suspending-resources-and-why-be-careful-with-this) for details on how to evict an app from flux's reconciliation.
+
+Once that's done, please run the `basic_checks.sh` script which will check that the app is in `deployed` state and that the `loki-canary` component is enabled.
+
+Once your app is correctly deployed and configured :
+
+- Let it run for a bit, like 10min or more.
+- Inspect the `Loki / Canary` dasboard which will give information on Loki's overall health.
+- If everything appears to be fine, then you can revert the flux's evicting procedure that you did and let it reconcile to its original version.
+
+Congratulations, ou have completed the manual e2e testing procedure ! Your PR is now ready to be merged.

tests/manual_e2e/basic_checks.sh

@@ -0,0 +1,40 @@
+#!/bin/bash
+
+# Helper function - prints an error message and exits
+exit_error() {
+    echo "Error: $*"
+    exit 1
+}
+
+echo "Checking if loki app is in deployed state"
+
+appStatus=$(kubectl get app -n giantswarm loki -o yaml | yq .status.release.status)
+
+[[ "$appStatus" != "deployed" ]] \
+  && exit_error "loki app is not in deployed state. Please fix the app before retrying"
+
+echo "loki app is indeed in deployed state"
+echo "Checking if loki-canary is enabled"
+
+canary=$(kubectl get cm -n giantswarm loki-chart-values -oyaml | yq .data.values | yq .loki.lokiCanary.enabled)
+
+[[ "$canary" != "true" ]] \
+  && exit_error "loki-canary is not enabled. Please enable it before retrying"
+
+echo "loki-canary is indeed enabled"
+echo "Checking if all loki pods are up and running"
+
+lokiComponents=("read" "write" "backend" "gateway" "canary")
+
+for component in "${lokiComponents[@]}"; do
+  podStatus=$(kubectl get pods -n loki -l app.kubernetes.io/name=loki,app.kubernetes.io/component=$component -o yaml | yq .items[].status.phase)
+
+  [[ -z "$podStatus" ]] && exit_error "No $component pods found. Please check it before retrying"
+
+  for status in $podStatus; do
+    [[ "$status" != "Running" ]] \
+      && exit_error "A $component pod is not running. Please check it before retrying"
+  done
+done
+
+echo "All loki pods are up and running"