Multi-distro support (Zarf Native Apply Work)

[jeff-mccoy]

Changelog

• add arm64/generic arch support
• adds in-memory embedded registry to serve registry image from "seed" config spec
• add local-distro image injection support
• handles helm charts install/upgrade/rollback/uninstall via helm directly
• manages raw manifests as zarf-generated helm charts for helm-native management
• add tunnel capability to access zarf endpoints without ingress config
• auto-inject imagePullSecret into any service account zarf touches and any namespace default service account
• auto-inject registry pull credentials into all namespaces for use by service accounts
• allow multiple helm charts to be installed in the same namespace without conflict
• flattens image collection to single tarball
• remove root / linux requirements for zarf deployments
• remove k3s install requirement
• remove traefik component requirement
• add K8s distro-detection and distro-specific seed image behaviors
• document local registry per https://github.com/kubernetes/enhancements/tree/master/keps/sig-cluster-lifecycle/generic/1755-communicating-a-local-registry
• reorganize logging, remove logrus, replace with pterm, add progress cli feedback
• add support for stacking helm values files
• normalize log levels: info, warn, debug, trace
• update go mod deps
• move zarf state to a k8s secret in the zarf namespace
• support zarf destroy for 3rd-party clusters
• move crane image cache to sub folder under home
• add support for multiple users with htpasswd
• add path key for git-based helm charts
• remove need for PKI prompts on default init
• add storage class tracking in zarf state for multi-distro compatibility
• reduce code for init/deploy to the same function call
• add command line aliases for the cli
• vendor k9s into zarf to support multi-arch/multi-os
• make more games for the game example to demonstrate multiple zarf connect in the same component
• add zarf prepare find-images to lookup images for a zarf config to help with package creation
• add zarf destroy --remove-components for removing zarf-installed charts outside of the zarf ns
• add zarf connect 🦄 combined with a new svc label, zarf.dev/connect-name=🦄 to make dynamic app tunnels

Fixes #140
Fixes #2
Fixes #16
Fixes #109
Fixes #17
Fixes #35
Fixes #36

Manual Validation of Examples:

Some of the key examples should be confirmed to work both in the existing "appliance mode" and using this new external Kubernetes installation mode. Note that the external Kubernetes cluster currently has to be a local cluster, so validation of the new capabilities should be done via Kind. The manual tests should include the following steps:

External Cluster

1. Show output of kubectl get all -A
2. Create and deploy the Zarf package for the example using the documentation
3. Verify all components up and healthy
4. Verify ingresses/UIs up and healthy
5. Teardown/Destroy installation
6. Ensure output of kubectl get all -A matches output from 1.

Appliance Mode

1. Create and deploy the Zarf package for the example using the documentation
2. Verify all components up and healthy
3. Verify ingresses/UIs up and healthy
4. Teardown/Destroy installation

• Big Bang
  • Update Docs (@RothAndrew)
  • KIND installation successful (@RothAndrew)
  • Appliance installation successful (zarf init w/k3s install) (@RothAndrew)
  • Revert back to "correct" way to do GitOps (potentially future backlog item) (Update the Big Bang example to do GitOps "correctly" #268)
• Data Injection
  • Update Docs
  • KIND installation successful
• Game (@RothAndrew) (Multi-Distro Support - Update Docs #289)
  • Update Docs (@RothAndrew)
  • Add asciinema graphics back in once they get updated (future backlog item) (@RothAndrew) (Add Asciinema recordings back in #286)
  • KIND installation successful
• Gitops-data (@RothAndrew)
  • Update Docs (@RothAndrew)
  • KIND installation successful
• Postgres-operator (@RothAndrew)
  • Update Docs (@RothAndrew)
  • KIND installation successful
• single-big-bang-package
  • Update Docs (@jeff-mccoy)
  • KIND installation successful
• tiny-kafka
  • Update Docs (@jeff-mccoy)
  • KIND installation successful

[jeff-mccoy]

/test all

[matt-strong]

pulling-images.mov

Not a huge deal, but on zarf package create for big-bang the pulling images step always goes up to 63% then immediately completes.

[jeff-mccoy]

pulling-images.mov
Not a huge deal, but on zarf package create for big-bang the pulling images step always goes up to 63% then immediately completes.

yeah I've struggled with how to handle this--that's actually the caching system, it can't really do progress. If you have time could you delete ~/.zarf-image-cache/ and try again?

[matt-strong]

Weird, seeing the same thing with an empty image cache folder (on a m1 mac mini).

[jeff-mccoy]

thanks for the very detailed PR updates @runyontr, looks way better with all that data

[YrrepNoj]

/test all

[jeff-mccoy]

/test all

[jeff-mccoy]

/test all

.

[jeff-mccoy]

/test all

[jeff-mccoy]

Data Injection example

[jeff-mccoy]

omg...I think we're there.......