Big Bang Core Example

[RothAndrew]

Note: I'm mostly using this branch to learn how to use Zarf. Some of what is here may not be appropriate to merge into the trunk. No way man, we're doing this for realz!

Objectives

• Figure out how to derive a list of all git repos and docker images that will be needed to deploy Big Bang Core
• Build a zarf.yaml that packages all of them and successfully loads them into Gitea and the private registry when deployed
• Update the vagrantfile to provide a suitable VM for Big Bang's fat ass
• ???
• Profit

What/Why

• Add new Example - Big Bang Core! By deploying BB to a utility cluster, you get an easy but very powerful demo with little fuss!
  • New zarf.yaml, some manifests and kustomizations, changes to a makefile, stuff like that
• Update the Vagrantfile by adding a new VM target called bigbangdemo that provisions more CPU, more memory, and more drive space. Other VM targets should be mostly unaffected, except for:
• Update the port forwarding of the VMs to directly forward 80 and 443 to 80 and 443 instead of 80 to 8080 and 443 to 8443, so that I could also forward 8080 and 8443 without weirdness. Note that this will change the functionality of the other VM targets by making Gitea accessible from the host on https://localhost:443 instead of https://localhost:8443, but IMO that's better any way?
• Add more domains to /etc/hosts in the Vagrantfile for better "simulated airgap"
• Add new file registries.yaml that will get placed in /etc/rancher/k3s/registries.yaml so that K3s will automatically use zarf.localhost as a registry mirror. I'm not quite as sure about this approach. It was definitely needed for my BB demo but I'm not sure whether it is appropriate to put it in the Zarf Init package like it is. Feel free to 💩 on it!
• Add .editorconfig file Moved to separate PR Add .editorconfig #37
• Minor housekeeping

Followup

• Enable the NetworkPolicies - They got disabled to resolve an issue connecting to the k8s cluster API server, which is fine for a demo but unacceptable in production

Closes #8

[jeff-mccoy]

I think we should have all big bang-specific stuff in that example folder, including the vagrantfile, is that doable?

[RothAndrew]

I thought about that, but it would mean doing a different make target, which felt less user friendly. I don't have a strong opinion either way.

[jeff-mccoy]

mind if I play with it a little?

[RothAndrew]

Nope, go for it.

[jeff-mccoy]

Thanks for working this @RothAndrew and getting it close to the finish line, I couldn't get it to work, but it's close. Few observations I think we need to think through some more:

• How is mirroring/proxying being done? I'm getting image pull backoff as the Big Bang images are still pointing to repo1 instead of the local registry server and there is no mirror config set that I see okay I see where this is now, this is useful, probably should just cherrypick this and cut a release with that now to 0.10.3. Thoughts @RothAndrew?
• I think using the experimental resize for vagrant is better (and transparent) than forcing a plugin to be installed
• What's going on with the ports? The vagrantfile had 443->443 mapped but your screenshots showed 8443? Also, < 1024 ports requires root and breaks on my (most ppls) machines. It's not recommended by Virtualbox for that reason.
• I probably need to look at order of precedence for applying things as it takes flux an extra long time to do initial reconciliation since it's manifests are applied but the remote assets take a long time to load.
• I don't think examples should assume someone can/knows how to build the CLI or will cut a release themselves, so I changed the makefile to instead pull the latest tag down (this will need to pull a real release once we finish migration)
• Also stripped down the vagrant file in the examples folder to only be for Ubuntu, we don't really need to show ppl the examples in every OS, this is more about showing the examples than Zarf running on different platforms

Okay that's a lot, I'm going to push my changes here, feel free to throw flaming darts my way when you get back online.

[jeff-mccoy]

Relevant screenshots for the above comments.

[jeff-mccoy]

It's probably also worth discussing the use if interal K8s routes for git and zarf.localhost for the registry mirror. Ideally they'd both use HTTPS and the ingress consistently.

[RothAndrew]

TL/DR: Looks good, couple of tweaks and a bit more testing on my end then let's merge.

How is mirroring/proxying being done? I'm getting image pull backoff as the Big Bang images are still pointing to repo1 instead of the local registry server and there is no mirror config set that I see okay I see where this is now, this is useful, probably should just cherrypick this and cut a release with that now to 0.10.3. Thoughts @RothAndrew?

Works for me. I'll work on that here in a few minutes.

I think using the experimental resize for vagrant is better (and transparent) than forcing a plugin to be installed

I was having a hard time getting it to work, but it may have been the growpart stuff that I was missing. I'll look at it again.

What's going on with the ports? The vagrantfile had 443->443 mapped but your screenshots showed 8443? Also, < 1024 ports requires root and breaks on my (most ppls) machines. It's not recommended by Virtualbox for that reason.

I didn't think about needing root for <1024. The screenshots were showing services exposed by Istio, which is running on 8443. I can change it to forward Traefik to 8080/8443 and Istio to 9080/9443. So, to hit Gitea you'll go to https://localhost:8443, to hit one of the services exposed by Istio, you'll go to https://*.bigbang.dev:9443. The way it is now in examples/Vagrantfile won't expose any of the Istio-exposed services at all, just the Traefik stuff.

I probably need to look at order of precedence for applying things as it takes flux an extra long time to do initial reconciliation since it's manifests are applied but the remote assets take a long time to load.

I wouldn't worry too much about it. It feels like it is taking the same amount of time as any other BB deployment.

I don't think examples should assume someone can/knows how to build the CLI or will cut a release themselves, so I changed the makefile to instead pull the latest tag down (this will need to pull a real release once we finish migration)

Works for me. It might need some tweaking for ease of use but it works for now.

Also stripped down the vagrant file in the examples folder to only be for Ubuntu, we don't really need to show ppl the examples in every OS, this is more about showing the examples than Zarf running on different platforms

Agreed.

[jeff-mccoy]

Copy all thanks. Sounds like the only outstanding work now is the port mapping, I’ll leave it up to you, but we do need ports above 1024. This is fantastic work @RothAndrew. 🦄

[RothAndrew]

Copy all thanks. Sounds like the only outstanding work now is the port mapping, I’ll leave it up to you, but we do need ports above 1024. This is fantastic work @RothAndrew. 🦄

Thanks! Working on it today between meetings.

[RothAndrew]

• Updates to makefile to support building from source as well as downloading latest published artifacts
• Add forwarded ports 8080->9080 and 8443->9443 for Istio's ingressgateway
• minor housekeeping

I'm happy, think we're ready to merge.

[RothAndrew]

Moving back to Draft for right now until after my demo today. I fixed the tiny-kafka example but it should go in its own PR

[RothAndrew]

It's probably also worth discussing the use if interal K8s routes for git and zarf.localhost for the registry mirror. Ideally they'd both use HTTPS and the ingress consistently.

Created #43 to address this later

[RothAndrew]

Pulled out the tiny-kafka example fix, ready for merge again

[RothAndrew]

@jeff-mccoy do you have any objections to merging this now? It's starting to get a little long in the tooth

[jeff-mccoy]

LGTM, great work