Investigate ephemeral MCs for MC app testing

[yulianedyalkova]

Desired outcome is that we know how an ephemeral MC looks like and how to create one.

Acceptance criteria:

• It's clear what configuration needs to be pre-setup for an ephemeral MCs
• There is a clearly defined and tested process for creating an ephemeral MCs
• There is sufficient documentation explaining how to create an ephemeral MC

[AverageMarcus]

So I don't forget, the new ephemeral MCs should go into AWS account 905418424421

[AverageMarcus]

For the name of the MCs we decided to use gawstest1 => gawstest10 (or however many we configure)

[AverageMarcus]

An update on where things are at as I plan to park this issue until I'm back from KubeCon / KCD Denmark:

• The docs in mc-bootstrap have been updated to reflect what is currently needed when setting up a new ephemeral CAPA MC.
• So far, I have created the initial config (via make create-config) for a new MC named gawstest1. This has setup various external resources (dex, etc.), relevant items in LastPass and created all the new auto-branches in the various repos.
• I have attempted to create the new MC using this WiP PR as a trigger but unfortunately hit issues that prevented the MC from coming up.

Full details of the issue experienced can be found in this Slack Thread but to summarise:

The new AWS that we are using for these ephemeral MCs doesn't have the root hosted zone setup in Route53 (as this is already setup in the other account that goten is currently using). It's currently not clear if it would be best to have a new hosted zone per AWS account or if aws-resolver-rules-operator should be updated to support cross-account hosted zones. Phoenix are going to discuss this in the team and report back here what they decide.

[fiunchinho]

After discussing in Phoenix stand up the problem with aws-resolver-rules-oeprator and the AWS accounts, we decided that the best approach would be to use a different hosted zone base domain for MCs using different AWS Accounts.

[AverageMarcus]

@JosephSalisbury 👆 FYI as it effects the MC moves we're wanting to do and might need to be included in some of your issues about account separation.

Any suggestions for what we should use for the new base domain?

@fiunchinho do you know if we have docs anywhere on how we set up a new hosted zone for giant swarm addresses? Or should I maybe speak with an SRE?

[fiunchinho]

@fiunchinho do you know if we have docs anywhere on how we set up a new hosted zone for giant swarm addresses? Or should I maybe speak with an SRE?

I added this, I hope that helps https://github.com/giantswarm/mc-bootstrap/pull/1062

[AverageMarcus]

@fiunchinho do you know if we have docs anywhere on how we set up a new hosted zone for giant swarm addresses? Or should I maybe speak with an SRE?

I added this, I hope that helps giantswarm/mc-bootstrap#1062

🫶 You spoil me! Thats great! Thank you!