Karpenter support

[drmorr0]

Right now we interact with Cluster Autoscaler via an externalgrpc cloud provider. It would be really great if we can also support Karpenter, since there is a lot of interest and excitement about this project right now. If we end up moving to KWOK (#70) we will just write a KWOK provider for karpenter, otherwise we might have to build something inside SimKube itself. This is a tracking issue for this project.

[Huang-Wei]

Awesome! would love to see it come into play.

cc @wzshiming

[wzshiming]

Awesome!

FYI: It looks like karpenter has been using kwok for performance testing.

[drmorr0]

The link has moved now that karpenter is a CNCF project: https://github.com/kubernetes-sigs/karpenter/tree/main/kwok

[drmorr0]

The kwok provider included with Karpenter only supports a hard-coded list of node types that don't actually correspond to any "real" node types. If we want to run a scheduling simulation with Karpenter in the same way that we do with Cluster Autoscaler, we somehow need to tell Karpenter what the "real" list of node types it can draw from is... One idea would be to just make Karpenter read from the kwok-provider-templates configmap that cluster autoscaler does, but I feel like this limits/doesn't provide a super fair comparison for Karpenter, since one of its selling points is being able to draw from any instance type that's available. It might be a good starting point, though.

Another point to consider (this could be an issue with Cluster Autoscaler, too, but probably more-so with Karpenter) is that SimKube and/or KWOK and/or the autoscaler needs to know something about the "simulated" availability of an instance type. Ideally we'd like to be able to simulate scenarios where the "best" or "desired" instance type is unavailable so that we can see what our autoscaler does in response. I'm not 100% sure how to implement this yet.

[drmorr0]

I'm going to attend the CA/Karpenter meeting on Monday Feb 4 to talk about this a bit more (WG agenda here)

[drmorr0]

And here's an issue I opened in the Karpenter repo: kubernetes-sigs/karpenter#987

[drmorr0]

The karpenter project now lets you define custom instance types for the KWOK provider, going to close this out.