Controller doesnt reenque on instance group GC failure

[nikhiljindal]

Found while debugging kubemci failures.

Steps to repro:

• Create an MCI ingress using kubemci
• Delete the MCI using kubemci CLI

Expected: All GCP resources are deleted

Actual: Instance group is not deleted

Cause:
In kubemci we first delete the ingress and then delete the GCP resources (including backend service).
When ingress is deleted from cluster, ingress-gce controller tries to delete the corresponding instance group, but it fails since the backend service is still referring it.

Proposed fix:

• For now, I will update kubemci to delete GCP resources first so that the backend service is deleted by the time we delete the ingress.
• ingress-gce controller should be fixed to retry the deletion.

cc @bowei @nicksardo @MrHohn @freehan

[nikhiljindal]

Steps to repo without using kubemci:

• Create an ingress.
• Optional: Add annotation kubernetes.io/ingress.class: gce-multi-cluster so that it only creates instance group and not the full stack.
• Create a backend service pointing to the instance group for that ingress.
• Delete the ingress.

Note that the instance group will not be deleted even after you delete the backend service

[nikhiljindal]

Sent GoogleCloudPlatform/k8s-multicluster-ingress#169 to update kubemci

[nicksardo]

This is actually by design. ILBs use instance groups... Until we have a single controller responsible for managing the instance groups or we have a different set of instance groups for ILBs, I don't see this logic changing.

[rramkumar1]

/close