Setting devWorkspace controllerImage has no effect for airgapped environment

[thenrich]

Describe the bug

Setting the following in the checluster CRD has no effect:

spec:
  devWorkspace:
    controllerImage: path/to/controller/image

The default devWorkspace controller image is still used.

Che version

7.44@latest

Steps to reproduce

1. Deploy Che with chectl server:deploy -a operator -b my.domain.org -p k8s --skip-oidc-provider-check --che-operator-cr-patch-yaml=che-cr-patch.yaml --che-operator-image=custom-operator-image

che-cr-patch.yaml:

spec:
  auth:
    externalIdentityProvider: true
    identityProviderURL: ...
    nativeUserMode: true
    oAuthClientName: kubernetes
    oAuthSecret: ...
    openShiftoAuth: false
    updateAdminPassword: true
  devWorkspace:
    controllerImage: path/to/custom/image
  database:
    postgresImage: path/to/custom/image
  server:
    customCheProperties:
      CHE_OIDC_USERNAME__CLAIM: email
    airGapContainerRegistryHostname: custom_host
    airGapContainerRegistryOrganization: custom_org
    devfileRegistryImage:path/to/custom/image
    singleHostGatewayImage: path/to/custom/image

2. Che deploys successfully and comes online with several custom images
3. Once devworkspace-controller namespace becomes available, kubectl -n devworkspace-controller describe pods shows the devworkspace-controller container with the default quay.io image and kube-rbac-proxy -- not the image defined by spec.auth.devWorkspace.controllerImage or the airgap host/org.

Expected behavior

devworkspace-controller should honor the controllerImage or use the airgapped registry hostname/org

Runtime

Kubernetes (vanilla)

Screenshots

No response

Installation method

chectl/latest

Environment

Linux, Amazon

Eclipse Che Logs

No response

Additional context

No response

[tolusha]

Here is a fix overriding controller image eclipse-che/che-operator#1341
Regarding others devworkspace image, I would recommend to patch DW deployment.
Maybe @amisevsk can shed light if DWO can work in an air gapped environment.

[thenrich]

Thanks @tolusha. I had to make several other changes to get kube-rbac-proxy and the RELATED_IMAGEs to work as well -- see https://github.com/eclipse-che/che-operator/pull/1342/files

[tolusha]

@thenrich
Thank you! Could you open that PR against main branch?
I will close mine in favor if yours.

[amisevsk]

Maybe @amisevsk can shed light if DWO can work in an air gapped environment.

I believe DWO has all the pieces required for airgap support, and IIRC @ScrewTSW tested it out a few times without issue, so it shouldn't be a problem.