-
Notifications
You must be signed in to change notification settings - Fork 13
Module manager cannot fetch module OCI images from local registry in k3d setup as registry aliases of k3s are not respected #136
Comments
Thanks for your detailed issue description: I went through this step-by-step, following things came up:
=> this is because after adding the entry to
kubectl run -i --tty clustercurl --image=alpine --restart=Never -- sh -c 'apk --update add curl && curl registry.localhost:5001/v2/_catalog'; k delete pod clustercurl
kubectl run -i --tty clustercurl --image=alpine --restart=Never -- sh -c 'apk --update add curl && curl registry.localhost:5000/v2/_catalog'; k delete pod clustercurl and observing that The reason for this is that while k3s DOES respect There is an easy fix for you right now: After creating the ModuleTemplate, edit the ModuleTemplate with
This will result in the ModuleManager picking up the internal port instead of the external port, and it will be able to resolve the registry correctly. Note that while I tested your walkthrough (kyma-project/keda-manager@a399b60), module-manager still does not have the permissions in local mode for creating CRDs (since in local-mode it uses service account, while in remote mode, an administrative kubeconfig is expected):
This can be circumvented by giving the ClusterRole
This will then lead to a successful installation of keda-manager. I want to note that currently there is a misconfiguration of the default configuration of keda-manager, as it tries to install RBACs for Please Note that this RBAC change will not be suitable for production and can only be used for testing purposes and to circumvent a least-privilege module-manager in local mode. In the real world, module-manager in local mode should be installed with the correct privilege set by the runtime administrator (knowing which modules will be installed on the cluster). We are still figuring out what the best solution is here for automation, so stay tuned until we have a better solution for the RBAC escalation here. For remote management, we will work with kubeconfigs assigned to a service account which will be auto-provisioned based on the modules. (@janmedrek FYI this is a big - if not the biggest - problem scenario for RBAC Provisioning with an Operator for remote clusters) |
Thanks @jakobmoellersap. The adjustments in template and adding extra permissions to module manager made it work. |
Hey @jakobmoellersap, many thx for your input.
This was inherited (together with fixed os) from sample project, we will address it here.
We did not have yet time to fine tune the project (however we've prepared the PR to migrate to ginkgo v2 which will be a first step)
It sounds like it could be documented or/and added to Makefile in sample project.
We already have this issue open - we should discuss RBACs there.
We are aware of this keda issue however it's beyond the scope of this issue. |
A quick update on this ticket:
With this said, I would consider this issue closed. Please feel free to reopen if you face further issues with the ports. |
Description
When I try to setup local setup in single cluster mode (k3d) I end up getting errors when trying to enable keda module:
Module manager cannot connect to the local registry containing the module image.
Please see the detailed description how I execute it.
Steps to reproduce
etc/hosts
entry to register the local docker registry under aregistry.localhost
nametemplate.yaml
file and change thetarget
tocontrol-plane
Kyma installation is ready, but no module is activated yet
Keda Module is a known module, but not activated
Edit Kyma CR ...
..to add Keda module
Troubleshoot
When I inspect the registries at my k3d cluster i see
The text was updated successfully, but these errors were encountered: