Flux can't check for updated images in a private registry

[brantb]

When using a private docker registry, flux doesn't seem to be able to check for updated images using the default ServiceAccount's imagePullSecrets.

I tested this out on a brand new Azure Container Registry instance and k8s namespace created for this purpose.

# Created using:
# kubectl create secret docker-registry registry-secret -o json \
#   --docker-server=branttest.azurecr.io \
#   --docker-email=xyz@example.com \
#   --docker-username=branttest \
#   --docker-password=<redacted>
apiVersion: v1
data:
  .dockerconfigjson: <snip>
kind: Secret
metadata:
  name: registry-secret
  namespace: flux-auth
type: kubernetes.io/dockerconfigjson
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: default
  namespace: flux-auth
imagePullSecrets:
  - name: registry-secret
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  labels:
    app: party-clippy
  name: party-clippy
  namespace: flux-auth
spec:
  selector:
    matchLabels:
      app: party-clippy
  template:
    metadata:
      labels:
        app: party-clippy
    spec:
      containers:
        - image: branttest.azurecr.io/party-clippy:latest
          imagePullPolicy: Always
          name: party-clippy
          args: ["-p", "80"]

k8s is able to pull and run the container. However, after killing flux and memcached (to wipe out any cached creds) I still see this in the logs:

ts=2018-09-26T20:14:29.675347521Z caller=warming.go:162 component=warmer canonical_name=branttest.azurecr.io/party-clippy auth={map[]} err="requesting tags: Get https://branttest.azurecr.io/v2/party-clippy/tags/list: unauthorized: authentication required"

Adding an imagePullSecret directly to the Deployment makes this error go away (presumably because it's working).

[brantb]

It looks like I was actually running an older version of flux in this cluster (v1.5.0). I'll reopen this once I get a chance to test with the latest if it's still an issue.

[sc250024]

Getting the same error; adding the...

imagePullSecrets:
  - name: registry-secret

...didn't do anything, and I'm running the latest version of Flux (1.7.1).

[Morriz]

Can we reopen this? I can't seem to get it to work either. Secret exists and is made as per your instructions...

[ericbarch]

Also seeing this issue on Flux 1.7.1 using Azure Kubernetes Service (brand new RBAC cluster).

[ggolin]

Still seeing this in 1.9 w/private registry that uses an internally signed certificate.

[4c74356b41]

yeah, nothing works, adding imagepullsecrets to deployment\serviceaccout. doesnt matter. ACR, flux 1.9, AKS 1.11.5 @brantb

[4c74356b41]

apparently its not a thing as of now: #1694

[pierretasci]

The merged PR fixes this issue for Azure's container registry but what about other private registries? I feel like the real solution here is to use the imagePullSecrets from the pod if it is available.

[kwontae]

I'm having the same error. Not using AKS. Using our own k8s cluster.
I'm currently using flux:1.15.0
Currently, I have this in my flux deployment yaml file.

spec:
      serviceAccountName: flux
      imagePullSecrets:
        - name: speppeacr

I'm still getting ts=2019-11-05T22:59:08.348762584Z caller=warming.go:180 component=warmer canonical_name=examplePrivate.azurecr.io/project auth={map[]} err="requesting tags: Get https://examplePrivate.azurecr.io/v2/project/tags/list: unauthorized: authentication required

[AntoineGodeau]

@kwontae Have you resolve this issue ? I have the same one

[kwontae]

Hey @AntoineGodeau #2589 (comment) this has helped me resolve my issue!