buildx push to self-signed private registry error

[lazywhite]

I deployed a harbor registry and enabled http/https protocol at same time

here is my buildkitd.toml

debug = true
insecure-entitlements = [ "network.host", "security.insecure" ]

[registry."docker.io"]
  mirrors = [
    #"mirror.baidubce.com",
    #"hub-mirror.c.163.com",
    "49ri251n.mirror.aliyuncs.com"
  ]

[registry."harbor.demohub.com"]
  http = true
  insecure = true

although I configured my private registry as insecure, buildx still failed because of x509

docker buildx build \
	--platform linux/amd64,linux/arm64 \
	--output type=image \
	--tag harbor.demohub.com/library/alpine:3.16 \
	--network host \
	--push \
	.
[+] Building 67.1s (9/9) FINISHED                                                             
 => [internal] load build definition from Dockerfile                                     0.1s
 => => transferring dockerfile: 54B                                                      0.0s
 => [internal] load .dockerignore                                                        0.1s
 => => transferring context: 2B                                                          0.0s
 => [linux/amd64 internal] load metadata for docker.io/library/alpine:3.16              66.7s
 => [linux/arm64 internal] load metadata for docker.io/library/alpine:3.16              66.3s
 => [auth] library/alpine:pull token for registry-1.docker.io                            0.0s
 => [auth] library/alpine:pull token for registry-1.docker.io                            0.0s
 => [linux/arm64 1/1] FROM docker.io/library/alpine:3.16@sha256:b95359c2505145f16c6aa38  0.1s
 => => resolve docker.io/library/alpine:3.16@sha256:b95359c2505145f16c6aa384f9cc74eeff7  0.0s
 => [linux/amd64 1/1] FROM docker.io/library/alpine:3.16@sha256:b95359c2505145f16c6aa38  0.1s
 => => resolve docker.io/library/alpine:3.16@sha256:b95359c2505145f16c6aa384f9cc74eeff7  0.0s
 => ERROR exporting to image                                                             0.1s
 => => exporting layers                                                                  0.0s
 => => exporting manifest sha256:3349e20df077de5eba3cf212edf5d028e1658c8ab1222cbe617cf9  0.0s
 => => exporting config sha256:3de62af236353810c436f3eed6684e67a6270251dbcaf886a5bdb570  0.0s
 => => exporting manifest sha256:fe04bcb3e8fc9f8976c29b900662f7e3671d3a7f4b8e7bbbe7358c  0.0s
 => => exporting config sha256:8aa37678b2f06847f906c397082aada09154408018687e20f87c27ee  0.0s
 => => exporting manifest list sha256:04c38a1345227d30da8875c809d5d6a7f625b4f214f8323a5  0.0s
 => => pushing layers                                                                    0.0s
------
 > exporting to image:
------
ERROR: failed to solve: failed to do request: Head "https://harbor.demohub.com:443/v2/library/alpine/blobs/sha256:8aa37678b2f06847f906c397082aada09154408018687e20f87c27eed44d3f93": x509: certificate signed by unknown authority
make: *** [Makefile:7: images] Error 1

then I fixed the problem by this buildkitd.toml

debug = true
insecure-entitlements = [ "network.host", "security.insecure" ]

[registry."docker.io"]
  mirrors = [
    #"mirror.baidubce.com",
    #"hub-mirror.c.163.com",
    "49ri251n.mirror.aliyuncs.com"
  ]

[registry."harbor.demohub.com"]
  ca=["/data/cert/ca.crt"]

my questions:

1. why buildx still used https even configured with insecure and http
2. can we make buildx skip verify private registry cert like docker ( /etc/docker/daemon.json insecure-registries) , then we have no need to provide CA cert for buildx instance.

[jedevc]

I'm having a look at this, not quite finished diving into it, but it does seem related to moby/buildkit#3382.

[lazywhite]

I found that this problem is caused by harbor, when enabled both http & https , harbor will 302 redirect http to https, this behavior is commented in harbor.yml

hostname: harbor.demohub.com
# http related config
http:
  # port for http, default is 80. If https enabled, this port will redirect to https port
  port: 80

so we have 2 solutions

1. enable http of harbor only, and configure http = true only in buildkitd.toml
2. enable https of harbor only, and configure insecure = true only in buildkitd.toml, no need to provide ca.crt

attention:

1. should not configure http = true and insecure = true at same time in buildkitd.toml, see this issue http and insecure are mutually exclusive, when used on a registry with a non-standard port moby/buildkit#3382
2. comment http config can't disable harbor http actually, I changed it to a port other than 80