update-ca-certificates fails

[BrianHicks]

When run under Docker, update-ca-certificates (from the package ca-certificates) fails with the line . All the information I can find seems to suggest it's a locale issue but the system profile seems to have the correct information. Any ideas?

The failure:

bash-4.3# update-ca-certificates
WARNING: ca-certificates.crt does not contain exactly one certificate or CRL: skipping

[andyshinn]

That isn't necessarily a failure. It is just saying "the ca-certificates.crt file has multiple certificates so we are not considering it to be symlinked". What is the actual problem here? Do you have an example I can try to reproduce?

[BrianHicks]

sure, CiscoCloud/nginx-consul (currently on the Docker registry as asteris/nginx-consul.) If you mount a directory with CA certs in the place mentioned in the README and run update-ca-certificates that happens. Am I misunderstanding the package instructions?

[andyshinn]

Do you have an example CA PEM or a command I can use to generate one? I'll see if I can reproduce.

To clarify: the CA is not getting added to the ca-certificates.crt file regardless of the error message?

[BrianHicks]

correct. I don't have one, but you can get one by running the security-setup script in CiscoCloud/microservices-infrastructure.

[andyshinn]

Indeed, looks like an upstream bug. The line at https://github.com/alpinelinux/aports/blob/master/main/ca-certificates/update-ca-certificates#L50 should be for key,fn in pairs(certlist) do. I'll poke upstream and try to get a patch out.

[BrianHicks]

Great, thank you!

[andyshinn]

Patch submitted: http://patchwork.alpinelinux.org/patch/104/.

[andyshinn]

Merged in alpinelinux/aports@3faf2e0. New package pushed to edge. Give it a try and let me know if we can close this.

[andyshinn]

This fix should also be in Alpine 3.2 as well. We just pushed a new 3.2 tag. Reopen if still an issue.

[sudoforge]

This looks to be an issue in alpine:3.3:

/ # apk add ca-certificates
OK: 28 MiB in 31 packages
/ # update-ca-certificates 
WARNING: ca-certificates.crt does not contain exactly one certificate or CRL: skipping

[andyshinn]

What specifically is the issue?

[sudoforge]

On second inspection, I don't think ca-certificates is the root of my issue, but it may be worth patching anyhow (that's obviously at your discretion).

[andyshinn]

The warning is just a warning. It doesn't affect anything.

[dmugtasimov]

FROM alpine:3.5

RUN apk add --no-cache ca-certificates && \
    update-ca-certificates

WARNING: ca-certificates.crt does not contain exactly one certificate or CRL: skipping

The message says that something is skipped. It makes an impression that user get incomplete set of root certificates. Could you, please, rephrase, remove or fix this message, so people are not getting confused about the message?

[andyshinn]

This should be brought up on http://bugs.alpinelinux.org/projects/alpine/issues. It comes from the utility in the ca-certificates package at https://git.alpinelinux.org/cgit/aports/tree/main/ca-certificates/c_rehash.c#n210. I agree, the warning could be better or ignored altogether if the file is ca-certificates.crt.

[alwqx]

+1

[sudoforge]

@adolphlwq, your "+1" comment doesn't really give us any new information.

If you're experiencing the same issue, have none of the above comments helped? As @andyshinn noted two full moons ago, this issue doesn't really belong here.

[pedromanoel]

@andyshinn mentioned that this is just a warning and shouldn't affect anything. However, update-ca-certificates is exiting with code 1, and I took some time to find out why my command list was failing...

[sergeyklay]

What about this guys:

FROM alpine:3.7

RUN apk update \
    && apk upgrade \
    && apk add --no-cache \
        ca-certificates \
    && update-ca-certificates 2>/dev/null || true

[danielpalstra]

I think it's great for a workaround, but the command shouldn't exit 1 on a warning. Or we should be able to ignore warnings. I have implemented this workaround in my Dockerfile, but I don't think I can really assume that everything is fine certificate wise anymore.

[T4toh]

/ # mv DigiCertSHA2SecureServerCA.crt.pem /usr/local/share/ca-certificates/DigiCertSHA2SecureServerCA.crt / # update-ca-certificates WARNING: ca-certificates.crt does not contain exactly one certificate or CRL: skipping
I'm running this in the image itself and get that warning, and I don't know if my certificates are updated or not.

[Lauriy]

Just don't use Alpine & Docker. Nobody cares if you shave 100 MB off an image. 2 hours of your work is a terabyte disk.

[GabLeRoux]

Message is misleading, but it's only a warning. It doesn't mean it failed to run update-ca-certificates as according to exit code, it did succeed if it continues to next step.

In case you have access to python inside your docker image and really want to make sure the certificate was correctly added, here's an example with a verification.

ARG CA_BUNDLE_SOURCE=https://s3.amazonaws.com/rds-downloads/rds-combined-ca-bundle.pem
ARG CA_BUNDLE_DESTINATION=/usr/local/share/ca-certificates/rds-combined-ca-bundle.pem

ADD $CA_BUNDLE_SOURCE $CA_BUNDLE_DESTINATION
RUN update-ca-certificates
RUN python -c "x=open('$CA_BUNDLE_DESTINATION').read(); y=open('/etc/ssl/certs/ca-certificates.crt').read(); exit(0) if x in y else exit(-1)"

Above python trick comes from https://unix.stackexchange.com/a/114882/63222 and was adapted to fail if file doesn't contain other file. There are probably much better ways out there.

At this point, it's probably easier to append the bundle directly to /etc/ssl/certs/ca-certificates.crt like this:

ARG CA_BUNDLE_SOURCE=https://s3.amazonaws.com/rds-downloads/rds-combined-ca-bundle.pem
ARG CA_BUNDLE_DESTINATION=/usr/local/share/ca-certificates/rds-combined-ca-bundle.pem

ADD $CA_BUNDLE_SOURCE $CA_BUNDLE_DESTINATION
RUN cat $CA_BUNDLE_DESTINATION >> /etc/ssl/certs/ca-certificates.crt

[bbolli]

I have looked into this a bit closer.

The reason for this warning is the c_rehash program that runs at the very end of update-ca-certificates. Its purpose is to (re)generate the hash symlinks in /etc/ssl/certs. This is only possible if a file contains a single certificate (or revocation list).

When c_rehash encountersca-certificates.crt which contains the concatenation of all certificates, it rightly complains that this file cannot be hashed. But as noted above, this is indeed just a warning and the update of the certificates did work.

Source: https://git.alpinelinux.org/ca-certificates/

I suggest to close this issue.

[nikskiz]

It may be a warning, but why does the message include skipping. This is misleading as others have mentioned.

[bbolli]

but why does the message include skipping

Because the file mentioned contains more than one certificate and thus can't be hashed. This is output from c_rehash.

[franz-josef-kaiser]

The issue still persists. I get the warning both for a cert and the store file:

Reproduce:

1. Try to install cert

    /etc/ssl/certs # update-ca-certificates 
   WARNING: ca-certificates.crt does not contain exactly one certificate or CRL: skipping
   WARNING: ca-cert-<DOMAIN>.<TLD>.pem does not contain exactly one certificate or CRL: skipping

2. Checking, if the cert was added to the store: The solution by @GabLeRoux is good, but probably an overkill. There's no need for python in your container: Alpine Linux ships with comm (a reverse diff):

   comm -12 /etc/ssl/certs/ca-cert-<DOMAIN>.<TLD>.pem  /etc/ssl/certs/ca-certificates.crt

   The result, that you should see if the cert was added to the ca-certificates.crt-store is:

   -----BEGIN CERTIFICATE-----
   

   …or nothing. If you see a complete cert, then it tells you that the cert wasn't successfully added.
   In my case, the cert was added, still the WARNING about the single cert is there.

3. Try to fetch from a local URl:

   /etc/ssl/certs # wget -qO- http://127.0.0.1/nginx_status 
   ssl_client: <DOMAIN>.<TLD>: certificate verification failed: unable to get local issuer certificate
   wget: error getting response: Connection reset by peer

4. Ignore the cert for a wget call. Does not work for other connections where you do not have access to request args:

   /etc/ssl/certs # wget --no-check-certificate -qO- http://127.0.0.1/nginx_status 
   Active connections: 1 
   server accepts handled requests
   10 10 8 
   Reading: 0 Writing: 1 Waiting: 0

[rustyx]

@franz-josef-kaiser that is another issue, your cert ca-cert-<DOMAIN>.<TLD>.pem is skipped from rehashing because it didn't contain exactly one cert or CRL (like the message says).

ca-certificates.crt isn't really used. Instead a certificate is looked up by its hash - a symlink like fe8a2cd8.0. These symlinks are created by the program c_rehash which is invoked as part of update-ca-certificates. And that's where the warnings come from.

The warning about the bundle is really just a warning though. It's because update-ca-certificates is writing the bundle before invoking c_rehash /etc/ssl/certs/. I've submitted a PR#1 to Alpine to address this. Just tested and seems to be working fine.

~ # update-ca-certificates
~ #

[franz-josef-kaiser]

@rustyx Thanks for clarifying that (and submitting the PR to Alpine@GitLab)

Instead a certificate is looked up by its hash - a symlink like fe8a2cd8.0.

[yellowmegaman]

Still a problem, wow.

[1in9e]

A problem.
FROM python:3.8-alpine
bash-5.1# update-ca-certificates WARNING: ca-certificates.crt does not contain exactly one certificate or CRL: skipping

[dualmacops]

From Home Assistant core-2021.8.8

bash-5.1# update-ca-certificates
WARNING: ca-certificates.crt does not contain exactly one certificate or CRL: skipping

[franz-josef-kaiser]

The following warning is not an error. It's just how the command works: It checks all certs, including the main file, which contains more than one file. The error message just could be improved:

WARNING: ca-certificates.crt does not contain exactly one certificate or CRL: skipping

[lhzw]

It's much better if it outputs the result as normal linux, then we know the cert has been added, so it's just a warning. At first glance at this warning and no other outputs, I thought it did not add the cert.

update-ca-certificates
Updating certificates in /etc/ssl/certs...
1 added, 0 removed; done.
Running hooks in /etc/ca-certificates/update.d...

updates of cacerts keystore disabled.
done.

[journaux]

haha. oh my. still an issue all these years later.

[JonathonReinhart]

haha. oh my. still an issue all these years later.

@vitruvvius What have you contributed to fixing the issue?

[amoona6]

First-time poster! Please excuse any informal etiquette.

I spent 2-hours troubleshooting a similar issue in Ubuntu, but was able to come up with a solution (my thought is maybe my findings could be helpful for others). Firstly, I started from scratch:

• removed all entries and files related to my .crt file (find / -type f -name "[certname].*")
• ran sudo update-ca-certificates --fresh
• follow up on any loose ends (had to remove one line from /etc/ca-certificates.conf)

Important:

• Follow all the usual steps, up until running update-ca-certificates
• Before running the updater, set the CA certificate file permissions to 644 (sudo chmod 644 ./file.crt). Originally, in my case, my certificate had 755 permissions
• Run sudo update-ca-certificates

After running the update, the certificate was available.

[huchim]

Hi!

This is not Alpine, but I figured out my certificates files was using UTF-8 BOM and saved without BOM and certificates was included without warnings.

FROM node:14
COPY CPTTRootCertificateAuthority.crt /usr/local/share/ca-certificates/
COPY CPTTIntermediateCertificate.crt /usr/local/share/ca-certificates/
RUN update-ca-certificates

Maybe this comment is not useful, but works for me.

[enderandpeter]

For what it's worth, the exit code for this warning now appears to be 0, fortunately:

bash-5.1# update-ca-certificates --fresh
WARNING: ca-certificates.crt does not contain exactly one certificate or CRL: skipping
bash-5.1# echo $?
0

[JoelTrain]

I see this "warning" does not contain exactly one certificate or CRL
and then immediately try to use my supposedly newly trusted certificate to make a TLS connection using curl and it fails
curl: (60) SSL certificate problem: self signed certificate in certificate chain

So in my case it does seem the addition to the trust store was indeed skipped as well.
I guess results may vary depending on versions or some other lurking variable.

[rwjack]

It seems the container recognizes my CA (docker:dind - which is alpine basically), and running update-ca-certificates shows nothing within the container, but the error everyone is talking about is constantly showing up on my gitlab CI jobs.
WARNING: ca-certificates.crt does not contain exactly one certificate or CRL: skipping

Fixed:
runner: config.toml:

[runners.docker]
#volumes = ["/cache", "/opt/gitlab-runner/config/certs/ca.crt:/etc/gitlab-runner/certs/ca.crt:ro"]
volumes = ["/cache"]

The runner tries to be smart and if /etc/gitlab-runner/certs/ca.crt is mapped as a volume, it will constantly try to inject it, hence the error messages.

[Acerinth]

@rwjack I have the same issue in GitLab CI - only I'm not mounting it with Runner config, but tried installing during a job directly inside a docker-dind image:

services:
    - name: docker:20.10-dind
      command:
        - /bin/sh
        - -c
        - echo "$CI_SERVER_TLS_CA_FILE" > /usr/local/share/ca-certificates/my-ca.crt && update-ca-certificates && dockerd-entrypoint.sh || exit

Did you found a workaround, apart from creating your own dind image with injected CA's certs?

[rwjack]

@Acerinth I'm not sure honestly, though I managed to evade that warning somehow. I still mount the CA in the gitlab-runner config.toml as shown above, but I also build my own dind image.

FROM docker:dind

COPY CA.pem /usr/local/share/ca-certificates/ca.crt # <- IMPORTANT to be *.crt
RUN update-ca-certificates --fresh

[JoelTrain]

Does anyone have a script that splits a single .pem file with multiple certificates in it into individual files?
That way I can actually import and trust them without the warning/skip?

I am encountering this "warning" again and would like all the certificates in my file to actually be imported and trusted.

[JoelTrain]

Also, I am sorry. I think this is the wrong repo to report this issue.
So if you can point me to where to report the problem to the correct source repo, that would be appreciated as well.

I think skipping certs in a file should be an error, not a warning. (whether they are actually skipped or not is up for debate in this thread, but in some cases the warning about skipping actually does mean they are skipped and not added to the trust store)