X509Stores, HttpClient: Support custom CA bundle via environment variables

[tmds]

Something similar to curl's CURL_CA_BUNDLE.

from man curl:

--cacert
(TLS) Tells curl to use the specified certificate file to verify the peer. The file may contain multiple CA certificates. The certificate(s) must be in PEM format. Normally curl is built to use a default file for this, so this option is typically used to alter that default file.
curl recognizes the environment variable named 'CURL_CA_BUNDLE' if it is set, and uses the given path as a path to a CA cert bundle. This option overrides that variable.

This provides a global way to provide custom CA for all dotnet application. It avoids applications breaking and having to implement their own.

This is in particular useful in corporate environments where the https proxy is signed by a custom CA.

@stephentoub @davidsh @geoffkizer @wfurt @karelz

[stephentoub]

ManagedHandler just uses SslStream, which in turn uses all of the X509Certificate intrastructure, X509Stores, etc. Why is that insufficient?

[tmds]

ManagedHandler just uses SslStream, which in turn uses all of the X509Certificate intrastructure, X509Stores, etc. Why is that insufficient?

Maybe it is in some way supported already. I've updated the title to request this as a feature of the X509Stores.
Where does X509Store retrieve its certificates? Can a user (non-admin), in a way that affects all his dotnet applications, add custom CAs to that?

[stephentoub]

@bartonjs, the only thing that would be needed here is helper code to parse a bundle and add the certs to the appropriate store, right? Or am I missing a larger piece of the puzzle?

[tmds]

From the libcurl description, I have the impression the envvar actually becomes the entire store. So it isn't 'adding' but 'replacing'.
Does that make sense? Or are other things needed as part of the store that would not be in the bundle.

[stephentoub]

I have the impression the envvar actually becomes the entire store. So it isn't 'adding' but 'replacing'.

And that's important? We should ensure we have the right functionality; for something like this, I don't think it needs to be a drop-in replacement with all possible libcurl features.

[tmds]

And that's important? We should ensure we have the right functionality; for something like this, I don't think it needs to be a drop-in replacement with all possible libcurl features.

It makes it possible to replace everything. But for the use-cases I listed it is not a must. It may be nice if the same file is usable by curl, dotnet, ...

[bartonjs]

We respect the SSL_CERT_DIR and SSL_CERT_FILE environment variables from OpenSSL when loading the LocalMachine Root store. curl backed by OpenSSL should also respect those environment variables, since it's really OpenSSL that is using them. (Though there are things a caller can do to suppress their usage, so it might not work with curl).

For best results, make a new directory, put the one bundle file you are interested in in it, and point both variables appropriately

[tmds]

@bartonjs thank you! I will try this out next week.

[tmds]

@bartonjs @stephentoub I gave this a try and SSL_CERT_FILE is used by curl but not by .NET Core.

On Fedora 25 using .NET Core 2.0.

Using curl:

$ cp /etc/pki/tls/certs/ca-bundle.crt /tmp/ca-bundle.crt
$ chmod +w /tmp/ca-bundle.crt
$ gedit /tmp/ca-bundle.crt 
# remove 'VeriSign Class 3 Public Primary Certification Authority - G4'
$ curl https://www.microsoft.com
$ SSL_CERT_FILE=/tmp/ca-bundle.crt curl https://www.microsoft.com
curl: (60) Peer's Certificate issuer is not recognized.
More details here: https://curl.haxx.se/docs/sslcerts.html

curl performs SSL certificate verification by default, using a "bundle"
 of Certificate Authority (CA) public keys (CA certs). If the default
 bundle file isn't adequate, you can specify an alternate file
 using the --cacert option.
If this HTTPS server uses a certificate signed by a CA represented in
 the bundle, the certificate verification probably failed due to a
 problem with the certificate (it might be expired, or the name might
 not match the domain name in the URL).
If you'd like to turn off curl's verification of the certificate, use
 the -k (or --insecure) option.

Using .NET Core, performing HttpClient.GetAsync("https://www.microsoft.com") in console.dll

$ dotnet bin/Debug/netcoreapp2.0/console.dll
Response code: OK
$ SSL_CERT_FILE=/tmp/ca-bundle.crt dotnet bin/Debug/netcoreapp2.0/console.dll
Response code: OK

.NET Core is still using the system bundle:

$ SSL_CERT_FILE=/tmp/ca-bundle.crt strace -f -e open dotnet bin/Debug/netcoreapp2.0/console.dll 2>&1 | grep ca-bundle
[pid  5842] open("/etc/pki/tls/certs/ca-bundle.crt", O_RDONLY) = 36

[bartonjs]

@tmds The problem is you didn't also set SSL_CERT_DIR. (From curl that would end up as "read the first certificate from each file in the directory", I guess; whereas we seem to be "read all certs from all files in the directory")

jbarton@jsb-ubuntu2:~/coreclr$ ./corerun PrintStore.exe Root LocalMachine | grep found
147 certificate(s) found
jbarton@jsb-ubuntu2:~/coreclr$ SSL_CERT_FILE=/etc/foo ./corerun PrintStore.exe Root LocalMachine | grep found
147 certificate(s) found
jbarton@jsb-ubuntu2:~/coreclr$ SSL_CERT_DIR="" SSL_CERT_FILE=/etc/foo ./corerun PrintStore.exe Root LocalMachine | grep found
0 certificate(s) found
jbarton@jsb-ubuntu2:~/coreclr$ SSL_CERT_DIR="" SSL_CERT_FILE=dsa.cer ./corerun PrintStore.exe Root LocalMachine | grep found
1 certificate(s) found

[tmds]

@tmds The problem is you didn't also set SSL_CERT_DIR. (From curl that would end up as "read the first certificate from each file in the directory", I guess; whereas we seem to be "read all certs from all files in the directory")

Setting SSL_CERT_FILE=/tmp/ca-bundle.crt works for curl.
To what should I set SSL_CERT_DIR and SSL_CERT_FILE to use the bundle at /tmp/ca-bundle.crt?
Even when I set both vars, it still goes to the system bundle:

$ SSL_CERT_DIR=/tmp SSL_CERT_FILE=ca-bundle.crt strace -f -e open dotnet bin/Debug/netcoreapp2.0/console.dll 2>&1 | grep ca-bundle
[pid  8569] open("/etc/pki/tls/certs/ca-bundle.crt", O_RDONLY) = 71

Perhaps Ubuntu is behaving different from Fedora?

[bartonjs]

To what should I set SSL_CERT_DIR and SSL_CERT_FILE to use the bundle at /tmp/ca-bundle.crt?

SSL_CERT_DIR="" SSL_CERT_FILE=/tmp/ca-bundle.crt

Perhaps Ubuntu is behaving different from Fedora?

Maybe. I don't have a Fedora machine available, but I checked it with RHEL:

using System;
using System.Security.Cryptography.X509Certificates;

namespace PrintStore
{
    class Program
    {
        static void Main(string[] args)
        {
            Console.WriteLine("Opening LocalMachine\\Root store");

            using (X509Store store = new X509Store(StoreName.Root, StoreLocation.LocalMachine, OpenFlags.ReadOnly))
            {
                var certs = store.Certificates;
                Console.WriteLine($"Store contains {certs.Count} certificate(s)...");
            }
        }
    }
}

$ dotnet run
Opening LocalMachine\Root store
Store contains 193 certificate(s)...
$ SSL_CERT_DIR="" SSL_CERT_FILE="" dotnet run
Opening LocalMachine\Root store
Store contains 0 certificate(s)...
$ SSL_CERT_DIR="" SSL_CERT_FILE="/home/jbarton/trust/test.cer" dotnet run
Opening LocalMachine\Root store
Store contains 1 certificate(s)...
$ SSL_CERT_FILE="/home/jbarton/trust/test.cer" dotnet run
Opening LocalMachine\Root store
Store contains 194 certificate(s)...

It's maybe possible that the environment variable names are different on Fedora? You could do something like this to check:

$ locate libcrypto
/usr/lib64/.libcrypto.so.1.0.2k.hmac
/usr/lib64/.libcrypto.so.10.hmac
/usr/lib64/libcrypto.so.1.0.2k
/usr/lib64/libcrypto.so.10
$ cd /usr/lib64/
$ strings libcrypto.so.10 | grep SSL_CERT_
SSL_CERT_DIR
SSL_CERT_FILE

My system data

$ dotnet --info
.NET Command Line Tools (2.0.0)

Product Information:
 Version:            2.0.0
 Commit SHA-1 hash:  cdcd1928c9

Runtime Environment:
 OS Name:     rhel
 OS Version:  7
 OS Platform: Linux
 RID:         rhel.7-x64
 Base Path:   /opt/rh/rh-dotnet20/root/usr/lib64/dotnet/sdk/2.0.0/

Microsoft .NET Core Shared Framework Host

  Version  : 2.0.0
  Build    : N/A

Our functions are fairly straightforward if you want to try debugging to see what's happening: https://github.com/dotnet/corefx/blob/release/2.0.0/src/Native/Unix/System.Security.Cryptography.Native/pal_x509_root.cpp

[tmds]

@bartonjs I will try it next week. We are testing in a different way. Can you check if HttpClient uses those envvars (that is how I am testing)?

[tmds]

Fedora behaves the same for the X509Store:

$ SSL_CERT_DIR="" SSL_CERT_FILE="" dotnet run
Opening LocalMachine\Root store
Store contains 0 certificate(s)...

So the managed http handler will pick those up. It's a bit weird you need to set both variables to affect the X509Store and only one to affect the curl command. Not sure where the difference in behavior stems from.

The curl handler in .NET Core 2.0 (HttpClient) ignores those variables for each attempt I have done (on Fedora 24, 25 and RHEL 7.3).

[tmds]

@stephentoub @bartonjs the user of libcurl is responsible for picking up these envvars and passing them to curl via CURLOPT_CAINFO and CURLOPT_PROXY_CAINFO. corefx is not doing this.

[tmds]

For reference:
This is the curl code that reads the environment variables: https://github.com/curl/curl/blob/10a659dbf64602c13875af4f79360eaccab9e6d3/src/tool_operate.c#L306-L344
And here they are passed to libcurl: https://github.com/curl/curl/blob/10a659dbf64602c13875af4f79360eaccab9e6d3/src/tool_operate.c#L1095-L1125

[tmds]

@stephentoub If there will still be a curlhandler in 2.1, I can implement the passing SSL_CERT_{DIR,FILE}. wdyt?

[stephentoub]

If there will still be a curlhandler in 2.1

There very likely will be. We'd only be able to remove it if we're able to implement both HTTP/2 and Negotiate/NTLM auth in ManagedHandler in time, and while I think there's a reasonable chance we'll be able to do some of that, I don't expect we'll be able to do all of it (unless others want to jump in and help).

I can implement the passing SSL_CERT_{DIR,FILE}. wdyt?

I'm not entirely following this thread. Can you summarize for me what the issue is, what this would provide, etc.? Thanks!

[tmds]

I'm not entirely following this thread. Can you summarize for me what the issue is, what this would provide, etc.? Thanks!

Sometimes you want to override the CAs trusted by an application.
A very typical case is to trust the corporate https proxy that is setup with a custom CA.

The openssl way of enabling this is using the SSL_CERT_{DIR,FILE} envvars.
X509Stores (and thus the managed http handler) respects these variables.
The curl handler is not using them.

The curl command picks up those variables and passes them to libcurl. I can add the equivalent code to corefx.

[stephentoub]

The curl command picks up those variables and passes them to libcurl.

Ah, this is the part I was missing. So you're saying that these env vars are respected by curl, and they're respected by ManagedHandler because they're respected by X509Store because they're respected by OpenSSL, but they're not respected by libcurl itself and thus not by CurlHandler?

[tmds]

respected by X509Store because they're respected by OpenSSL

This is how I understood it from @bartonjs

but they're not respected by libcurl itself and thus not by CurlHandler?

Yes. Libcurl isn't picking these up by default. It has options for them.

[tmds]

Implemented in dotnet/corefx#25219