NTLM authentication HttpClient in Core

[FenrisWolfAtMiddleEarth]

I am trying to use the HttpClient to access a REST service which requires NTLM authentication. However I keep getting a 401 Unauthorized. My code looks like this

    private static void Main()
    {
        var uri = new Uri("http://localhost:15001");
        var credentialsCache = new CredentialCache { { uri, "NTLM", CredentialCache.DefaultNetworkCredentials } };
        var handler = new HttpClientHandler { Credentials = credentialsCache };
        var httpClient = new HttpClient(handler) { BaseAddress = uri, Timeout = new TimeSpan(0, 0, 10) };
        httpClient.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
        var response = httpClient.GetAsync("api/MyMethod").Result;
    }

My target framework is netcoreapp2.0. If I change to net461, it will work. Not sure what I am doing wrong?

[EDIT] Add C# syntax highlighting by @karelz

[davidsh]

Is this on Windows or Linux? Is there any redirection involved with your REST service? Can you attach a full repro of the problem including the code for the localhost server?

[FenrisWolfAtMiddleEarth]

• It is Windows platform, or I could not verify that it works as net461
• No redirection is needed to reproduce the problem
• The attached solution can reproduce the problem. Run the host part on a different machine, as it cannot be reproduced on localhost.

Change the target framework of the client project to net461, and it will work

NtlmAuthentication.zip

[davidsh]

[Edit]

I think this is related to a problem that was supposedly fixed involving how default credentials are transmitted. (#8434).

However, you mention that this works on loopback (localhost) but doesn't work on remote servers. I think it is a problem with how .NET Core (Windows) sets the internal WINHTTP_AUTOLOGON_SECURITY settings for how/when to release default credentials to the remote server.

Since you are explicitly requesting DefaultCredentials, they should be sent. We will need to investigate and look at traces to root cause.

cc: @karelz

[FenrisWolfAtMiddleEarth]

Ok. Thanks for the update.
I can see the issue has been added to the 2.1.0 milestone. ETA for that is Q2, right ?

[karelz]

Yes, ETA is currently Q2: https://github.com/dotnet/core/blob/master/roadmap.md#upcoming-ship-dates (it could change)

I set the milestone based on my best assessment of the impact (compat with .NET Framework and @davidsh's notes). It is however possible we may change it later and push it out of 2.1.

[karelz]

Triage: @FenrisWolfAtMiddleEarth we are confused - your repro uses localhost, but you mention it does not work on remote machines and works on localhost. Can you please clarify?
HttpClient policy would be to not send default credentials to non-intranet machines.

[FenrisWolfAtMiddleEarth]

My client in the repro uses a make-up DNS-name "remotehost". Replace that with there host-name where you run the NtlmAuthenticationHost project:

        private static void Main()
        {
            CallService("http://remotehost:9876", "api/Hello");
            Console.ReadKey();
        }

That should reproduce the problem. I.e. it will fail.
If you change it to localhost and run both on the same machine it will not fail.

[karelz]

Thanks for clarification.

[Priya91]

That should reproduce the problem. I.e. it will fail.
If you change it to localhost and run both on the same machine it will not fail.

As @karelz mentioned in the previous comment, this is by design that the handlers don't send out default credentials to non-intranet machines. This is by design.

[karelz]

Closing per @Priya91's comment.

@FenrisWolfAtMiddleEarth if you believe there is still some issue lurking, it would help to be crystal clear and provide isolated repro with all details in a single post to avoid further confusion. Thanks!

[FenrisWolfAtMiddleEarth]

We finally figured out a solution to this problem. The solution though indicates that there is a bug in .net Core.
What triggered the problem was that the http host accepted both kerberos and ntlm authentication.. Our host is Owin, and the code to accept both looks like this:

                listener.AuthenticationSchemes = AuthenticationSchemes.IntegratedWindowsAuthentication;

Even though I specifically set the client to use NTLM, it would not respond to a challange which took both kerberos and ntlm. My http client was created like this:

            var uri = new Uri("http://service-endpoint");
            var credentialsCache = new CredentialCache { { uri, "NTLM", CredentialCache.DefaultNetworkCredentials } };
            var handler = new HttpClientHandler { Credentials = credentialsCache };
            var httpClient = new HttpClient(handler) { BaseAddress = uri, Timeout = new TimeSpan(0, 0, 10) };
            httpClient.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));

What solved the problem was to change the host to only accept ntlm

                listener.AuthenticationSchemes = AuthenticationSchemes.Ntlm;

That very much sounds like a bug in the .net core HttpClient

[FenrisWolfAtMiddleEarth]

@karelz
Please consider opening this issue again.
Thanks

[karelz]

I don't think the code above specifically sets client to use NTLM. It just provides NTLM credentials, that is all.
Kerberos is more secure than NTLM and therefore picked as it is supported by both client and the server.

@davidsh @wfurt is that correct summary? Is there a way to force client to use specific authentication scheme?

[davidsh]

From the description above, this is a bug in our HTTP client stack.

The server can offer multiple schemes such as Negotiate (Kerberos) and NTLM. The client will pick the strongest but only if the credential can be used for that scheme.

When a NetworkCredential object is used on the client, that credential is valid for all schemes (Basic, Digest, NTLM, Negotiate). But when a CredentialCache is used, that credential is only valid for the scheme assigned to it in the cache.

In this issue, the server is presenting 2 possible schemes: Negotiate and NTLM. But the client is using a CredentialCache and only has a single credential with just NTLM. We did have a bug in our HTTP stack where we would first pick Negotiate because it is the strongest. But then fail to find a credential that can be used (since we're using CredentialCache) and result in no credential being used at all. Instead, we should fallback to another scheme (such as NTLM) that the server supports and find a matching credential in our cache.

I thought we fixed this bug at least in WinHttpHandler. See: PR dotnet/corefx#28105

Not sure if we fixed it in all handlers. @rmkerr might have more information. Either way, this issue should be re-opened but it might be fixed in .NET Core 2.1.

[wfurt]

dotnet/corefx#28105. Targeting netcore2.1 should give you the fix @FenrisWolfAtMiddleEarth

[rmkerr]

The issue @wfurt linked to is the correct one. This should be fixed in 2.1.

[karelz]

Duplicate of dotnet/corefx#27672

[FenrisWolfAtMiddleEarth]

I can confirm it is working with core 2.1
Problem is that it will most likely be months before our build servers are upgraded with core 2.1. I guess that is something I will have to discuss with our TFS team

[danmoseley]

@FenrisWolfAtMiddleEarth definitely something to pursue as now that 2.1 is out, 2.0 will go out of support in 3 months (https://github.com/dotnet/core/blob/master/microsoft-support.md)

[SakulRelda]

@FenrisWolfAtMiddleEarth
Could you give a code example how NTLM Authentication with .NET Core 2.1 (Console-Application / WebApp) is working? I tried the following:

var handler = new HttpClientHandler();
            var credential = new NetworkCredential("", "", "");
            handler.AllowAutoRedirect = true;
            var ccache = new CredentialCache
            {
                { new Uri(baseUrl), "NTLM", CredentialCache.DefaultNetworkCredentials }
            };
            handler.Credentials = ccache;

            HttpClient client;
            client = new HttpClient(handler)
            {
                BaseAddress = new Uri("http://deagxcuewebt02.sickcn.net")
            };
            client.DefaultRequestHeaders.Accept.Clear();
            client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));


            HttpContent contentPost = new StringContent(parameterHeaddata, Encoding.UTF8, "application/json");
            HttpResponseMessage response = client.PostAsync(targetUrl_HeadData, contentPost).Result;`

But it doesn't work

[FenrisWolfAtMiddleEarth]

@SakulRelda

This code works for me in .net core 2.1.
What I have not been able to solve, is reuse of connections. It appears .net Core will reconnect every time, which is not very usefull, when many requests are issued to the same endpoint

        public HttpClient Create()
        {
            var uri = new Uri(_urlRoot);
            var credentialsCache = new CredentialCache { { uri, "NTLM", CredentialCache.DefaultNetworkCredentials } };
            var handler = new HttpClientHandler { Credentials = credentialsCache };
            var httpClient = new HttpClient(handler) { BaseAddress = uri  };
            httpClient.DefaultRequestHeaders.ConnectionClose = false;
            httpClient.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));

            ServicePointManager.FindServicePoint(uri).ConnectionLeaseTimeout = 120 * 1000;  // Close connection after two minutes

            return httpClient;
        }

[d4hines]

@FenrisWolfAtMiddleEarth, I'm using the above code in a Docker container targeting Linux running core 2.1, and I'm still getting a 401 error. Any suggestions? @SakulRelda, did you ever find a fix?

[IharYakimush]

Had same issue with application running in Docker (linux) requesting REST service with enabled both Kerberos and NTLM authentication.

Resolved by

1. setting AppContext.SetSwitch("System.Net.Http.UseSocketsHttpHandler", false); as described in https://github.com/dotnet/corefx/issues/28961
2. forcing application to use NTLM only by providing credentials via CredentialCache for NTLM only

new CredentialCache
                {
                    {
                        rootUrl, "NTLM",
                        new NetworkCredential("domain\name", "pwd")
                    }
                }

[futuralogic]

This seems to be an issue that may reflect more broadly with HttpClient. Oddly enough occurring in a WPF app targeting .NET Framework 4.6.2. It didn't seem clear to me that the scenarios enumerated here were on .net framework but only on core. However, I am observing this behavior on 4.6.2 as well.

My client code:

var handler = new HttpClientHandler {					
	UseDefaultCredentials = true
};

_http = new HttpClient( handler );

My error response was HTTP 401. Full exception:

System.Net.Http.HttpRequestException: An error occurred while sending the request. ---> System.Net.WebException: The remote server returned an error: (401) Unauthorized. ---> System.ComponentModel.Win32Exception: The target principal name is incorrect
   at System.Net.NTAuthentication.GetOutgoingBlob(Byte[] incomingBlob, Boolean throwOnError, SecurityStatus& statusCode)
   at System.Net.NTAuthentication.GetOutgoingBlob(String incomingBlob)
   at System.Net.NegotiateClient.DoAuthenticate(String challenge, WebRequest webRequest, ICredentials credentials, Boolean preAuthenticate)
   at System.Net.NegotiateClient.Authenticate(String challenge, WebRequest webRequest, ICredentials credentials)
   at System.Net.AuthenticationManagerDefault.Authenticate(String challenge, WebRequest request, ICredentials credentials)
   at System.Net.AuthenticationState.AttemptAuthenticate(HttpWebRequest httpWebRequest, ICredentials authInfo)
   at System.Net.HttpWebRequest.CheckResubmitForAuth()
   at System.Net.HttpWebRequest.CheckResubmit(Exception& e, Boolean& disableUpload)
   --- End of inner exception stack trace ---
   at System.Net.HttpWebRequest.EndGetResponse(IAsyncResult asyncResult)
   at System.Net.Http.HttpClientHandler.GetResponseCallback(IAsyncResult ar)

Solution for me was to remove "Negotiate" from the list of providers in IIS app under "Authentication", "Windows Authentication". Thus, only "NTLM" exists in my list of Windows Auth providers.

The only way I could get the client to work, without changing the server's config was:

var handler = new HttpClientHandler {					
	//UseDefaultCredentials = true
       // Notable here that my "user" could NOT have the domain prefxed on it.
       // If I specified domain\user it would fail.  Leaving out domain worked.
	Credentials = new NetworkCredential("user","pass")
};

However, this defeats the purpose of using NTLM for me as I do not have the user / pass and want it to pass-through.

I modified IIS provider list and simply set UseDefaultCredentials = true.

I do not know if it matters on the server side, but I am hosting a .NET Core 2.1 Web API on IIS.

[karelz]

@futuralogic do you have problems with .NET Core? (2.1+)
Note that we do not track .NET Framework issues on GitHub (see our main page for guidance how to get help on .NET Framework)

[futuralogic]

@karelz Issue is not .NET core specific.

[karelz]

@futuralogic do you mean that it reproduces only on .NET Framework, or that reproduces on both .NET Framework and .NET Core? (which versions?)

[myfriendarnab]

I am still using the issue with .net core2.1 client while calling an endpoint with explicitly setting authentication scheme to "NTLM"
I am getting "The HTTP request is unauthorized with client authentication scheme 'Ntlm'. The authentication header received from the server was 'Negotiate, NTLM'" as error.

[davidsh]

@myfriendarnab It is unclear whether your issue is related to this one.

Please open a new GitHub issue. And please include enough information for us to reproduce your problem. I.e. attach a solution and/or attach WireShark, Fiddler or other traces so that we can accurately diagnose the problem.

[dfoulk]

@FenrisWolfAtMiddleEarth
Could you give a code example how NTLM Authentication with .NET Core 2.1 (Console-Application / WebApp) is working? I tried the following:

var handler = new HttpClientHandler();
            var credential = new NetworkCredential("", "", "");
            handler.AllowAutoRedirect = true;
            var ccache = new CredentialCache
            {
                { new Uri(baseUrl), "NTLM", CredentialCache.DefaultNetworkCredentials }
            };
            handler.Credentials = ccache;

            HttpClient client;
            client = new HttpClient(handler)
            {
                BaseAddress = new Uri("http://deagxcuewebt02.sickcn.net")
            };
            client.DefaultRequestHeaders.Accept.Clear();
            client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));


            HttpContent contentPost = new StringContent(parameterHeaddata, Encoding.UTF8, "application/json");
            HttpResponseMessage response = client.PostAsync(targetUrl_HeadData, contentPost).Result;`

But it doesn't work

We perform NTLM like this (this example is a snippet from a .NET Core app that uses the new HttpClientFactory- that's why HttpClient is a property here...):

public class ApiClient : IApiClient
{
    public ApiClient(NetworkCredential networkCredential = null)
    {
        // Use passed in credentials (if any), otherwise use the user's AD credentials
        var httpClientHandler = new HttpClientHandler
        {
            Credentials = networkCredential ?? CredentialCache.DefaultNetworkCredentials
        };

        Client = new HttpClient(httpClientHandler)
        {
            BaseAddress = new Uri("https://api.example.com/api/")
        };
    }

    public HttpClient Client { get; set; }
}

It's also worth mentioning that Windows Integrated Authentication appeared to not be working in our applications (even using this example) until we realized that Chrome and Firefox require some additional configuration to enable WIA. Seriously, we spent weeks debugging all to wind up finding this little gem by accident:

https://ping.force.com/Support/PingFederate/Integrations/How-to-configure-supported-browsers-for-Kerberos-NTLM

Finally, make sure that your Active Directory is configured properly (I think this tool is what we used to check):

https://www.microsoft.com/en-us/download/details.aspx?id=39046

It's a major headache to get this all working, but well-worth the effort now! Good luck!

[ejbaum]

I still had to use @IharYakimush 's solution in dotnet core 3. Is this because it is coded as designed and that is the solution or is there still a bug here? Without those fixes @IharYakimush highlighted, I got the app to work on my windows machine but not on my mac.

Here is my SO question before I found this thread: https://stackoverflow.com/questions/59271689/using-network-creds-in-dotnet-core-app-on-a-mac-using-httpclient

This is the post I am referring to: https://github.com/dotnet/corefx/issues/25988#issuecomment-412534360

[davidsh]

Is this because it is coded as designed and that is the solution or is there still a bug here?

The Mac has limitations. The Mac doesn't have an SPNEGO provider which is what HTTP Negotiate really requires. SPNEGO is the official protocol specification for what the 'Negotiate' authentication scheme means. SPNEGO will try Kerberos first. If that fails then it switches to NTLM automatically. In your case, your Mac is not part of the Kerberos environment/realm, so it has to use NTLM.

Since SPNEGO is not supported on the Mac, .NET Core will use Kerberos directly as part of handling the 'Www-authenticate: Negotiate' header. But that will fail here. When it fails, .NET Core will not try another 'Www-authenticate' header even though 'Www-authenticate: NTLM' is present.

The solution posted in the StackOverflow article is correct in this case. Instead of simply saying to use any authentication scheme provided by the server, the solution explicitly uses a CredentialCache object and binds the NetworkCredential object to a specific scheme, 'NTLM' in this case. And this works for this server because it is responding with two 'Www-authenticate' response headers. So, .NET Core will thus ignore the 'Www-authenticate: Negotiate' scheme and instead use the 'Www-authenticate: NTLM' scheme directly.

The Mac does support raw 'NTLM' protocol as long as the right NTLM plug-in is installed. From a pure technical perspective, the NTLM protocol is actually different from SPNEGO-NTLM. The latter uses SPNEGO packets wrapping NTLM protocol.

So, bottom line summary is that for Mac, this is the only solution here if your Mac is not on a real Kerberos realm with the server that you are communicating with.

[bpun]

@SakulRelda

This code works for me in .net core 2.1.
What I have not been able to solve, is reuse of connections. It appears .net Core will reconnect every time, which is not very usefull, when many requests are issued to the same endpoint

        public HttpClient Create()
        {
            var uri = new Uri(_urlRoot);
            var credentialsCache = new CredentialCache { { uri, "NTLM", CredentialCache.DefaultNetworkCredentials } };
            var handler = new HttpClientHandler { Credentials = credentialsCache };
            var httpClient = new HttpClient(handler) { BaseAddress = uri  };
            httpClient.DefaultRequestHeaders.ConnectionClose = false;
            httpClient.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));

            ServicePointManager.FindServicePoint(uri).ConnectionLeaseTimeout = 120 * 1000;  // Close connection after two minutes

            return httpClient;
        }

Thank you for saving my time (y). it works for me.

[Anhbta]

Same code above does not work in .net core 3.1 anymore. It works in .net core 2.x.

[Anhbta]

@SakulRelda
This code works for me in .net core 2.1.
What I have not been able to solve, is reuse of connections. It appears .net Core will reconnect every time, which is not very usefull, when many requests are issued to the same endpoint

        public HttpClient Create()
        {
            var uri = new Uri(_urlRoot);
            var credentialsCache = new CredentialCache { { uri, "NTLM", CredentialCache.DefaultNetworkCredentials } };
            var handler = new HttpClientHandler { Credentials = credentialsCache };
            var httpClient = new HttpClient(handler) { BaseAddress = uri  };
            httpClient.DefaultRequestHeaders.ConnectionClose = false;
            httpClient.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));

            ServicePointManager.FindServicePoint(uri).ConnectionLeaseTimeout = 120 * 1000;  // Close connection after two minutes

            return httpClient;
        }

Thank you for saving my time (y). it works for me.

This doesn't work in .net core 3.1.

[karelz]

@Anhbta please make sure to test a small standalone HelloWorld-style repro (e.g. using ServicePointManager above is no-op for HttpClient) and file a new issue.

[jacobmessner]

Is this because it is coded as designed and that is the solution or is there still a bug here?

The Mac has limitations. The Mac doesn't have an SPNEGO provider which is what HTTP Negotiate really requires. SPNEGO is the official protocol specification for what the 'Negotiate' authentication scheme means. SPNEGO will try Kerberos first. If that fails then it switches to NTLM automatically. In your case, your Mac is not part of the Kerberos environment/realm, so it has to use NTLM.

Since SPNEGO is not supported on the Mac, .NET Core will use Kerberos directly as part of handling the 'Www-authenticate: Negotiate' header. But that will fail here. When it fails, .NET Core will not try another 'Www-authenticate' header even though 'Www-authenticate: NTLM' is present.

The solution posted in the StackOverflow article is correct in this case. Instead of simply saying to use any authentication scheme provided by the server, the solution explicitly uses a CredentialCache object and binds the NetworkCredential object to a specific scheme, 'NTLM' in this case. And this works for this server because it is responding with two 'Www-authenticate' response headers. So, .NET Core will thus ignore the 'Www-authenticate: Negotiate' scheme and instead use the 'Www-authenticate: NTLM' scheme directly.

The Mac does support raw 'NTLM' protocol as long as the right NTLM plug-in is installed. From a pure technical perspective, the NTLM protocol is actually different from SPNEGO-NTLM. The latter uses SPNEGO packets wrapping NTLM protocol.

So, bottom line summary is that for Mac, this is the only solution here if your Mac is not on a real Kerberos realm with the server that you are communicating with.

This helped me resolve my issue or at least figure out what was going on. The frustrating thing was I had a very simple project that was in .Net Framework 4.7 and I wanted to move to .Net Core 3 or .Net 5. Everything was working in Framework, but started getting 401 errors when I moved to .Net Core. I would have expected a "PlatformNotSupportedException" rather than just a 401 unauthorized error.