Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

.NET Core 2.1 SocketsHttpHandler proxy authentication using Windows auth is broken #26397

Closed
bgiot opened this issue Jun 6, 2018 · 68 comments
Closed
Assignees
Labels
bug tenet-compatibility Incompatibility with previous versions or .NET Framework
Milestone

Comments

@bgiot
Copy link

bgiot commented Jun 6, 2018

When being behind a proxy requesting authentication, .net core 2.1 HttpClientHandler fails to properly authenticate against the proxy. The following code is working in .net core 2.0 but doesn't work anymore with .net core 2.1:

            var handler = new HttpClientHandler();
            handler.DefaultProxyCredentials = CredentialCache.DefaultCredentials;

            var client = new HttpClient(handler);
            var response = client.GetAsync("https://www.mocky.io/v2/5185415ba171ea3a00704eed").GetAwaiter().GetResult();
            var content = response.Content.ReadAsStringAsync().GetAwaiter().GetResult();
@stephentoub
Copy link
Member

stephentoub commented Jun 6, 2018

I ran:

using System;
using System.Net;
using System.Net.Http;
using System.Threading.Tasks;

class Program
{
    static void Main()
    {
        var handler = new HttpClientHandler();
        handler.DefaultProxyCredentials = CredentialCache.DefaultCredentials;

        var client = new HttpClient(handler);
        var response = client.GetAsync("https://www.mocky.io/v2/5185415ba171ea3a00704eed").GetAwaiter().GetResult();
        var content = response.Content.ReadAsStringAsync().GetAwaiter().GetResult();

        Console.WriteLine(content);
    }
}

on both 2.0 and 2.1 on Windows, and on both I get:

{"hello": "world"}

What am I supposed to get instead? What OS are you on? You're using the released version of 2.1 or a preview?

@stephentoub
Copy link
Member

When being behind a proxy requesting authentication

Ah, I'm not behind a proxy so my test wasn't valid...

@wfurt?

@wfurt
Copy link
Member

wfurt commented Jun 6, 2018

What platform and authentication scheme @dotconsulting ?
Best way would be positing full repro I can run.

@bgiot
Copy link
Author

bgiot commented Jun 6, 2018

Windows world. The proxy requests Windows authentication. I don't know exactly which scheme is used. But for example, with some tools we have to define proxy settings like this : http://{domain\user}:{password}@{proxy}:{port}. I don't know if it helps.

@karelz
Copy link
Member

karelz commented Jun 6, 2018

Would it help to get Fiddler/Wireshark traces? To see if we have any proxy info available at all.

@rmkerr
Copy link
Contributor

rmkerr commented Jun 6, 2018

Yeah, I think Fiddler or Wireshark traces would be really useful here.

@davidsh
Copy link
Contributor

davidsh commented Jun 6, 2018

Windows world. The proxy requests Windows authentication. I don't know exactly which scheme is used. But for example, with some tools we have to define proxy settings like this : http://{domain\user}:{password}@{proxy}:{port}. I don't know if it helps.

Putting the username and password in the URL is not supported. It is considered insecure. Our HTTP stacks in general don't support that way for both .NET Framework and .NET Core.

handler.DefaultProxyCredentials = CredentialCache.DefaultCredentials;

The code above here assumes two things. First, that the proxy settings are found via the "default system proxy" settings. On Windows machines, that typically means IE (Internet Explorer) settings. Second, using .DefaultCredentials only works when the proxy supports Windows authentication schemes such as Negotiate or NTLM. If the proxy only asks for Basic or Digest, then .DefaultCredentials doesn't work. You would have to use an explicit NetworkCredential object.

@bgiot
Copy link
Author

bgiot commented Jun 6, 2018

@davidsh:
Maybe back to the original message i wrote. The code properly works with .net core 2.0 and properly authenticates against the proxy. With .net core 2.1 we receive an "access denied" from the proxy saying no credentials are provided.
the "username and password url" sample was just to provide some hints about how the proxy works.
.DefaultCredentials works with .net core 2.0 but not with .net core 2.1. I think there is a regression bug.

@davidsh
Copy link
Contributor

davidsh commented Jun 6, 2018

.DefaultCredentials works with .net core 2.0 but not with .net core 2.1. I think there is a regression bug.

In the short term, you might want to opt-out of the new SocketsHttpHandler HTTP stack in .NET Core 2.1.

See the release notes for instructions on switching back to the HTTP stack (WinHttpHandler for Windows) that was used in .NET Core 2.0:

https://github.com/dotnet/core/blob/master/release-notes/2.1/2.1.0.md

Adding a line of code like this to your app will switch you back to the .NET Core 2.0 HTTP stack:

AppContext.SetSwitch("System.Net.Http.UseSocketsHttpHandler", false);

@bgiot
Copy link
Author

bgiot commented Jun 6, 2018

I'll try that workaround. I'll keep out posted. Thanks!

@rmkerr
Copy link
Contributor

rmkerr commented Jun 6, 2018

This isn't a case I considered when writing the default proxy string parser, so if WinHttp supports providing credentials that way there is a regression here.

@bgiot
Copy link
Author

bgiot commented Jun 6, 2018

@rmkerr:
The issue is with .DefaultProxyCredentials = CredentialCache.DefaultCredentials;
It seems that the credentials (the current logged user) are not passed to the proxy for authentication.

@davidsh
Copy link
Contributor

davidsh commented Jun 6, 2018

@dotconsulting

@rmkerr:
The issue is with .DefaultProxyCredentials = CredentialCache.DefaultCredentials;
It seems that the credentials (the current logged user) are not passed to the proxy for authentication.

Can you try an experiment? Instead of using CredentialCache.DefaultCredentials, can you use something like this new NetworkCredential("username", "password", "domain"); ?

Just want to see if the problem is .DefaultCredentials or a generalized problem with Windows Authentication for proxy handling in SocketsHttpHandler? Thx.

@bgiot
Copy link
Author

bgiot commented Jun 6, 2018

@davidsh
I already tried what you proposed. Same issue when using a specific NetworkCredential instance. I tried also to play with Proxy parameter with a WebProxy class but not better.

@davidsh
Copy link
Contributor

davidsh commented Jun 6, 2018

I already tried what you proposed. Same issue when using a specific NetworkCredential instance. I tried also to play with Proxy parameter with a WebProxy class but not better.

Thanks for trying. This looks like there is a bug with Windows Authentication for proxy credentials in SocketsHttpHandler.

To speed up our investigation, is it possible for you to take WireShark traces? I'd like to see if there is actual network traffic attempting Windows authentication (but failing) or just not attempting it at all. Please remove any confidential information from the traces. Thx.

@bgiot
Copy link
Author

bgiot commented Jun 6, 2018

@davidsh
Sure I'll try to produce a WireShark trace. I'm in Belgium, so now it's evening time (8:00pm). I'll be at the office tomorrow. Some advice to setup WireShark with .net core console ?
Thanks!

@davidsh
Copy link
Contributor

davidsh commented Jun 6, 2018

Some advice to setup WireShark with .net core console ?

There is nothing specific w/ .NET Core regarding WireShark. Once you set it up on your client machine, you can start the trace, run your app long enough to produce the failure and then stop tracing. Then you can process the trace and remove unneeded TCP sessions, etc. from it to reduce it down.

If you haven't used WireShark before, then you can go here to get more info:
https://www.wireshark.org/

@davidsh
Copy link
Contributor

davidsh commented Jun 6, 2018

I've reproduced the bug. SocketsHttpHandler isn't even beginning the Windows authentication exchange to the proxy.

I set up a repro using a test proxy server that uses Negotiate (it actually isn't a fully working implementation of Negotiate..but it's enough to see there is a problem with SocketsHttpHandler).

On .NET Core 2.1 using SocketsHttpHandler. I see this exchange thru the proxy:

W:\HappySockets>happysockets proxy server -Negotiate
Listening on port 3128 using authentication Negotiate ...
Connected: [::1]:1436
GET http://corefx-net.cloudapp.net/echo.ashx HTTP/1.1
Host: corefx-net.cloudapp.net

HTTP/1.0 407 Proxy Authentication Required
Proxy-agent: Netscape-Proxy/1.1
Proxy-Authenticate: Negotiate
Content-Length: 0
Connection: close
Date: Wed, 06 Jun 2018 19:06:24 GMT

So, you can see that SocketsHttpHandler didn't even try to send Windows credentials to the proxy. I.e., it never sent an initial "Proxy-Authorization: Negotiate Base64blob" to the proxy. It appeared to stop the HTTP exchange right away the proxy sent back "Proxy-Authenticate: Negotiate".

On .NET Core 2.1, using the AppContextSwitch to switch back to WinHttpHandler, I see this (note: I edited the Negotiate strings to remove confidential info).

W:\HappySockets>happysockets proxy server -Negotiate
Listening on port 3128 using authentication Negotiate ...
Connected: [::1]:1633
GET http://corefx-net.cloudapp.net/echo.ashx HTTP/1.1
Proxy-Connection: Keep-Alive
Accept-Encoding: peerdist
X-P2P-PeerDist: Version=1.1
X-P2P-PeerDistEx: MinContentInformation=1.0, MaxContentInformation=2.0
Host: corefx-net.cloudapp.net

HTTP/1.0 407 Proxy Authentication Required
Proxy-agent: Netscape-Proxy/1.1
Proxy-Authenticate: Negotiate
Content-Length: 0
Connection: close
Date: Wed, 06 Jun 2018 19:08:39 GMT

Connected: [::1]:1634
GET http://corefx-net.cloudapp.net/echo.ashx HTTP/1.1
Proxy-Connection: Keep-Alive
Accept-Encoding: peerdist
X-P2P-PeerDist: Version=1.1
X-P2P-PeerDistEx: MinContentInformation=1.0, MaxContentInformation=2.0
Host: corefx-net.cloudapp.net
Proxy-Authorization: Negotiate YH4GBisGAQUFA...

@davidsh davidsh changed the title .net core 2.1 HttpClientHandler proxy authentication broken .Net core 2.1 HttpClientHandler proxy authentication using Windows auth is broken Jun 6, 2018
@davidsh davidsh changed the title .Net core 2.1 HttpClientHandler proxy authentication using Windows auth is broken .NET Core 2.1 HttpClientHandler proxy authentication using Windows auth is broken Jun 6, 2018
@davidsh
Copy link
Contributor

davidsh commented Jun 6, 2018

It seems that SocketsHttpHandler is requiring the proxy to send back a 'Proxy-Support' header in order to do any Windows authentication to either the proxy or a destination server:

Proxy-Support: Session-Based-Authentication

This header normally describes how a proxy supports connection persistence to a destination server. That persistence is required in order to do Windows auth schemes such as Negotiate/NTLM to a destination server (i.e. 401). However, this header doesn't necessarily describe client HTTP stack requirements when doing Windows auth to the proxy server itself (i.e. 407).

When I modified my test proxy server to send back the "Proxy-Support: Session-Based-Authentication" response header, I got a different behavior from SocketsHttpHandler. It threw an exception:

System.Net.Http.HttpRequestException: Authentication failed because the connection could not be reused.
at System.Net.Http.HttpConnection.DrainResponseAsync(HttpResponseMessage response)
at System.Net.Http.AuthenticationHelper.SendWithNtAuthAsync(HttpRequestMessage request, Uri authUri, ICredentials credentials, Boolean isProxyAuth, HttpConnection connection, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)
at System.Net.Http.AuthenticationHelper.SendWithAuthAsync(HttpRequestMessage request, Uri authUri, ICredentials credentials, Boolean preAuthenticate, Boolean isProxyAuth, Boolean doRequestAuth, HttpConnectionPool pool, CancellationToken cancellationToken)
at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
at System.Net.Http.DiagnosticsHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
at System.Net.Http.HttpClient.FinishSendAsyncBuffered(Task`1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)
at NetCoreProxyTest.Program.Main(String[] args) in C:\Users\dshulman\Documents\Visual Studio 2017\Projects\NetCoreProxyTest\NetCoreProxyTest\Program.cs:line 27

So, there is probably a few different things to fix in SocketsHttpHandler for this scenario.

System.Net.Http.HttpRequestException: Authentication failed because the connection could not be reused.

This exception that SocketsHttpHandler is throwing isn't valid in this scenario. Many servers including proxy servers will close the connection the first time they send back a 401 or 407. The exception shouldn't be thrown in those cases. SocketsHttpHandler should just use a new connection. Only in cases during the multi-leg back-and-forth of the Windows auth and the connection being closed should result in an error. And it's not clear that error should be an exception but rather just let the 401/407 be the final thing.

@bgiot
Copy link
Author

bgiot commented Jun 7, 2018

I tried to use

AppContext.SetSwitch("System.Net.Http.UseSocketsHttpHandler", false);

But then I got an exception coming from WinHttpAuthHelper class:

System.NullReferenceException: Object reference not set to an instance of an object.
at System.Net.Http.WinHttpAuthHelper.CheckResponseForAuthentication(WinHttpRequestState state, UInt32& proxyAuthScheme, UInt32& serverAuthScheme)
at System.Net.Http.WinHttpHandler.StartRequest(WinHttpRequestState state)
at System.Net.Http.HttpClient.FinishSendAsyncBuffered(Task`1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)
at TestProxyAuthentication.Program.Main(String[] args) in C:\Users\giotb\source\temp\TestProxyAuthentication\TestProxyAuthentication\Program.cs:line 21

Here is the code (line 21 is the client.GetAsync):

    class Program
    {
        static void Main(string[] args)
        {

            AppContext.SetSwitch("System.Net.Http.UseSocketsHttpHandler", false);
            var handler = new HttpClientHandler();
            handler.DefaultProxyCredentials = CredentialCache.DefaultCredentials;

            var client = new HttpClient(handler);
            var response = client.GetAsync("https://www.mocky.io/v2/5185415ba171ea3a00704eed").GetAwaiter().GetResult();
            var content = response.Content.ReadAsStringAsync().GetAwaiter().GetResult();

        }
    }

@dyamo
Copy link

dyamo commented Jun 7, 2018

I've experienced the same issue with proxy authentication in Windows 10 with a proxy autoconfigured via a pac file.

In 2.0 this code works.

var handler = new HttpClientHandler();
handler.DefaultProxyCredentials = CredentialCache.DefaultCredentials;

It doesn't work in 2.1, regardless of whether 'System.Net.Http.UseSocketsHttpHandler' is true or false, although an exception is thrown if it is false.

I've tried a few combination of settings to get proxy authentication to work in 2.1. The code below was the only thing that appeared to work. Not ideal as the proxy settings had to be specified explicitly.

AppContext.SetSwitch("System.Net.Http.UseSocketsHttpHandler", false);
var httpClient = new HttpClient(
new HttpClientHandler()
{
	Proxy = new WebProxy("http://proxy:8080") { UseDefaultCredentials = true }
});

@DaniilSokolyuk
Copy link

DaniilSokolyuk commented Jun 7, 2018

NuGet affected too (dotnet restore)
NuGet/Home#6978

@davidsh
Copy link
Contributor

davidsh commented Jun 7, 2018

@dyamo

It doesn't work in 2.1, regardless of whether 'System.Net.Http.UseSocketsHttpHandler' is true or false, although an exception is thrown if it is false.

This is suprising. When the AppContext switch is used to switch off SocketsHttpHandler, proxy detection and use is done by the original WinHttpHandler HTTP stack. I haven't seen any problems with that.

Can you show a repro and the callstack / exception you're getting? Thx.

@davidsh
Copy link
Contributor

davidsh commented Jun 7, 2018

@bgiot

But then I got an exception coming from WinHttpAuthHelper class:
System.NullReferenceException: Object reference not set to an instance of an object.
at System.Net.Http.WinHttpAuthHelper.CheckResponseForAuthentication(WinHttpRequestState state, UInt32& proxyAuthScheme, UInt32& serverAuthScheme)
at System.Net.Http.WinHttpHandler.StartRequest(WinHttpRequestState state)
at System.Net.Http.HttpClient.FinishSendAsyncBuffered(Task`1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)
at TestProxyAuthentication.Program.Main(String[] args) in C:\Users\giotb\source\temp\TestProxyAuthentication\TestProxyAuthentication\Program.cs:line 21

Can you describe in more detail how the proxy is configured in your scenario? What are the IE settings? Is it auto-discovery...i.e. looking for a PAC file on the network? What kind of proxy server is it? Etc. Thx.

@bgiot
Copy link
Author

bgiot commented Jun 7, 2018

@davidsh
"Automatically detect settings" is checked in the IE settings. So, probably a pac file loaded from the network.
(Please note - in case of - I'm the creator of this issue entry. I just changed my profile from dotconsulting to bgiot; thanks!)

@davidsh
Copy link
Contributor

davidsh commented Jun 7, 2018

What about the kind of proxy server? Is it using Negotiate or NTLM (or both)? Can you do some WireShark traces of this? We'd like to understand the error you're getting when using WinHttpHandler. There are probably 2 different bugs here: one with SocketsHttpHandler that we understand already, second with WinHttpHandler that we can't repro yet.

@davidsh
Copy link
Contributor

davidsh commented Jun 7, 2018

@bgiot

I was able to reproduce this exception in WinHttpHandler when I use a PAC file. I used explicit PAC file settings instead of "automatic discovery" since it was easier to repro it that way.

image

where the contents of the PAC file are a simple function to a localhost test proxy server that requires authentication.

function FindProxyForURL(url, host) { return "PROXY [::1]:3128"; }

But I got the same NullReferenceException.

image

@davidsh
Copy link
Contributor

davidsh commented Jun 7, 2018

We will probably split this bug into separate bugs since the root causes are different between WinHttpHandler vs. SocketsHttpHandler.

@CJHarmath
Copy link

This causes PowerShell Core 6.1 to fail with SPNego as well, so it's not just with Proxies but seems to be an issue with Negotiate auth in general.

i.e.: SocketsHttpHandler not using Negotiate / SPNego

This PowerShell issue has some more details as well.
PowerShell/PowerShell#7801

Testing

As per testing this, IIS can be configured to only support Negotiate:Kerberos as the windows auth provider. i.e.: disable NTLM. So maybe you can do a quicker / easier / cheaper setup with IIS to make sure that SocketsHttpHandler works with negotiate.

@PatrickLang has a work in progress for how to setup containers with gMSA's SPN's, etc for a Kerberos setup MicrosoftDocs/Virtualization-Documentation#592

@karelz
Copy link
Member

karelz commented Sep 17, 2018

@csharmath I am not 100% sure how is your comment related to this issue.
Given that the main investigation with proxies has been moved into separate issues dotnet/corefx#30327, dotnet/corefx#30330, and both have been addressed in 2.1.x, I would recommend to:

  • Try your repro on 2.1.x (daily builds as the last fix went into not yet-released 2.1.5 release).
  • If it still repros, isolate a repro without PowerShell (as part of minimizing the problem) and file a new bug.

@tebeco
Copy link

tebeco commented Sep 17, 2018

i think the biggest issue is that now SocketsHttpHandler supports the Negociate it still does not goes for PlatformSpecific api to get the proxy define at the OS level

in Corporation don’t ever try to use env var like HTTP_PROXY (even with NO_PROXY)

for example my current client have 3 proxy url and use a PAC script to resolve the proxy url to use based on the url we try to reach

i would expect some code to check the current Platform and use respective standard in each one of them to actually get that last piece of code working

every single third part on nuget using HttpClieny is broken right now if they did not designed the API so you can pass the proxy url (wich is bad for PAC scenario)

the easiest example today is dotnet restore —force
as we use internal feed (Lan) + vsts private feed (subnet) + nuget.org
nuget use an different proxy url than the vsts private feed (LAN vs DMZ/ etc ...,)

the result is that dotnet restore is fully broken
so basically every time i fork an msft repo and i want to restore it i have to ask the team responsible of private feed to create my a proxy for every https://dotnet.myget.orf/F/whatever

@karelz
Copy link
Member

karelz commented Sep 17, 2018

@tebeco good points (esp. on the dotnet restore scenario). Let's take the discussion into separate issue. We'll be interested in concrete information on which parts of proxy don't work today on which OS.
To my best knowledge we addressed all known problems and differences from .NET Core 2.0 in 2.1.5 (to be released). If you are aware of scenarios which still don't work (with 2.1.5 daily builds), please file them as issues, so that we can address them. Thanks!

As I mentioned earlier in https://github.com/dotnet/corefx/issues/30166#issuecomment-396725994, the proxy support came in late. We had the option to not support proxies at all in 2.1, or to make an honest attempt with potential tail of bugs. While it is not ideal, it went quite ok IMO - we had a few bugs reported, couple of dozens customers impacted, but it was not super-massive + there is always opt-out to the old way.

@tebeco
Copy link

tebeco commented Sep 17, 2018

To my best knowledge we addressed all known problems and differences from .NET Core 2.0 in 2.1.5 (to be released)

you lost me here, 2.1.5 runtime (hence the TBR ?) ?
something is this release should affect the http handlers ?

there is always opt-out to the old way.

you mean the env var to disable SocketHttpHandler ? i think i tried it already and did not seem to affect the restore

not sure what i missed

@karelz
Copy link
Member

karelz commented Sep 17, 2018

you lost me here, 2.1.5 runtime (hence the TBR ?) ?
something is this release should affect the http handlers ?

AFAIK all known proxy related bugs are fixed in 2.1.5 (to be released).

you mean the env var to disable SocketHttpHandler? i think i tried it already and did not seem to affect the restore

Yes, that is what I meant. Maybe dotnet/nuget are doing something special and use SocketsHttpHandler directly. That would be specific to them - please file a new bug (in CLI repo I think) and let's get to the bottom of this there.

@tebeco
Copy link

tebeco commented Sep 17, 2018

i’m not sure moving to the CLI is a good idea
i feel like it should be dealt here

for example when using AddAzureKeyVault on aspnet core bootstrap i have to implement my own IKeyVaultClient

it could break CosmosDb api (i did not check if it does not allows to feed an httpClient)

same question for EfCore

@tebeco
Copy link

tebeco commented Sep 17, 2018

i just noticed that you saw this one
https://github.com/dotnet/corefx/issues/24574

i could try 2.1.5 but honestly the main issue is the loss of windows specific code
so unless all Platform align on a real standard (and i mean a standard used by corporation)
if not, coreFx will have to use Platefrom specific api
something like the Windows Compatibility pack :

the httpclient is instanciated using reflection to check if it’s running using fullfx or netcore to call a private/internal ctor. Why not resolving the proxy at that time too ?

@CJHarmath
Copy link

@karelz thanks for pointing out that I am mixing a different topic to this proxy only issue, adding a new issue about SPNego.

@tebeco
Copy link

tebeco commented Sep 20, 2018

@karelz so yesterday we tried to use CosmosDb with DocumentDb client api
i’m not sure we found the actual repo for DocumentDb.Core (the netstandard package)
still after décompilation it seems that a private handler is is nested next to the httpclient and all the API is closed

if you look for “DocumentDb” you’ll find answers telling you to do :

<system.net>
    <defaultProxy enabled="true" useDefaultCredentials="true">
        <proxy usesystemdefault="True" />
    </defaultProxy>
</system.net>

this is why this issue probably need to be dealt in coreFx and not over every single repo
i’m surprised there’s not much people around asking why lots of there third part tool stopped working when switching to netcoreapp ^^

  • either corporation don’t bother anymore with proxy (i highly doubt it)
  • did they rolled back to fullFx
  • did they used tools like CNTLM (wich is really bad)
  • only few moved to netcoreapp (weird because i would say no)

so i wonder as there was on Opt_out from SocketHandler
if there no way to add an opt_in only to inject the proxy from si does settings in any way

  • Context switch ?
  • platform specific code using compat pack code ?
  • a Configuration Section (kinda like the fullfx way)

again ... we cannot control third part that were released and was never designed to take care of proxy properly

@karelz
Copy link
Member

karelz commented Sep 20, 2018

@tebeco it seems that we have duplicate conversation here and in #23811. It is also confusing to see several scenarios thrown into the same pile and mixed.
Let's try to untangle things:

The main problem seems to be that proxy on Windows does not use default credentials unless you explicitly ask for it via HttpClientHandler.DefaultProxyCredentials = CredentialCache.DefaultCredentials. That can be problematic in 3rd party libraries if they do not expose HttpClientHandler configuration.
The scenario seems to work with disabled SocketsHttpHandler on Windows. It also works fine on .NET Framework if you set config file to <defaultProxy enabled="true" useDefaultCredentials="true">.
The ask is to provide a way (default or explicit) to set process-wide proxy credentials (or even proxy settings), without need to pass the information programatically to each HttpClientHandler created in the process (incl. those created by 3rd party libraries).

Current workaround: Disable SocketsHttpHandler in the app via env. settings.

Is that good summary?
If yes, we should file a new issue to clearly track the feature ask. It seems legit to me and hinted in other issues as well. It could use good title and clear description though.

Did I miss other problems / bugs / suggestions in your replies?

@tebeco
Copy link

tebeco commented Sep 20, 2018

I did not have time yesterday to disable SocketHttpHandler it is supposed to fallback to WinHttpHandler IIRC
I think i tried when sdk 2.1.400 was released but at that time i did not managed to make it work either (still 407), i might have missed something

On thursday we had a contractor for Microsoft that was introduced to this issue.
The answer was simple : as we are in 2018 it is time for corporation to change their network / proxy and they should use transparent proxy, as we speak he already had that discussion internally with ... i don’t know who and the conclusion that there is no plan to add Platform specific (windows here) to read proxy settings from IE

side note : the process that run on premise are hosting in windows services lets forget that part

@karelz
Copy link
Member

karelz commented Sep 20, 2018

@tebeco I don't quite understand what you meant in your last post. Can you please rephrase /explain in details what you meant?

make proxy detect IE settings

Currently we use IE settings. Why do you think we don't?

they try to convince corporation to use transparent proxy unless they (corpo) prouve msft that they (msft) will loose money <== can you confirm / inform that ?

No idea what you meant here. I would suggest to stick to facts - what works vs. what does not. Let's avoid any speculations what MS is trying to convince developers to do.
I can speak on behalf of Networking area - believe me, we are just trying to light up all important scenarios (in priority order) and we are trying to understand which scenarios we missed. BTW: You're the first one to poke into this scenario.

also i think i did try to opt out from sockethandler with no luck

Can you please double check? If it never worked even in .NET Core 2.0 (same as disabled SocketsHttpHandler), that would be good to know.

does it work for windows service without app.config?

What should work where without app.config? On .NET Framework I assume app.config is necessary.

@tebeco
Copy link

tebeco commented Sep 22, 2018

@karelz done, sry about that

@JohannesHoppe
Copy link

JohannesHoppe commented Oct 2, 2018

3rd party libraries

This also includes the Azure WebJobs SDK. I was never able to get it running behind our evil company proxy (System.Net.Http.UseSocketsHttpHandler does not work). For several years .Net Core is an endless source of frustration. 👎

@tolgabalci
Copy link

In case anyone else runs in to this, using HttpClient to call out to WEBAPIs inside of IIS was causing the error:

HttpRequestException: Response status code does not indicate success: 403 (This URL is not allowed to be proxied)

When troubleshooting with Fiddler, the error would go away and everything would work fine.

Solution was to add the following to startup.cs:
AppContext.SetSwitch("System.Net.Http.UseSocketsHttpHandler", false);

@davidshen84
Copy link

dotnet --version
2.1.700

I am still seeing this issue, and my company does not have a proxy. Any update on the status of this bug?

@davidsh
Copy link
Contributor

davidsh commented Jul 3, 2019

I am still seeing this issue, and my company does not have a proxy. Any update on the status of this bug?

This issue is closed. It was fixed in .NET Core 3.0 which is available in Preview right now.

@vologar
Copy link

vologar commented Nov 25, 2019

Same issue here. Trying to connect from Azure function emulator to Azure blob storage. TCP Viewer shows the calls are attempted directly to Azure ip without any knowledge of Proxy settings. Net version woks as expected. Setting AppContext.SetSwitch("System.Net.Http.UseSocketsHttpHandler", false) resolved the issue, but we had to spend hours to understand the root cause. Has it ever been tested from behind the firewalls ?

@vologar
Copy link

vologar commented Nov 25, 2019

Azure Functions SDK still uses .Net Core 2.*

@davidsh
Copy link
Contributor

davidsh commented Nov 25, 2019

This issue was only fixed in .NET Core 3.0. It was not fixed in .NET Core 2.*. That's why you still see the problem.

When Azure Functions SDK is upgraded to .NET Core 3.x, then the problem should be resolved for you.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug tenet-compatibility Incompatibility with previous versions or .NET Framework
Projects
None yet
Development

No branches or pull requests