-
Notifications
You must be signed in to change notification settings - Fork 511
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Bring HttpClient from CoreFX. #7346
Conversation
This bumps Mono to use mono/mono#17645 (which is the 2019-10 backport of mono/mono#17628). The big user-visible change is in regards to certificate validation, everything below are just some minor adjustments to tests. ## SocketsHttpHandler CoreFX uses a completely new `HttpClientHandler` implementation called `SocketsHttpHandler`, which you can find at https://github.com/dotnet/corefx/tree/release/3.0/src/System.Net.Http/src/System/Net/Http/SocketsHttpHandler. Since this is not based on the web stack anymore, it does not use any of the related APIs such as `ServicePointManager` or `WebException`. ## Certificate Validation Changes There is a new API called `HttpClientHandler.ServerCertificateCustomValidationCallback`. - https://docs.microsoft.com/en-us/dotnet/api/system.net.http.httpclienthandler.servercertificatecustomvalidationcallback?view=netframework-4.8 - https://github.com/dotnet/corefx/blob/c1778515a3bee34cc09c757b5563d0af0c8b1e99/src/System.Net.Http/src/System/Net/Http/HttpClientHandler.Unix.cs#L154 - https://github.com/dotnet/corefx/blob/c1778515a3bee34cc09c757b5563d0af0c8b1e99/src/System.Net.Http/src/System/Net/Http/HttpClientHandler.Windows.cs#L383 The `ServicePointManager.ServerCertificateValidationCallback` is no longer invoked and on certificate validation failure, `AuthenticationException` (from `System.Security.Authentication`) is thrown instead of `WebException`. At the moment, the `NSUrlSessionHandler` still uses it's own validation callback and also still throws `WebException` on failure; we should probably look into making this consistent with the other handlers. ## Minor adjustments related to internal Mono APIs * `HttpContent.SerializeToStreamAsync()` is now `protected` (changed from `protected internal`). - src/Foundation/NSUrlSessionHandler.cs: changed overload accordingly. - src/System.Net.Http/CFContentStream.cs: likewise. * `HttpHeaders.GetKnownHeaderKind()` is an internal Mono API. There is a new internal API called `System.Net.Http.PlatformHelper.IsContentHeader(key)` which exists in both the old as well as the new implementation. The correct way of doing it with the CoreFX handler is `HeaderDescriptor.TryGet (key, out var descriptor) && descriptor.HeaderType == HttpHeaderType.Content` ## Minor adjustments to tests. * `HttpClientHandler.MaxRequestContentBufferSize` is now longer supported, you can set it to any non-negative value, the getter will always return 0. See https://github.com/dotnet/corefx/blob/c1778515a3bee34cc09c757b5563d0af0c8b1e99/src/System.Net.Http/src/System/Net/Http/HttpClientHandler.Core.cs#L18. - tests/linker/ios/link sdk/HttpClientHandlerTest.cs: removed assertion from test. * `HttpMessageInvoker.handler` is a `protected private` field - in the CoreFX handler, it is called `_handler` and `private`. This is accessed via reflection by some of the tests, which are now using the new name. - tests/mmptest/src/MMPTest.cs: here - tests/mtouch/MTouch.cs: here * tests/monotouch-test/System.Net.Http/MessageHandlers.cs: Adjust `RejectSslCertificatesServicePointManager` to reflect the certificate validation changes described above. - FIXME: There was an `Assert.Ignore()` related to `NSUrlSessionHandler` and macOS 10.10; I removed that to reenable the test because the description linked to an old issue in the private repo that was referenced by several "Merged" PR's, so it looked to me that this might have already been fixed - and I also didn't see why it would fail there.
Build failure 🔥 Build failed 🔥 |
Changes LGTM WRT we can't expose this in However we can document how to map it with the existing certificate handler - so the price to pay becomes optional for people wanting to share/re-use the same code. |
Would it make sense to expose that as extension method? Presumably it would get linked out if not used. |
@filipnavara no, in this case that would not help. The reason is that it's a delegate used as a callback. IOW the handler code (not the user code) has to create an Those types will recursively mark a lot of other (mostly security/crypto) types that most application are likely not using (if using the This was the problem with the original approach in https://github.com/xamarin/xamarin-macios/pull/5733/files#diff-7f619be41b43eaaed811ff8b330f769cR511 that resulted in a 1.3MB app size increase. The (existing) native handler version of the callback workaround this by using fewer types (mostly only The trick is (or will be) to set that (existing) delegate to call the one defined in the new handler, i.e. get the |
Perhaps I should have been more clear but that approach still sounds like what I had in mind:
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👀
@spouliot we can adde the extension method, I can create a PR with it. |
build |
Build failure 🔥 Build failed 🔥 |
Build failure 🔥 Build failed 🔥 |
❌ Build was (probably) aborted 🔥 Jenkins job (on internal Jenkins) failed in stage(s) 'Build' 🔥 : hudson.AbortException: script returned exit code 2 🔥 Build failed 🔥 |
❌ Build was (probably) aborted 🔥 Jenkins job (on internal Jenkins) failed in stage(s) 'Packaging' 🔥 : hudson.AbortException: script returned exit code 2 ✅ Build succeeded |
Build failure |
build |
Build failure |
build |
Build failure |
build |
Build failure ✅ Build succeeded |
❌ Build was (probably) aborted 🔥 Jenkins job (on internal Jenkins) failed in stage(s) 'Packaging' 🔥 : hudson.AbortException: script returned exit code 2 ✅ Build succeeded |
Build failure ✅ Build succeeded |
❌ Build was (probably) aborted 🔥 Jenkins job (on internal Jenkins) failed in stage(s) 'Packaging' 🔥 : hudson.AbortException: script returned exit code 2 ✅ Build succeeded |
Build failure Test results2 tests failed, 886 tests' device not found, 161 tests passed.Failed tests
|
Packaging fails due to https://github.com/xamarin/maccore/issues/2053. @mandel-macaque could you have a look? |
Merging, and I'll fix the pkg issue in a diff PR. |
Build failure |
❌ Build was (probably) aborted 🔥 Jenkins job (on internal Jenkins) failed in stage(s) 'Build' 🔥 : hudson.AbortException: script returned exit code 2 |
## Miscellaneous fixes * Fixed `/Users/builder/jenkins/workspace/xamarin-macios/xamarin-macios/builds/mono-ios-sdk-destdir/ios-sources/external/linker/src/linker/Linker.Steps/OutputStep.cs(110,15): error CS0246: The type or namespace name ‘OutputException’ could not be found (are you missing a using directive or an assembly reference?) [/Users/builder/jenkins/workspace/xamarin-macios/xamarin-macios/tools/mmp/mmp.csproj]` * Changed the name of the method that is used from linker. Because of this commit dotnet/linker@6be2677 * Added `OutputException.cs` file on `mtouch.csproj`. * Removing enter_gc_safe and exit_gc_safe because now it's already gc_safe in this part of code, after a mono change. * Added known exceptions to LLVM exception list. * Needs `ifdef` because of this mono/mono#17260. * Bump MIN_MONO_VERSION to 6.8.0.41 and point MIN_MONO_URL to the PR. * Add ENABLE_IOS=1 and ENABLE_MAC=1. * Added switch to disable packaged mono build * [Tests] Ignore tests that fail on 32b. Ignore the test on 32b, and filled issue: mono/mono#17752 * [Tests] Ignore a couple of tests causing OOM. Hopefully fixes xamarin/maccore#1659 for good. * Ignore `MM0135` test on Catalina+ because it needs Xcode 9.4. * [monotouch-test] Add null checks for teardown when test didn't run because of a too early OS version. * [CFNetwork]: Http 2.0 requires OS X 10.11 or later. Check whether `_HTTPVersion2_0` is available and fallback to HTTP 1.1 otherwise. ## Bring HttpClient from CoreFX * #7346 * This bumps Mono to use mono/mono#17645 (which is the 2019-10 backport of mono/mono#17628). * The big user-visible change is in regards to certificate validation, everything below are just some minor adjustments to tests. ### SocketsHttpHandler CoreFX uses a completely new `HttpClientHandler` implementation called `SocketsHttpHandler`, which you can find at https://github.com/dotnet/corefx/tree/release/3.0/src/System.Net.Http/src/System/Net/Http/SocketsHttpHandler. Since this is not based on the web stack anymore, it does not use any of the related APIs such as `ServicePointManager` or `WebException`. ### Certificate Validation Changes There is a new API called `HttpClientHandler.ServerCertificateCustomValidationCallback`. - https://docs.microsoft.com/en-us/dotnet/api/system.net.http.httpclienthandler.servercertificatecustomvalidationcallback?view=netframework-4.8 - https://github.com/dotnet/corefx/blob/c1778515a3bee34cc09c757b5563d0af0c8b1e99/src/System.Net.Http/src/System/Net/Http/HttpClientHandler.Unix.cs#L154 - https://github.com/dotnet/corefx/blob/c1778515a3bee34cc09c757b5563d0af0c8b1e99/src/System.Net.Http/src/System/Net/Http/HttpClientHandler.Windows.cs#L383 The `ServicePointManager.ServerCertificateValidationCallback` is no longer invoked and on certificate validation failure, `AuthenticationException` (from `System.Security.Authentication`) is thrown instead of `WebException`. At the moment, the `NSUrlSessionHandler` still uses it's own validation callback and also still throws `WebException` on failure; we should probably look into making this consistent with the other handlers. ### Minor adjustments related to internal Mono APIs * `HttpContent.SerializeToStreamAsync()` is now `protected` (changed from `protected internal`). - src/Foundation/NSUrlSessionHandler.cs: changed overload accordingly. - src/System.Net.Http/CFContentStream.cs: likewise. * `HttpHeaders.GetKnownHeaderKind()` is an internal Mono API. There is a new internal API called `System.Net.Http.PlatformHelper.IsContentHeader(key)` which exists in both the old as well as the new implementation. The correct way of doing it with the CoreFX handler is `HeaderDescriptor.TryGet (key, out var descriptor) && descriptor.HeaderType == HttpHeaderType.Content` ### Minor adjustments to tests. * `HttpClientHandler.MaxRequestContentBufferSize` is now longer supported, you can set it to any non-negative value, the getter will always return 0. See https://github.com/dotnet/corefx/blob/c1778515a3bee34cc09c757b5563d0af0c8b1e99/src/System.Net.Http/src/System/Net/Http/HttpClientHandler.Core.cs#L18. - tests/linker/ios/link sdk/HttpClientHandlerTest.cs: removed assertion from test. * `HttpMessageInvoker.handler` is a `protected private` field - in the CoreFX handler, it is called `_handler` and `private`. This is accessed via reflection by some of the tests, which are now using the new name. - tests/mmptest/src/MMPTest.cs: here - tests/mtouch/MTouch.cs: here * tests/monotouch-test/System.Net.Http/MessageHandlers.cs: Adjust `RejectSslCertificatesServicePointManager` to reflect the certificate validation changes described above. - FIXME: There was an `Assert.Ignore()` related to `NSUrlSessionHandler` and macOS 10.10; I removed that to reenable the test because the description linked to an old issue in the private repo that was referenced by several "Merged" PR's, so it looked to me that this might have already been fixed - and I also didn't see why it would fail there.
* Fixed `/Users/builder/jenkins/workspace/xamarin-macios/xamarin-macios/builds/mono-ios-sdk-destdir/ios-sources/external/linker/src/linker/Linker.Steps/OutputStep.cs(110,15): error CS0246: The type or namespace name ‘OutputException’ could not be found (are you missing a using directive or an assembly reference?) [/Users/builder/jenkins/workspace/xamarin-macios/xamarin-macios/tools/mmp/mmp.csproj]` * Changed the name of the method that is used from linker. Because of this commit dotnet/linker@6be2677 * Added `OutputException.cs` file on `mtouch.csproj`. * Removing enter_gc_safe and exit_gc_safe because now it's already gc_safe in this part of code, after a mono change. * Added known exceptions to LLVM exception list. * Needs `ifdef` because of this mono/mono#17260. * Bump MIN_MONO_VERSION to 6.8.0.41 and point MIN_MONO_URL to the PR. * Add ENABLE_IOS=1 and ENABLE_MAC=1. * Added switch to disable packaged mono build * [Tests] Ignore tests that fail on 32b. Ignore the test on 32b, and filled issue: mono/mono#17752 * [Tests] Ignore a couple of tests causing OOM. Hopefully fixes xamarin/maccore#1659 for good. * Ignore `MM0135` test on Catalina+ because it needs Xcode 9.4. * [monotouch-test] Add null checks for teardown when test didn't run because of a too early OS version. * [CFNetwork]: Http 2.0 requires OS X 10.11 or later. Check whether `_HTTPVersion2_0` is available and fallback to HTTP 1.1 otherwise. * xamarin#7346 * This bumps Mono to use mono/mono#17645 (which is the 2019-10 backport of mono/mono#17628). * The big user-visible change is in regards to certificate validation, everything below are just some minor adjustments to tests. CoreFX uses a completely new `HttpClientHandler` implementation called `SocketsHttpHandler`, which you can find at https://github.com/dotnet/corefx/tree/release/3.0/src/System.Net.Http/src/System/Net/Http/SocketsHttpHandler. Since this is not based on the web stack anymore, it does not use any of the related APIs such as `ServicePointManager` or `WebException`. There is a new API called `HttpClientHandler.ServerCertificateCustomValidationCallback`. - https://docs.microsoft.com/en-us/dotnet/api/system.net.http.httpclienthandler.servercertificatecustomvalidationcallback?view=netframework-4.8 - https://github.com/dotnet/corefx/blob/c1778515a3bee34cc09c757b5563d0af0c8b1e99/src/System.Net.Http/src/System/Net/Http/HttpClientHandler.Unix.cs#L154 - https://github.com/dotnet/corefx/blob/c1778515a3bee34cc09c757b5563d0af0c8b1e99/src/System.Net.Http/src/System/Net/Http/HttpClientHandler.Windows.cs#L383 The `ServicePointManager.ServerCertificateValidationCallback` is no longer invoked and on certificate validation failure, `AuthenticationException` (from `System.Security.Authentication`) is thrown instead of `WebException`. At the moment, the `NSUrlSessionHandler` still uses it's own validation callback and also still throws `WebException` on failure; we should probably look into making this consistent with the other handlers. * `HttpContent.SerializeToStreamAsync()` is now `protected` (changed from `protected internal`). - src/Foundation/NSUrlSessionHandler.cs: changed overload accordingly. - src/System.Net.Http/CFContentStream.cs: likewise. * `HttpHeaders.GetKnownHeaderKind()` is an internal Mono API. There is a new internal API called `System.Net.Http.PlatformHelper.IsContentHeader(key)` which exists in both the old as well as the new implementation. The correct way of doing it with the CoreFX handler is `HeaderDescriptor.TryGet (key, out var descriptor) && descriptor.HeaderType == HttpHeaderType.Content` * `HttpClientHandler.MaxRequestContentBufferSize` is now longer supported, you can set it to any non-negative value, the getter will always return 0. See https://github.com/dotnet/corefx/blob/c1778515a3bee34cc09c757b5563d0af0c8b1e99/src/System.Net.Http/src/System/Net/Http/HttpClientHandler.Core.cs#L18. - tests/linker/ios/link sdk/HttpClientHandlerTest.cs: removed assertion from test. * `HttpMessageInvoker.handler` is a `protected private` field - in the CoreFX handler, it is called `_handler` and `private`. This is accessed via reflection by some of the tests, which are now using the new name. - tests/mmptest/src/MMPTest.cs: here - tests/mtouch/MTouch.cs: here * tests/monotouch-test/System.Net.Http/MessageHandlers.cs: Adjust `RejectSslCertificatesServicePointManager` to reflect the certificate validation changes described above. - FIXME: There was an `Assert.Ignore()` related to `NSUrlSessionHandler` and macOS 10.10; I removed that to reenable the test because the description linked to an old issue in the private repo that was referenced by several "Merged" PR's, so it looked to me that this might have already been fixed - and I also didn't see why it would fail there.
* Fixed `/Users/builder/jenkins/workspace/xamarin-macios/xamarin-macios/builds/mono-ios-sdk-destdir/ios-sources/external/linker/src/linker/Linker.Steps/OutputStep.cs(110,15): error CS0246: The type or namespace name ‘OutputException’ could not be found (are you missing a using directive or an assembly reference?) [/Users/builder/jenkins/workspace/xamarin-macios/xamarin-macios/tools/mmp/mmp.csproj]` * Changed the name of the method that is used from linker. Because of this commit dotnet/linker@6be2677 * Added `OutputException.cs` file on `mtouch.csproj`. * Removing enter_gc_safe and exit_gc_safe because now it's already gc_safe in this part of code, after a mono change. * Added known exceptions to LLVM exception list. * Needs `ifdef` because of this mono/mono#17260. * Bump MIN_MONO_VERSION to 6.8.0.41 and point MIN_MONO_URL to the PR. * Add ENABLE_IOS=1 and ENABLE_MAC=1. * Added switch to disable packaged mono build * [Tests] Ignore tests that fail on 32b. Ignore the test on 32b, and filled issue: mono/mono#17752 * [Tests] Ignore a couple of tests causing OOM. Hopefully fixes xamarin/maccore#1659 for good. * Ignore `MM0135` test on Catalina+ because it needs Xcode 9.4. * [monotouch-test] Add null checks for teardown when test didn't run because of a too early OS version. * [CFNetwork]: Http 2.0 requires OS X 10.11 or later. Check whether `_HTTPVersion2_0` is available and fallback to HTTP 1.1 otherwise. * #7346 * This bumps Mono to use mono/mono#17645 (which is the 2019-10 backport of mono/mono#17628). * The big user-visible change is in regards to certificate validation, everything below are just some minor adjustments to tests. CoreFX uses a completely new `HttpClientHandler` implementation called `SocketsHttpHandler`, which you can find at https://github.com/dotnet/corefx/tree/release/3.0/src/System.Net.Http/src/System/Net/Http/SocketsHttpHandler. Since this is not based on the web stack anymore, it does not use any of the related APIs such as `ServicePointManager` or `WebException`. There is a new API called `HttpClientHandler.ServerCertificateCustomValidationCallback`. - https://docs.microsoft.com/en-us/dotnet/api/system.net.http.httpclienthandler.servercertificatecustomvalidationcallback?view=netframework-4.8 - https://github.com/dotnet/corefx/blob/c1778515a3bee34cc09c757b5563d0af0c8b1e99/src/System.Net.Http/src/System/Net/Http/HttpClientHandler.Unix.cs#L154 - https://github.com/dotnet/corefx/blob/c1778515a3bee34cc09c757b5563d0af0c8b1e99/src/System.Net.Http/src/System/Net/Http/HttpClientHandler.Windows.cs#L383 The `ServicePointManager.ServerCertificateValidationCallback` is no longer invoked and on certificate validation failure, `AuthenticationException` (from `System.Security.Authentication`) is thrown instead of `WebException`. At the moment, the `NSUrlSessionHandler` still uses it's own validation callback and also still throws `WebException` on failure; we should probably look into making this consistent with the other handlers. * `HttpContent.SerializeToStreamAsync()` is now `protected` (changed from `protected internal`). - src/Foundation/NSUrlSessionHandler.cs: changed overload accordingly. - src/System.Net.Http/CFContentStream.cs: likewise. * `HttpHeaders.GetKnownHeaderKind()` is an internal Mono API. There is a new internal API called `System.Net.Http.PlatformHelper.IsContentHeader(key)` which exists in both the old as well as the new implementation. The correct way of doing it with the CoreFX handler is `HeaderDescriptor.TryGet (key, out var descriptor) && descriptor.HeaderType == HttpHeaderType.Content` * `HttpClientHandler.MaxRequestContentBufferSize` is now longer supported, you can set it to any non-negative value, the getter will always return 0. See https://github.com/dotnet/corefx/blob/c1778515a3bee34cc09c757b5563d0af0c8b1e99/src/System.Net.Http/src/System/Net/Http/HttpClientHandler.Core.cs#L18. - tests/linker/ios/link sdk/HttpClientHandlerTest.cs: removed assertion from test. * `HttpMessageInvoker.handler` is a `protected private` field - in the CoreFX handler, it is called `_handler` and `private`. This is accessed via reflection by some of the tests, which are now using the new name. - tests/mmptest/src/MMPTest.cs: here - tests/mtouch/MTouch.cs: here * tests/monotouch-test/System.Net.Http/MessageHandlers.cs: Adjust `RejectSslCertificatesServicePointManager` to reflect the certificate validation changes described above. - FIXME: There was an `Assert.Ignore()` related to `NSUrlSessionHandler` and macOS 10.10; I removed that to reenable the test because the description linked to an old issue in the private repo that was referenced by several "Merged" PR's, so it looked to me that this might have already been fixed - and I also didn't see why it would fail there.
This bumps Mono to use mono/mono#17645 (which is the 2019-10 backport of mono/mono#17628).
The big user-visible change is in regards to certificate validation, everything below are just some minor adjustments to tests.
SocketsHttpHandler
CoreFX uses a completely new
HttpClientHandler
implementation calledSocketsHttpHandler
, which you can find here.Since this is not based on the web stack anymore, it does not use any of the related APIs such as
ServicePointManager
orWebException
.Certificate Validation Changes
There is a new API called
HttpClientHandler.ServerCertificateCustomValidationCallback
. You can find the documentation here and the code hereThe
ServicePointManager.ServerCertificateValidationCallback
is no longer invoked and on certificate validation failure,AuthenticationException
(fromSystem.Security.Authentication
) is thrown instead ofWebException
.At the moment, the
NSUrlSessionHandler
still uses it's own validation callback and also still throwsWebException
on failure; we should probably look into making this consistent with the other handlers.Minor adjustments related to internal Mono APIs
HttpContent.SerializeToStreamAsync()
is nowprotected
(changed fromprotected internal
).HttpHeaders.GetKnownHeaderKind()
is an internal Mono API.There is a new internal API called
System.Net.Http.PlatformHelper.IsContentHeader(key)
which exists in both the old as well as the new implementation.The correct way of doing it with the CoreFX handler is
HeaderDescriptor.TryGet (key, out var descriptor) && descriptor.HeaderType == HttpHeaderType.Content
Minor adjustments to tests.
HttpClientHandler.MaxRequestContentBufferSize
is now longer supported, you can set it to any non-negative value, the getter will always return 0. See the source code for details.HttpMessageInvoker.handler
is aprotected private
field - in the CoreFX handler, it is called_handler
andprivate
. This is accessed via reflection by some of the tests, which are now using the new name.tests/monotouch-test/System.Net.Http/MessageHandlers.cs:
Adjust
RejectSslCertificatesServicePointManager
to reflect the certificate validation changes described above.Assert.Ignore()
related toNSUrlSessionHandler
and macOS 10.10;I removed that to reenable the test because the description linked to an old issue in the private repo that was referenced by several "Merged" PR's, so it looked to me that this might have already been fixed - and I also didn't see why it would fail there.