-
Notifications
You must be signed in to change notification settings - Fork 4.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
GSS failures in System.Net.Http.Functional.Tests on Ubuntu 22.04 #67353
Comments
Tagging subscribers to this area: @dotnet/ncl Issue DetailsDescriptionRunning runtime tests on Ubuntu 22.04 (which adds OpenSSL 3.0 resulting in a number of changes under the hood), leads to a bunch of tests failing: https://dev.azure.com/dnceng/public/_build/results?buildId=1690650&view=ms.vss-test-web.build-test-results-tab&runId=46193442&resultId=189361&paneView=dotnet-dnceng.dnceng-anon-build-release-tasks.helix-anon-test-information-tab Some examples:
Reproduction StepsFrom helix:
Expected behaviorAll tests pass Actual behaviorTests fail with GSS exceptions. Regression?Yes, the tests pass on older versions of Ubuntu currently running in CI Known WorkaroundsNo response Configuration
Other informationNo response
|
Triage: Likely related to new image Ubuntu 22.04 (we do not have a queue yet). Seems to fail reliably. We should investigate. |
do you know @omajid if the image has the gss-ntlm package? Generally, I would think the OpenSSL is independent from Kerberos and GSS. |
Actually, you may be right about OpenSSL. It seems like
and https://github.com/gssapi/gss-ntlmssp/blob/734e522c14a9821d7c03f2ce1691706d3d8131ad/src/crypto.c#L149-L153 For now, I think we detect presence of the page and we would skip tests as needed. Short term fix may be removing the ntlm package from docker image. That of course leaves NTLM auth broken. The options would be to report/fix the package so it works with OpenSSL 3.x (e.g. add private fall-back implementation of md4) or switch to managed implementation #66879 |
Let's report upstream and see. Long term I am keen on making the Managed NTLM an option either through an app context switch, or as a fallback if gss-ntlmssp is not installed. |
Should we flag this as part of Ubuntu 22.04 support? Looking at dotnet/core#7038 it seems like everything is 100% functional? |
I'm not sure. This looks like distribution bug to me @omajid as the package they provide does not work. |
You need to load the legacy provider for that to work in OpenSSL 3. You can either do that in
However for the runtime, we explicitly load the "legacy" provider, so MD4 should be available. runtime/src/native/libs/System.Security.Cryptography.Native/openssl.c Lines 1175 to 1180 in fe0f600
|
One way to tell is to change
|
...but only after you use some crypto that initializes the OpenSSL native shim, right? |
Ah, you're right. I thought we always loaded the legacy provider, but we only do it when you use an algorithm that is in the legacy provider: Line 23 in 899bf97
Changing the |
This was fixed in gss-ntlm package. It is up to Ubuntu to pick up the fix. Big thanks to @simo5 who did the fix. |
@wfurt why did you send it back to triage? |
We can (should?) improve platform detection and skip the tests as needed instead of failing. This will bite us once #67345 is merged. |
Triage: Platform detection needs to be improved to handle the case as well. Note: This will be addressed once we have managed NTLM implementation - but there is no guarantee when it will happen. |
This isn't just a test-related issue. I just upgraded a system from Ubuntu 20.04 to Ubuntu 22.04 and a .NET 6 application could no longer use NTLM auth until I applied the workaround mentioned in #67353 (comment). Should this be documented somewhere as a compatibility issue for developers / end-users? |
We could but it is really difficult to trace and keep in sync all Linux distributions and versions. We could add note that the functionality depends on underlying OS capabilities. |
This helped. Thanks! :) |
Hi, We are planning to upgrade our .Net core MVC application from .Net 6 to .Net 8 version. It is written in C#. To prepare for that upgrade we first upgraded from ubuntu 20.04 to ubuntu 22.04. We target ubuntu 22.04 version jammy tag with amd64 architecture. This is our base image in the Docker file and the following line updates our package list. FROM artifactory.xyz.com/dockerhub-microsoft/dotnet/aspnet:6.0-jammy-amd64 AS base && apt-get update && apt-get install -y --no-install-recommends curl gss-ntlmssp tzdata \ When our application tries to authenticate and open SSRS reports we get "GSSAPI operation failed with error - Unspecified GSS failure. Minor code may provide more information (Crypto routine failure)" We were previously using focal base image in 20.04 version and was able to render SSRS reports. Did something change with gss-ntlmssp package in 22.04 version? I was reading some other posts where it says this issue is related to incompatibilities between OpenSSL 3.0 and the older cryptographic algorithms involved in NTLM authentication. Any thoughts on how to fix this issue? Appreciate your kind response. |
One of the fixes is literally the comment right above yours (enable the legacy crypto in OpenSSL). The other one is to get newer version of the gss-ntlmssp package or compile it yourself. |
there seems to be updated binaries in never Ubuntu. You may be able to get binaries from there with little bit of trickery (but I did not test it) |
and in .NET 8 they could force the managed implementation, right @filipnavara if all they need is NTLM? |
https://packages.ubuntu.com/search?keywords=gss-ntlmssp you want the 1.2.xxx version @SaravanakumBalach |
Thanks @wfurt . I looked at it but it appears like 1.2.xxx version is available only with ubuntu 23.x and above. But we are targeting ubuntu 22.04 which only supports GSSAPI 0.7.0 version. @filipnavara How can I enable the legacy crypto in OpenSSL? Any other thoughts pls? |
right, but
works because all dependencies are met
|
Description
Running runtime tests on Ubuntu 22.04 (which adds OpenSSL 3.0 resulting in a number of changes under the hood), leads to a bunch of tests failing: https://dev.azure.com/dnceng/public/_build/results?buildId=1690650&view=ms.vss-test-web.build-test-results-tab&runId=46193442&resultId=189361&paneView=dotnet-dnceng.dnceng-anon-build-release-tasks.helix-anon-test-information-tab
Some examples:
Reproduction Steps
From helix:
Expected behavior
All tests pass
Actual behavior
Tests fail with GSS exceptions.
Regression?
Yes, the tests pass on older versions of Ubuntu currently running in CI
Known Workarounds
No response
Configuration
5a0564b01442f8ea9247e27c4fab85ee0d457265
Other information
No response
The text was updated successfully, but these errors were encountered: