SharedTokenCacheCredential authentication fails in Azure.Identity on ubuntu

[thoemmi]

We have written an internal command line tool that allows us to read secrets from Azure Keyvault. This tool consumes Azure.Identity, Azure.Security.KeyVault.*, and Azure.ResourceManager.Resources

We run this tool both on our dev machines and in the Azure DevOps release pipeline. However, today we have updated Azure.Identity from 1.2.2 to 1.3.0. While the tool still runs without any issue on my Windows machine, it fails on the build agent in the release pipeline (ubuntu-18.04):

ERROR: Azure.Identity.AuthenticationFailedException: SharedTokenCacheCredential authentication failed: Persistence check failed. Data was written but it could not be read. Possible cause: on Linux, LibSecret is installed but D-Bus isn't running because it cannot be started over SSH.
    Microsoft.Identity.Client.Extensions.Msal.MsalCachePersistenceException: Persistence check failed. Data was written but it could not be read. Possible cause: on Linux, LibSecret is installed but D-Bus isn't running because it cannot be started over SSH.
      at Microsoft.Identity.Client.Extensions.Msal.MsalCacheStorage.VerifyPersistence() 
      at Microsoft.Identity.Client.Extensions.Msal.MsalCacheHelper.VerifyPersistence() 
      at Azure.Identity.MsalClientBase`1.GetClientAsync(Boolean async, CancellationToken cancellationToken) 
      at Azure.Identity.MsalClientBase`1.GetClientAsync(Boolean async, CancellationToken cancellationToken) 
      at Azure.Identity.MsalPublicClient.GetAccountsAsync(Boolean async, CancellationToken cancellationToken) 
      at Azure.Identity.SharedTokenCacheCredential.GetAccountAsync(Boolean async, CancellationToken cancellationToken) 
      at Azure.Identity.SharedTokenCacheCredential.GetTokenImplAsync(Boolean async, TokenRequestContext requestContext, CancellationToken cancellationToken) 
  at Azure.Identity.CredentialDiagnosticScope.FailWrapAndThrow(Exception ex) 
  at Azure.Identity.SharedTokenCacheCredential.GetTokenImplAsync(Boolean async, TokenRequestContext requestContext, CancellationToken cancellationToken) 
  at Azure.Identity.SharedTokenCacheCredential.GetTokenAsync(TokenRequestContext requestContext, CancellationToken cancellationToken) 
  at Azure.Identity.DefaultAzureCredential.GetTokenFromSourcesAsync(TokenCredential[] sources, TokenRequestContext requestContext, Boolean async, CancellationToken cancellationToken) 
  at Azure.Identity.DefaultAzureCredential.GetTokenImplAsync(Boolean async, TokenRequestContext requestContext, CancellationToken cancellationToken) 
  at Azure.Identity.CredentialDiagnosticScope.FailWrapAndThrow(Exception ex) 
  at Azure.Identity.DefaultAzureCredential.GetTokenImplAsync(Boolean async, TokenRequestContext requestContext, CancellationToken cancellationToken) 
  at Azure.Identity.DefaultAzureCredential.GetTokenAsync(TokenRequestContext requestContext, CancellationToken cancellationToken) 
  at Azure.Core.Pipeline.BearerTokenAuthenticationPolicy.AccessTokenCache.GetHeaderValueFromCredentialAsync(HttpMessage message, Boolean async, CancellationToken cancellationToken) 
  at Azure.Core.Pipeline.BearerTokenAuthenticationPolicy.AccessTokenCache.GetHeaderValueAsync(HttpMessage message, Boolean async) 
  at Azure.Core.Pipeline.BearerTokenAuthenticationPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async) 
  at Azure.Core.Pipeline.RetryPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async) 
  at Azure.Core.Pipeline.RetryPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async) 
  at Azure.Core.Pipeline.HttpPipelineSynchronousPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline) 
  at Azure.Core.Pipeline.HttpPipelineSynchronousPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline) 
  at Azure.Core.Pipeline.HttpPipelineSynchronousPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline) 
  at Azure.ResourceManager.Resources.ResourceGroupsRestOperations.GetAsync(String resourceGroupName, CancellationToken cancellationToken) 
  at Azure.ResourceManager.Resources.ResourceGroupsOperations.GetAsync(String resourceGroupName, CancellationToken cancellationToken) 

I don't know if this is caused by Azure.Identity itself or by the MSAL token cache extension you're using.

[jsquire]

Thank you for your feedback. Tagging and routing to the team member best able to assist.

[schaabs]

@thoemmi Thank you for filing this issue, I'm sorry you're running into this problem. I'll investigate the root cause of this regression and hopefully release a patch for it soon. In the meantime you should be able to work around this by excluding the SharedTokenCacheCredential from the DefaultAzureCredential.

var credential = new DefaultAzureCredential(new DefaultAzureCredentialOptions { ExcludeSharedTokenCacheCredential = true });

This should work around the issue your experiencing in your CI and should not impact your authentication in your developement environment as the VisualStudioCredential will provide authentication in lieu of the SharedTokenCacheCredential. As a matter of fact we're also planning on disabling the SharedTokenCacheCredential by default in our next release. More info on this can be found in this issue Azure/azure-sdk#1970, and PR #16615. Please let me know if you have further questions or have trouble getting the work around working.

[thoemmi]

@schaabs, thanks for your quick answer. The work around you proposed indeed does the trick, and our release pipeline is green again 👍

I guess I shall not close this issue as you've added it to the December milestone.

[schaabs]

@thoemmi I'm glad to hear that you're unblocked. Yes, please leave the issue open as we intend to fix this regression, and we'll close it once the fix is merged.

[JonRoosevelt]

Hello @schaabs
I'm having a similar issue and followed your work around.
Now it has raised another Exception, regarding Authentication, as follows:

Exception has occurred: CLR/Azure.Identity.AuthenticationFailedException

An unhandled exception of type 'Azure.Identity.AuthenticationFailedException' occurred in System.Private.CoreLib.dll: 'Azure CLI authentication failed due to an unknown error. ERROR: Get Token request returned http error: 400 and server response: 
{"error":"invalid_grant","error_description":"AADSTS50173: The provided grant has expired due to it being revoked, a fresh auth token is needed. The user might have changed or reset their password. The grant was issued on '2020-11-12T13:35:59.6312234Z' and the TokensValidFrom date (before which tokens are not valid) for this user is '2020-11-23T12:56:24.0000000Z'.\r\nTrace ID: XXXXXXXX (hidden) \r\nCorrelation ID: XXXXXX (hidden)\r\nTimestamp: 2020-12-01 15:28:26Z","error_codes":[50173],"timestamp":"2020-12-01 15:28:26Z","trace_id":**XXXXXX (hidden)**,"correlation_id":XXXX(hidden),"error_uri":"https://login.microsoftonline.com/error?code=50173"}

I kinda understand that it's related to refres token and even tried to, instead of creating a new DefaultAzureCredential, instantiating an existing one using clientId and clientSecret, but I'm not sure that's how it should be addressed.

[jongio]

@schaabs - This popped up for me today as well. By removing SharedTokenCacheCredential from DAC, are we saying it isn't support on Linux?

Exception has occurred: CLR/Azure.Identity.AuthenticationFailedException
An exception of type 'Azure.Identity.AuthenticationFailedException' occurred in System.Private.CoreLib.dll but was not handled in user code: 'SharedTokenCacheCredential authentication failed: Persistence check failed. Inspect inner exception for details'
 Inner exceptions found, see $exception in variables window for more details.
 Innermost exception 	 System.DllNotFoundException : Unable to load shared library 'libsecret-1.so.0' or one of its dependencies. In order to help diagnose loading problems, consider setting the LD_DEBUG environment variable: liblibsecret-1.so.0: cannot open shared object file: No such file or directory
   at Microsoft.Identity.Client.Extensions.Msal.Libsecret.secret_schema_new(String name, Int32 flags, String attribute1, Int32 attribute1Type, String attribute2, Int32 attribute2Type, IntPtr end)
   at Microsoft.Identity.Client.Extensions.Msal.LinuxKeyringAccessor.GetLibsecretSchema()
   at Microsoft.Identity.Client.Extensions.Msal.LinuxKeyringAccessor.Write(Byte[] data)
   at Microsoft.Identity.Client.Extensions.Msal.MsalCacheStorage.VerifyPersistence()

[R2J2]

Where do I need to add the workaround? Is it written to the application code or the dockerfile?

[jongio]

@R2J2 - See this comment. #17052 (comment)

You remove it in your code.

[dgerding]

This is happening to me currently (April '22) using release latest versions of all nugets.

OPening a new issue, but also pursuing with Azure support and will update if that yields any results.

[dgerding]

#28120 Help :)