AWSCredentialsFactory.TryGetAWSCredentials() returns invalid SSOAWSCredentials

[RamonEbdonSS]

Description

My company has an SSO setup in AWS (auth via Azure) to allow users to obtain temporary role credentials. Having just updated one of our .NET AWS applications to AWSSDK v3.7.x (via NuGET) from v3.3.x, I was no longer able to run the app locally to access AWS using my credentials (which can be successfully obtained via AWS CLI v2).

The app was previously (using AWSSDK v3.3.x) implicitly detecting my credentials in both ~/.aws/credentials and local Environment vars when creating an AmazonS3Client(), but this is now failing. According to the sequence of credentials detection, it should be fine, and it does appear as if the SDK is finding my shared credentials file, since it's returning the correct type, but the app is throwing an exception when trying to use the returned AWSCredentials object.

Following the basic guide to access credentials and profiles, creating a new SharedCredentialsFile() successfully locates and loads my credentials file, and the return out AWSCredentials is returning an SSOAWSCredentials object, but the object is invalid because two mandatory properties in the (SSOAWSCredentialsOptions)Options property are null: ClientName and SsoVerificationCallback.

When attempting to use the generated credentials in an AmazonS3Client(), exceptions are returned saying either:

    Value cannot be null. Parameter name: Options property cannot be empty: ClientName

Or:

    Value cannot be null. Parameter name: Options property cannot be empty: SsoVerificationCallback

But, if I manually set those two properties, it all works:

    ((SSOAWSCredentials)awsCredentials).Options.ClientName = "SSO";
    ((SSOAWSCredentials)awsCredentials).Options.SsoVerificationCallback = new Action<SsoVerificationArguments>(delegate (SsoVerificationArguments ssoArgs) {
        System.Diagnostics.Process.Start(new System.Diagnostics.ProcessStartInfo() {
            FileName = ssoArgs.VerificationUriComplete,
            UseShellExecute = true
        });
    });

ClientName appears to accept (and work with) any random string. It doesnt appear to be relevant to any credentials Profile names.
The new inline delegate() for SsoVerificationCallback is simply sending VerificationUriComplete to my systems default browser to do the SSO login and device authentication. (I have a check to make sure System.Environment.UserInteractive is true as well.)

Reproduction Steps

Attempt to access any AWS service using an SSOAWSCredentials object returned by AWSCredentialsFactory.TryGetAWSCredentials()

Environment

• SDK Version: AWSSDK.Core v3.7.0.2, AWSSDK.S3 v3.7.0.2, AWSSDK.SSO v3.7.0.1 AWSSDK.SSOOIDC v3.7.0.1
• OS Info: Windows 10 (20.04) v10.0.19041.867
• Build Environment: Visual Studio 2017
• Targeted .NET Platform: .NET Core v2.1

Resolution

Can the AWSSDK be updated to return some sort of default values in ClientName and SsoVerificationCallback in an SSOAWSCredentials.Options object?

Or at least update the linked documentation above with a notice to manually handle this scenario and specify that it must be dealt with?

---

This is a 🐛 bug-report

[ashishdhingra]

Examine behavior in other SDKs on how these handle client name and callback.

[McDoit]

As a original author of #1701 , do you have any input or better workaround for this @awschristou ?

[awschristou]

As a original author of #1701 , do you have any input or better workaround for this @awschristou ?

I don't believe the SDK can make assumptions around how to handle SSOAWSCredentials by default. For example, it may be reasonable to open a browser to log in when running an application that uses the SDK on a desktop, but if a system is running at the command line or within a service, different behavior may be required.

SSOAWSCredentials may also need to be configured to suit the security and auditing needs of a system, so a generalized configuration may not be ideal. Having said this, I defer to members of the SDK team (eg @ashishdhingra @ppittle @normj ) for their expertise in this space.

[WarrenFerrell]

THANK YOU the 2.1.0 release of Amazon.Extensions.Configuration.SystemsManager broke the IConfigurationBuilder.AddSystemsManager extension method (2.0.0 worked fine for SSO) for running locally and this is the only thing that fixed it.

[RamonEbdonSS]

@awschristou I agree that assuming default settings for authentication (especially SSO) is always fraught with danger and conflicts because everyone will have a slightly different set up. However, there is a reasonable case to be made that neither of these missing Options values require custom values.

If an SSOAWSCredentials object is being returned, it most likely implies that a (manual) authentication step is going to have to be taken to obtain the device token anyway, so defaulting SsoVerificationCallback to an action to trigger the user interaction process is not unreasonable.

ClientName is a mere string value. I'm happy to defer to anyone who can explain what it's supposed to actually be used for, but none of my testing showed that the value was particularly important. Changing it did not select a different profile, and it did not appear to affect any other profile overrides (as specified in my shared credentials file).

At the very least, even setting these values specifically to null would be better than returning empty objects.

[KenHundley]

I have a suggestion that I think would resolve this in a satisfactory way (hopefully). I validated when the ClientName and SsoVerificationCallback are provided, and the authentication is successful it caches the authorization in the .aws\sso\cache\ folder in the same way that aws sso login cli command caches the authorization token. This cache is then used to avoid any additional calls to the SSO login process if a valid token is present.

I think the biggest issue here is that it's currently checking for that ClientName and SsoVerificationCallback are provided even if it has a valid auth token cached and doesn't even need to execute an SSO login.

If it would delay the validation of those two properties until they were actually required then users could use the AWS CLI or similar method to execute their AWS SSO login. Then the code can happily use that login until that cached authorization is expired. If they would prefer that their project code does the login then they would need to add custom code like mentioned aboved to allow that individual project to execute the login process.

TL;DR: If it's going to use the cached login then don't check for these properties until a new login is needed. Then the project can support other methods of SSO login like using the AWS CLI's SSO login.

[boyersnet]

Based on @KenHundley's merged PR, I was able to get my .net core test project to work with AWS CLI v2 sso login. I did however have to keep the AWSSDK.SSOOIDC package reference.

I have created AWSSSO.S3Buckets as a sample project for demonstration purposes.

[ashishdhingra]

Hi @RamonEbdonSS,

Good afternoon.

Please review the comment from @boyersnet and see if the issue is resolved for you in the latest SDK version.

Thanks,
Ashish

[KenHundley]

I thought I had tested it without the SSOOIDC project, but I guess the dlls were still in the bin folder. I took a look at the code again and it would be pretty easy to remove the reference to SSOOIDC, but I would not be able to remove the AWSSDK.SSO reference since it's used to generate the credentials even when the token is cached.

However, if you don't want you include the SSO and SSOOIDC Dlls in your production release you can add <PrivateAssets>all</PrivateAssets>

    <PackageReference Include="AWSSDK.SSO" Version="3.7.0.23">
      <PrivateAssets>all</PrivateAssets>
    </PackageReference>
    <PackageReference Include="AWSSDK.SSOOIDC" Version="3.7.0.23">
      <PrivateAssets>all</PrivateAssets>
    </PackageReference>

This will include them when building locally, but exclude them when using publish to package the project for release.

[RamonEbdon]

Hi @ashishdhingra

Apologies for the delayed response as I am no longer working for that company (this is my personal GH account).

@KenHundley 's PR #1850 seems to at least allow a workflow where you obtain a token on the CLI first, then use those AWS Credentials without further modification. That's at least as good as the process when I was using saml2aws, so I'm happy to close this issue as "resolved".

Regards,
-Ramon

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.

[ardove]

Apologies for drudging up this (already closed) issue, but I believe I'm still experiencing the aformentioned symptoms and I'd love some help troubleshooting.

.NET Core version: netcoreapp3.1

AWS SDK Version(s)

<PackageReference Include="AWSSDK.Core" Version="3.7.2.3" />
<PackageReference Include="AWSSDK.S3" Version="3.7.1.23" />
<PackageReference Include="AWSSDK.SSO" Version="3.7.0.51" />
<PackageReference Include="AWSSDK.SSOOIDC" Version="3.7.0.51" />

Code in question

using System.Collections.Generic;
using System.Threading.Tasks;
using Amazon.Runtime;
using Amazon.Runtime.CredentialManagement;
using Amazon.S3;
using Amazon.S3.Model;
using FluentAssertions;
using Microsoft.VisualStudio.TestTools.UnitTesting;

[TestClass]
public class S3Tests
{
    [TestMethod]
    public async Task ListBucketsAsync_ShouldReturnSomething()
    {
        var credentialProfileStoreChain= new CredentialProfileStoreChain();
        credentialProfileStoreChain.TryGetAWSCredentials("MyProfileName", out AWSCredentials credentials);

        var client = new AmazonS3Client(credentials);
        ListBucketsResponse listBucketsResponse = await client.ListBucketsAsync();

        List<S3Bucket> buckets = listBucketsResponse.Buckets;
        buckets.Count.Should().BeGreaterThan(0);
    }
}

I've confirmed that the C# type of credentials is SSOAwsCredentials. The Options property does show that ClientName is, in fact, null so I can see why I'm getting the error, but I'm confused because I've already done the SSO login.

Steps I've taken on the CLI (note that I've swapped my real profile name with MyProfileName)

aws configure sso 
SSO start URL [None]: <url removed>
SSO Region [None]: us-east-1
There are N AWS accounts available to you.
Using the account ID <account id removed>
There are M roles available to you.
Using the role name "MyProfileName"
CLI default client Region [us-east-1]: us-west-2
CLI default output format [None]:
CLI profile name [MyProfileName]:

To use this profile, specify the profile name using --profile, as shown:

aws s3 ls --profile MyProfileName

I can confirm that at this stage, aws s3 ls --profile MyProfileName does indeed, return the list of buckets. I've also tried aws sso login --profile MyProfileName to no avail.

@KenHundley @ashishdhingra, any ideas? Are there steps I can take to troubleshoot why my token isn't being found in the SSO token cache originally implemented (I guess I'd need to verify that hashing the start url and hex-encoding the resulting bytes actually produes the correct identifier)? Should I open a new issue with this information?

[ardove]

Ah, ok, I think I found the issue by enabling logging to the console. Looks like the token file that's been generated in the cache has an unparseable ExpiresAt time. I'm not sure who's generating that file, but this seems like a bug somewhere. Please advise. @KenHundley @ashishdhingra

439 [11] ERROR Amazon.Runtime.Credentials.Internal.SsoTokenCache (null) - Unable to load token cache for start url:
System.FormatException: The string '2021-08-13T05:38:05UTC' is not a valid AllXsd value.
at System.Xml.XmlConvert.ToDateTime(String s, XmlDateTimeSerializationMode dateTimeOption)
at Amazon.Runtime.Credentials.Internal.SsoTokenUtils.FromJson(String json)
at Amazon.Runtime.Credentials.Internal.SsoTokenCache.Load()

Here's what the aws cli produces for me.

[KenHundley]

Are you running the latest CLI version? I think the may have changed the format of the cached date at some point.

[ardove]

Ah, thanks @KenHundley, that did it. All appears to be working now! Hope this helps the next reader.

[boyersnet]

@ardove - I'm curious to understand why you are getting the credentials out of the credential store?

var credentialProfileStoreChain= new CredentialProfileStoreChain(); credentialProfileStoreChain.TryGetAWSCredentials("MyProfileName", out AWSCredentials credentials);

If you use the options builder and specify the profile in the configuration you should get that "for free" like I've done here: https://github.com/boyersnet/AWSSSO.S3Buckets/blob/main/AWSSSO.S3Buckets/Program.cs#L15

[ardove]

@boyersnet for this specific use case, I was only using it to try to identify where my problem was coming from and limit the number of classes in play. We do actually use it in some legacy situations with .NET Framework though.

I'm already familiar with how the configuration system in .NET Framework & the AWS SDK works. Thanks!