Customised bearerTokenAuthenticationPolicy will fail to get access token at the first time

[MaryGao]

• Package Name: @azure/core-rest-pipeline
• Package Version: 1.8.2
• Operating system: Windows
• Is the bug related to documentation in
  • README.md
  • source code documentation
  • SDK API docs on https://docs.microsoft.com

Describe the bug

In HLC we have below codes to add bearerTokenAuthenticationPolicy to enable the CAE feature, but once the policy is enabled we fail to authenticate and got below error. If I didn't add this policy I could get the first token but failed to pass CAE process.

import * as coreRestPipeline from "@azure/core-rest-pipeline";

export class SubscriptionClient extends coreClient.ServiceClient {
constructor(
    credentials: coreAuth.TokenCredential,
    options?: SubscriptionClientOptionalParams
  ) {
   // Add our customised policy by function bearerTokenAuthenticationPolicy
   this.pipeline.addPolicy(
        coreRestPipeline.bearerTokenAuthenticationPolicy({
          scopes: `${optionsWithDefaults.baseUri}/.default`,
          challengeCallbacks: {
            authorizeRequestOnChallenge:
              coreClient.authorizeRequestOnClaimChallenge
          }
        })
      );
    }
}

Error

status: 401,
    headers: HttpHeadersImpl {
      _headersMap: Map(14) {
        'www-authenticate' => {
          name: 'www-authenticate',
          value: `Bearer authorization_uri="https://login.windows.net/", error="invalid_token", error_description="The 
authentication failed because of missing 'Authorization' header."`
        },

To Reproduce
Steps to reproduce the behavior:

1. Choose whatever the client you like, here I chose @azure/arm-subscriptions
2. Upgrade the client to use the @azure/core-rest-pipeline@1.8.2
3. Add the above policy to do authentication
4. Get the 401 errors
5. I debugged into the code I found below code failed to get access token

await callbacks.authorizeRequest({
        scopes: Array.isArray(scopes) ? scopes : [scopes],
        request,
        getAccessToken,
        logger,
      });

Expected behavior
Expect the auth could be successful and if we test in CAE tenant it could pass the challenges

Screenshots
If applicable, add screenshots to help explain your problem.

Additional context
If I didn't add our policy by function bearerTokenAuthenticationPolicy I could get the first auth token but fail to finish CAE process.

[azure-sdk]

Label prediction was below confidence level 0.6 for Model:ServiceLabels: 'Azure.Core:0.18170801,Storage:0.14154309,Docs:0.06791836'