Azure CLI GitHub Action fails with Azure CLI 2.30.0: Could not retrieve credential from local cache for service principal

[zwolf]

Describe the bug
I am using the Azure CLI Github Action to upload files to Azure Blob Storage. This worked until this morning, when latest became CLI 2.30.0 and uploads to Blob Storage began failing with a 404 ResourceNotFound error.

To Reproduce

• Create a storage principal with az ad sp create-for-rbac that has access to your storage.
• Store the creds in a Github organization secret
• Use the credentials to log in via the CLI in the action:

    - uses: azure/login@v1
      with:
        creds: ${{ secrets.AZURE_CREDENTIALS }}

• Build a Github Action that looks something like this:

    - name: Upload to blob storage
      uses: azure/CLI@v1
      with:
        inlineScript: |
            az storage blob upload \
              --account-name my-account \
              --container-name 'my-container' \
              --name 'path/to/file.html' \
              --file ./file.html

This will default to using azcliversion: latest and the new & unexpected output is:

WARNING: Skip querying account key due to failure: Could not retrieve credential from local cache for service principal asdfasdf-asdf-asdf-asdf-asdfasdfasdf. Run az login for this service principal.
ERROR: Client-Request-ID=asdfasdf-asdf-asdf-asdf-adsfasdfasdf Retry policy did not allow for a retry: Server-Timestamp=Tue, 02 Nov 2021 16:52:14 GMT, Server-Request-ID=asdfadsf-asdf-asdf-asdf-asdfasdfasdfasdf, HTTP status code=404, Exception=The specified resource does not exist. ErrorCode: ResourceNotFound<?xml version="1.0" encoding="utf-8"?><Error><Code>ResourceNotFound</Code><Message>The specified resource does not exist.RequestId:asdfasdfasdf-asdf-adsf-adsf-asdfasdfasdfadsfTime:2021-11-02T16:52:15.3159868Z</Message></Error>.

Running az login for the associated service principal works, but does not solve this problem.

Reverting to previous release by adding

  with:
    azcliversion: 2.29.2

to the above fixes the issue and uploads the file as expected.

Expected behavior
The latest version of the CLI should work the same as the previous version. There are no related changes in the release notes for 2.30.0..

Environment summary
Github's own Action runners using runs-on: ubuntu-latest (20.04.3 LTS) along with the Azure-created actions noted above, specifically 'azure/CLI@v1' (SHA:4b58c946a0f48d82cc2b6e31c0d15a6604859554).

[jiasli]

Symptom

GitHub Action azure/CLI@v1 fails with

ERROR: Could not retrieve credential from local cache for service principal ea94e231-9b94-4844-bc6f-d773f9e07965. Run `az login` for this service principal.

Root cause

The is because Azure Login still uses the old ADAL-based Azure CLI 2.29.0 while Azure CLI Action uses the latest MSAL-based Azure CLI 2.30.0.

After the ADAL->MSAL migration (#19853), the latest Azure CLI is not compatible with old versions.

This can be reproduced with

# File: .github/workflows/workflow.yml

on: [push]

name: AzureCLISample

jobs:

  build-and-deploy:
    runs-on: ubuntu-latest
    steps:

    - name: Azure Login
      uses: azure/login@v1
      with:
        creds: ${{ secrets.AZURE_CREDENTIALS }}

    - name: Azure CLI script
      uses: azure/CLI@v1
      with:
        azcliversion: latest
        inlineScript: |
          set -ex
          ls ~/.azure
          cat ~/.azure/versionCheck.json
          az --version
          az account show
          az group list

The output shows accessTokens.json and versionCheck.json from ADAL-based Azure CLI:

+ ls /root/.azure
accessTokens.json

+ cat /root/.azure/versionCheck.json
***"versions": ***"azure-cli": ***"local": "2.29.0", "pypi": "2.30.0"***, "core": ***"local": "2.29.0", "pypi": "2.30.0"***, "telemetry": ***"local": "1.0.6", "pypi": "1.0.6"***, "update_time": "2021-11-03 02:58:51.551907"***

Workaround (no longer needed now)

Change azcliversion in azure/CLI@v1 to 2.29.2 which is compatible with azure/login@v1:

    - name: Azure CLI script
      uses: azure/CLI@v1
      with:
        azcliversion: 2.29.2

Action plan

We are working with GitHub Action team to fix this issue with high priority.

[arthursantana]

Workaround

Change azcliversion in azure/CLI@v1 to 2.29.2 which is compatible with azure/login@v1:

This workaround stopped working for me today.
ERROR: Could not retrieve credential from local cache for service principal [...]

Undoing the downgrade seems to have fixed the issue. I guess the latest azure/login was changed to use MSAL-based Azure CLI and therefore 2.29.2 isn't compatible anymore.

[abhi-markan]

Workaround

Change azcliversion in azure/CLI@v1 to 2.29.2 which is compatible with azure/login@v1:

This workaround stopped working for me today. ERROR: Could not retrieve credential from local cache for service principal [...]

Undoing the downgrade seems to have fixed the issue. I guess the latest azure/login was changed to use MSAL-based Azure CLI and therefore 2.29.2 isn't compatible anymore.

Did you upgrade from 2.29.2 -> 2.30.0?

[arthursantana]

Workaround

Change azcliversion in azure/CLI@v1 to 2.29.2 which is compatible with azure/login@v1:

This workaround stopped working for me today. ERROR: Could not retrieve credential from local cache for service principal [...]
Undoing the downgrade seems to have fixed the issue. I guess the latest azure/login was changed to use MSAL-based Azure CLI and therefore 2.29.2 isn't compatible anymore.

Did you upgrade from 2.29.2 -> 2.30.0?

I upgraded 2.29.2 to latest, not sure what the specific version is. Here is the hash:
Download action repository 'azure/CLI@v1' (SHA:4b58c946a0f48d82cc2b6e31c0d15a6604859554)

[0gust1]

so the fix is

- name: Deploy
  uses: azure/CLI@v1
  with:
    azcliversion: latest

or can we remove the azcliversion ?

[abhi-markan]

azcliversion: latest

azcliversion: latest is the fix, I would not recommend removing the version declaration.

[jiasli]

2021-11-11 Update

As confirmed with GitHub Action team, Azure Login now uses the latest Azure CLI 2.30.0, so the workaround from #20154 (comment) is no longer needed now.

Instead, if you specify azcliversion as an version <2.30.0 in Azure CLI Action, the action will fail with

Could not retrieve credential from local cache for service principal xxx.

This is simply because Azure CLI 2.30.0 is not compatible with previous versions (<2.30.0).

Recommendation

You may leave azcliversion unspecified, so that it defaults to agentazcliversion and automatically uses the same version as Azure Login on the agent (Azure/cli#57).

Still, thanks to @abhi-markan for the explanation.