az ml ... command in Dev Ops does not install extension

[mwebb4]

Describe the bug
As I understand from the docs (and as has been working up until sometime around last week), the CLI is supposed to automatically install the ml extension when it sees a command that references is. Within the Dev Ops pipeline, this must usually be prefaced by a config setting allowing for an install without prompt. This has been working for us for some time. However, recently, I’m getting a lot of errors indicating a different behavior. Specifically, the CLI appears unable to recognize commands beginning az ml and will not install the extension (with or without the no_prompt preface). A minimal example of the error is below:

To Reproduce
This pipeline .yml file below (with names omitted) will produce an unexpected error:

trigger: none
pool:
  vmImage: 'ubuntu-18.04'

steps:
- task: AzureCLI@2
  inputs:
    azureSubscription: '<service conn name>'
    scriptType: 'bash'
    scriptLocation: 'inlineScript'
    inlineScript: |
      az config set extension.use_dynamic_install=yes_without_prompt
      az ml experiment list -g '<rg name>' -w '<workspace name>' --subscription-id '<sub id>'

The error given is:

WARNING: Command group 'config' is experimental and under development. Reference and support levels: https://aka.ms/CLI_refstatus
ERROR: 'ml' is misspelled or not recognized by the system.

Expected behavior
The expected behavior is that the pipeline build agent will install the ml extension and execute the command. This was working until recently.

Environment summary
We are executing the above .yml file within Dev Ops on a MS build agent. The initial output from the Run is:

Starting: AzureCLI
==============================================================================
Task         : Azure CLI
Description  : Run Azure CLI commands against an Azure subscription in a PowerShell Core/Shell script when running on Linux agent or PowerShell/PowerShell Core/Batch script when running on Windows agent.
Version      : 2.1.0
Author       : Microsoft Corporation
Help         : https://docs.microsoft.com/azure/devops/pipelines/tasks/deploy/azure-cli
==============================================================================
/usr/bin/az --version
WARNING: You have 2 updates available. Consider updating your CLI installation with 'az upgrade'
azure-cli                         2.23.0 *


Please let us know how we are doing: https://aka.ms/azureclihats
and let us know if you're interested in trying out our newest features: https://aka.ms/CLIUXstudy
core                              2.23.0 *
telemetry                          1.0.6

Extensions:
azure-devops                      0.18.0

Python location '/opt/az/bin/python3'
Extensions directory '/opt/az/azcliextensions'

Python (Linux) 3.6.10 (default, Apr 29 2021, 12:10:03) 
[GCC 7.5.0]

Legal docs and information: aka.ms/AzureCliLegal

Additional context
The above command was working in previous Pipelines. Has something changed?

[ghost]

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @azureml-github.

Issue Details

---

Describe the bug
As I understand from the docs (and as has been working up until sometime around last week), the CLI is supposed to automatically install the ml extension when it sees a command that references is. Within the Dev Ops pipeline, this must usually be prefaced by a config setting allowing for an install without prompt. This has been working for us for some time. However, recently, I’m getting a lot of errors indicating a different behavior. Specifically, the CLI appears unable to recognize commands beginning az ml and will not install the extension (with or without the no_prompt preface). A minimal example of the error is below:

To Reproduce
This pipeline .yml file below (with names omitted) will produce an unexpected error:

trigger: none
pool:
  vmImage: 'ubuntu-18.04'

steps:
- task: AzureCLI@2
  inputs:
    azureSubscription: '<service conn name>'
    scriptType: 'bash'
    scriptLocation: 'inlineScript'
    inlineScript: |
      az config set extension.use_dynamic_install=yes_without_prompt
      az ml experiment list -g '<rg name>' -w '<workspace name>' --subscription-id '<sub id>'

The error given is:

WARNING: Command group 'config' is experimental and under development. Reference and support levels: https://aka.ms/CLI_refstatus
ERROR: 'ml' is misspelled or not recognized by the system.

Expected behavior
The expected behavior is that the pipeline build agent will install the ml extension and execute the command. This was working until recently.

Environment summary
We are executing the above .yml file within Dev Ops on a MS build agent. The initial output from the Run is:

Starting: AzureCLI
==============================================================================
Task         : Azure CLI
Description  : Run Azure CLI commands against an Azure subscription in a PowerShell Core/Shell script when running on Linux agent or PowerShell/PowerShell Core/Batch script when running on Windows agent.
Version      : 2.1.0
Author       : Microsoft Corporation
Help         : https://docs.microsoft.com/azure/devops/pipelines/tasks/deploy/azure-cli
==============================================================================
/usr/bin/az --version
WARNING: You have 2 updates available. Consider updating your CLI installation with 'az upgrade'
azure-cli                         2.23.0 *


Please let us know how we are doing: https://aka.ms/azureclihats
and let us know if you're interested in trying out our newest features: https://aka.ms/CLIUXstudy
core                              2.23.0 *
telemetry                          1.0.6

Extensions:
azure-devops                      0.18.0

Python location '/opt/az/bin/python3'
Extensions directory '/opt/az/azcliextensions'

Python (Linux) 3.6.10 (default, Apr 29 2021, 12:10:03) 
[GCC 7.5.0]

Legal docs and information: aka.ms/AzureCliLegal

Additional context
The above command was working in previous Pipelines. Has something changed?

Author:	mwebb4
Assignees:	-
Labels:	Machine Learning, Service Attention, needs-triage, question
Milestone:	-

[yonzhan]

route to service team

[ghost]

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @lostmygithubaccount.

Issue Details

---

Describe the bug
As I understand from the docs (and as has been working up until sometime around last week), the CLI is supposed to automatically install the ml extension when it sees a command that references is. Within the Dev Ops pipeline, this must usually be prefaced by a config setting allowing for an install without prompt. This has been working for us for some time. However, recently, I’m getting a lot of errors indicating a different behavior. Specifically, the CLI appears unable to recognize commands beginning az ml and will not install the extension (with or without the no_prompt preface). A minimal example of the error is below:

To Reproduce
This pipeline .yml file below (with names omitted) will produce an unexpected error:

trigger: none
pool:
  vmImage: 'ubuntu-18.04'

steps:
- task: AzureCLI@2
  inputs:
    azureSubscription: '<service conn name>'
    scriptType: 'bash'
    scriptLocation: 'inlineScript'
    inlineScript: |
      az config set extension.use_dynamic_install=yes_without_prompt
      az ml experiment list -g '<rg name>' -w '<workspace name>' --subscription-id '<sub id>'

The error given is:

WARNING: Command group 'config' is experimental and under development. Reference and support levels: https://aka.ms/CLI_refstatus
ERROR: 'ml' is misspelled or not recognized by the system.

Expected behavior
The expected behavior is that the pipeline build agent will install the ml extension and execute the command. This was working until recently.

Environment summary
We are executing the above .yml file within Dev Ops on a MS build agent. The initial output from the Run is:

Starting: AzureCLI
==============================================================================
Task         : Azure CLI
Description  : Run Azure CLI commands against an Azure subscription in a PowerShell Core/Shell script when running on Linux agent or PowerShell/PowerShell Core/Batch script when running on Windows agent.
Version      : 2.1.0
Author       : Microsoft Corporation
Help         : https://docs.microsoft.com/azure/devops/pipelines/tasks/deploy/azure-cli
==============================================================================
/usr/bin/az --version
WARNING: You have 2 updates available. Consider updating your CLI installation with 'az upgrade'
azure-cli                         2.23.0 *


Please let us know how we are doing: https://aka.ms/azureclihats
and let us know if you're interested in trying out our newest features: https://aka.ms/CLIUXstudy
core                              2.23.0 *
telemetry                          1.0.6

Extensions:
azure-devops                      0.18.0

Python location '/opt/az/bin/python3'
Extensions directory '/opt/az/azcliextensions'

Python (Linux) 3.6.10 (default, Apr 29 2021, 12:10:03) 
[GCC 7.5.0]

Legal docs and information: aka.ms/AzureCliLegal

Additional context
The above command was working in previous Pipelines. Has something changed?

Author:	mwebb4
Assignees:	-
Labels:	ADO, ML-MLOps, Machine Learning, Service Attention, bug
Milestone:	-

[lostmygithubaccount]

hi @mwebb4, Azure ML has recently release the 2.0 CLI (article on that here): https://docs.microsoft.com/en-us/azure/machine-learning/how-to-configure-cli

the default for the ml subgroup has been updated to the 2.0 ml extension - to mitigate this, you can instead install the azure-cli-ml extension (az ml extension add -n azure-cli-ml -y) in your pipeline(s) to use the 1.0 CLI

[mwebb4]

Yes, I've since found that workaround.
It seems a strange behavior that the CLI neither recognizes nor installs either version of the ml extension - a breaking change for any Pipeline that was using it.

[lostmygithubaccount]

I am seeing a prompt to install the extension on a fresh codespace:

regardless you are correct that this is a breaking change for any pipeline which was relying on the azure-cli-ml extension being installed

[mwebb4]

For sure. Per the original post, the Dev Ops build agents only generate the error: 'ml' is misspelled or not recognized by the system. - which, as I understand from the docs, should never happen. It should attempt to install some version.

[ghost]

Hi @mwebb4. Thank you for opening this issue and giving us the opportunity to assist. We believe that this has been addressed. If you feel that further discussion is needed, please add a comment with the text “/unresolve” to remove the “issue-addressed” label and continue the conversation.

[ghost]

Hi @mwebb4, since you haven’t asked that we “/unresolve” the issue, we’ll close this out. If you believe further discussion is needed, please add a comment “/unresolve” to reopen the issue.