{CI} Fix Python wheels Test #27102
Merged
{CI} Fix Python wheels Test #27102
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
️✔️AzureCLI-FullTest
|
️✔️AzureCLI-BreakingChangeTest
|
ghost
added
microsoft-github-policy-service
bot
requested review from
wangzelin007,
yonzhan,
jiasli and
kairu-ms
|
CI |
|
Why is the log not in English but Azerbaijani?
|
jiasli
reviewed
Aug 16, 2023
| @@ -449,12 +449,12 @@ jobs: | |||
| echo "== Testing pip install on Python 3.9 ==" | |||
| docker run \ | |||
| --rm -v $PYPI_FILES:/mnt/pypi python:3.9 \ | |||
| /bin/bash -c "ls /mnt/pypi && pip install -f /mnt/pypi -q azure-cli==$CLI_VERSION.* && az self-test && az --version && sleep 5" | |||
There was a problem hiding this comment.
The test for Python 3.7 was removed by #27015.
.* was initially added by 4baa9d8 from #16362.
I made a test on the built artifact and noticed * is not expanded by the shell if no matching file exists:
$ curl https://artprodwus21.artifacts.visualstudio.com/A7b238909-6802-4b65-b90d-184bca47f458/5147fa83-336e-44ef-bbe0-c86b8ae86cbb/_apis/artifact/cGlwZWxpbmVhcnRpZmFjdDovL2F6Y2xpdG9vbHMvcHJvamVjdElkLzUxNDdmYTgzLTMzNmUtNDRlZi1iYmUwLWM4NmI4YWU4NmNiYi9idWlsZElkLzgyNDg5L2FydGlmYWN0TmFtZS9weXBp0/content?format=zip -o pypi.zip
$ unzip pypi.zip
$ cd pypi
$ echo azure-*
azure-cli-2.51.0.post20230815081512.tar.gz azure-cli-core-2.51.0.post20230815081512.tar.gz azure-cli-telemetry-1.1.0.post20230815081512.tar.gz
$ CLI_VERSION=2.51.0
$ echo azure-cli==$CLI_VERSION.*
azure-cli==2.51.0.*
Since #27001, 2.50.0.post* on the local disk is not compatible with Python 3.7, so pip falls back to a compatible version 2.50.0 from PyPI.
jiasli
reviewed
Aug 16, 2023
azure-pipelines.yml
Outdated
| @@ -449,12 +449,12 @@ jobs: | |||
| echo "== Testing pip install on Python 3.9 ==" | |||
| docker run \ | |||
| --rm -v $PYPI_FILES:/mnt/pypi python:3.9 \ | |||
| /bin/bash -c "ls /mnt/pypi && pip install -f /mnt/pypi -q azure-cli==$CLI_VERSION.* && az self-test && az --version && sleep 5" | |||
| /bin/bash -c "cd /mnt/pypi && ls && pip install -f ./ azure_cli-$CLI_VERSION*whl && az self-test && az --version && sleep 5" | |||
There was a problem hiding this comment.
Why cd into /mnt/pypi? Is -f ./ still necessary now?
There was a problem hiding this comment.
I don't want to write full path in -f and whl file.
Yes, the azure_cli-$CLI_VERSION*whl requires azure-cli-core-XX.post, we need to add this folder into additional repositories.
jiasli
reviewed
Aug 16, 2023
azure-pipelines.yml
Outdated
| @@ -449,12 +449,12 @@ jobs: | |||
| echo "== Testing pip install on Python 3.9 ==" | |||
| docker run \ | |||
| --rm -v $PYPI_FILES:/mnt/pypi python:3.9 \ | |||
| /bin/bash -c "ls /mnt/pypi && pip install -f /mnt/pypi -q azure-cli==$CLI_VERSION.* && az self-test && az --version && sleep 5" | |||
| /bin/bash -c "cd /mnt/pypi && ls && pip install -f ./ azure_cli-$CLI_VERSION*whl && az self-test && az --version && sleep 5" | |||
There was a problem hiding this comment.
If we force it to install the whl packages, there will be no need to care about $CLI_VERSION. I guess we may simply use azure_cli*.whl to install all local wheels:
Suggested change
| /bin/bash -c "cd /mnt/pypi && ls && pip install -f ./ azure_cli-$CLI_VERSION*whl && az self-test && az --version && sleep 5" | |
| /bin/bash -c "cd /mnt/pypi && ls && pip install azure_cli*.whl && az self-test && az --version && sleep 5" |
This will include
$ echo azure_cli*.whl
azure_cli-2.51.0.post20230815081512-py3-none-any.whl
azure_cli_core-2.51.0.post20230815081512-py3-none-any.whl
azure_cli_telemetry-1.1.0.post20230815081512-py3-none-any.whl
There was a problem hiding this comment.
pip install azure_cli*.whl also works, but I want to make this test command close to the real scenario: pip install azure-cli.
My OpenAI Translator extension is triggered unexpectedly. |
jiasli
approved these changes
Aug 17, 2023
jiasli
reviewed
Aug 17, 2023
| @@ -449,12 +449,12 @@ jobs: | |||
| echo "== Testing pip install on Python 3.9 ==" | |||
| docker run \ | |||
| --rm -v $PYPI_FILES:/mnt/pypi python:3.9 \ | |||
| /bin/bash -c "ls /mnt/pypi && pip install -f /mnt/pypi -q azure-cli==$CLI_VERSION.* && az self-test && az --version && sleep 5" | |||
| /bin/bash -c "cd /mnt/pypi && ls && pip install --find-links ./ azure_cli-$CLI_VERSION*whl && az self-test && az --version && sleep 5" | |||
There was a problem hiding this comment.
It'll be good practice to always make * match as little as possible:
Suggested change
| /bin/bash -c "cd /mnt/pypi && ls && pip install --find-links ./ azure_cli-$CLI_VERSION*whl && az self-test && az --version && sleep 5" | |
| /bin/bash -c "cd /mnt/pypi && ls && pip install --find-links ./ azure_cli-*-py3-none-any.whl && az self-test && az --version && sleep 5" |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
In this PR, I change the command to install the wheel directly, to make sure it does not install package from pypi
When releasing 2.51.0, Test Python Wheels fails with this error:
That's expected, as 2.51 does not support 3.7. I'm curious why this test does not find this error earlier.
In fact,
pip install -f /mnt/pypi -q azure-cli==$CLI_VERSION.*never runs as expected.This command add
/mnt/pypiinto search path, and try to installazure-cli==$CLI_VERSION.*. It has two problem:/mnt/pypidoes not have higher priority, if pypi contains the specific version, it may install from pypi instead of local$CLI_VERSION.*includes$CLI_VERSION, which can be found from pypi.Here is the log in python 3.7. I guess it tries to install local version first, which is not compatible with 3.7, then install 2.50 from pypi.
Here is the log for 3.8. It successfully installs the local wheel.