Always install into a python venv in ci containers #12663
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -18,28 +18,77 @@ | |
|
|
||
| set -e | ||
| set -u | ||
| set -o pipefail | ||
|
|
||
| set -x | ||
|
|
||
| if [ -z "${TVM_VENV+x}" ]; then | ||
| echo "ERROR: expect TVM_VENV env var to be set" | ||
| exit 2 | ||
| fi | ||
|
|
||
| apt-get update | ||
|
|
||
| # Ensure lsb-release is installed. | ||
| apt-install-and-clear -y \ | ||
| lsb-core | ||
|
|
||
| release=$(lsb_release -sc) | ||
| if [ "${release}" == "bionic" ]; then | ||
|
There was a problem hiding this comment. What if we use miniconda instead of the distribution's Python + venv? The end result would be the same but we would be able to say "we use version X of Python in CI" instead of "it depends" There was a problem hiding this comment. When I was migrating teams off of conda I used pyenv which would compose into our system with Poetry? |
||
| PYTHON_VERSION=3.7 | ||
| elif [ "${release}" == "focal" ]; then | ||
| PYTHON_VERSION=3.8 | ||
| else | ||
| echo "Don't know which version of python to install for lsb-release ${release}" | ||
| exit 2 | ||
| fi | ||
|
|
||
| # Install python and pip. Don't modify this to add Python package dependencies, | ||
| # instead modify install_python_package.sh | ||
| apt-install-and-clear -y software-properties-common | ||
| apt-install-and-clear -y \ | ||
| acl \ | ||
| python${PYTHON_VERSION} \ | ||
| python${PYTHON_VERSION}-dev \ | ||
| python3-pip \ | ||
| python${PYTHON_VERSION}-venv | ||
|
|
||
| update-alternatives --install /usr/bin/python3 python3 /usr/bin/python${PYTHON_VERSION} 1 | ||
|
|
||
| # Allow disabling user site-packages, even with sudo; this makes it harder to repro CI failures | ||
| # locally because it's hard to tell what might be in this directory. | ||
| echo "Defaults env_keep += \"PYTHONNOUSERSITE\"" >/etc/sudoers.d/91-preserve-python-nousersite | ||
| export PYTHONNOUSERSITE=1 | ||
|
|
||
| venv_dir="$(python3 -c "import os.path;print(os.path.dirname(\"${TVM_VENV}\"))")" | ||
| mkdir -p "${venv_dir}" | ||
| python3 -mvenv "${TVM_VENV}" | ||
| . "${TVM_VENV}/bin/activate" | ||
|
|
||
| # Update pip to match version used to produce requirements-hashed.txt. This step | ||
| # is necessary so that pip's dependency solver is recent. | ||
| pip_spec=$(cat /install/python/bootstrap/lockfiles/constraints-${PYTHON_VERSION}.txt | grep 'pip==') | ||
| pip3 install -U --require-hashes -r <(echo "${pip_spec}") \ | ||
| -c /install/python/bootstrap/lockfiles/constraints-${PYTHON_VERSION}.txt | ||
|
|
||
| # Python configuration | ||
| pip3 config set global.no-cache-dir true # Never cache packages | ||
|
|
||
| # Now install the remaining base packages. | ||
| pip3 install \ | ||
| --require-hashes \ | ||
| -r /install/python/bootstrap/lockfiles/constraints-${PYTHON_VERSION}.txt | ||
|
|
||
| addgroup tvm-venv | ||
| chgrp -R tvm-venv "${TVM_VENV}" | ||
| setfacl -R -d -m group:tvm-venv:rwx "${TVM_VENV}" | ||
|
|
||
| # Prevent further use of pip3 via the system. | ||
| # There may be multiple (i.e. from python3-pip apt package and pip3 install -U). | ||
| deactivate | ||
| while [ "$(which pip3)" != "" ]; do | ||
| rm "$(which pip3)" | ||
| done | ||
| while [ "$(which pip)" != "" ]; do | ||
| rm "$(which pip)" | ||
| done | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.