Skip to content

Commit

Permalink
Fix using Poetry with outdated Python patch versions
Browse files Browse the repository at this point in the history
The existing Poetry bootstrap process added in #1682 used the pip wheel
bundled within the Python stdlib.

This required use of pip's `--python` option, which was added to pip in
v22.3 in 2022. All of the major Python versions we support have been
updated to that pip version or newer, the older patch releases of some
of those major Python versions can contain pip versions that are older.

Whilst we strongly recommend upgrading to newer patch releases (since
older versions are missing security updates and so likely insecure),
we still want to support using Poetry on these versions, so the I've
adjusted the bootstrap process to no longer use `--python`.
  • Loading branch information
edmorley committed Nov 6, 2024
1 parent 158def0 commit 981ba20
Show file tree
Hide file tree
Showing 7 changed files with 66 additions and 15 deletions.
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

## [Unreleased]

- Fixed Poetry installation when using outdated patch versions of Python 3.8, 3.9 and 3.10, whose bundled pip doesn't support the `--python` option. ([#1687](https://github.com/heroku/heroku-buildpack-python/pull/1687))

## [v264] - 2024-11-06

Expand Down
2 changes: 1 addition & 1 deletion bin/compile
Original file line number Diff line number Diff line change
Expand Up @@ -168,7 +168,7 @@ case "${package_manager}" in
pipenv::install_pipenv
;;
poetry)
poetry::install_poetry "${python_home}" "${CACHE_DIR}" "${EXPORT_PATH}"
poetry::install_poetry "${CACHE_DIR}" "${EXPORT_PATH}"
;;
*)
utils::abort_internal_error "Unhandled package manager: ${package_manager}"
Expand Down
25 changes: 11 additions & 14 deletions lib/poetry.sh
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,8 @@ set -euo pipefail
POETRY_VERSION=$(utils::get_requirement_version 'poetry')

function poetry::install_poetry() {
local python_home="${1}"
local cache_dir="${2}"
local export_file="${3}"
local cache_dir="${1}"
local export_file="${2}"

# We store Poetry in the build cache, since we only need it during the build.
local poetry_root="${cache_dir}/.heroku/python-poetry"
Expand Down Expand Up @@ -39,19 +38,17 @@ function poetry::install_poetry() {
# The Poetry directory will already exist in the relocated cache case mentioned above.
rm -rf "${poetry_root}"

python -m venv --without-pip "${poetry_venv_dir}"

# We use the pip wheel bundled within Python's standard library to install Poetry.
# Whilst Poetry does still require pip for some tasks (such as package uninstalls),
# it bundles its own copy for use as a fallback. As such we don't need to install pip
# into the Poetry venv (and in fact, Poetry wouldn't use this install anyway, since
# it only finds an external pip if it exists in the target venv).
local bundled_pip_module_path
bundled_pip_module_path="$(utils::bundled_pip_module_path "${python_home}")"
# We can't use the pip wheel bundled within Python's standard library to install Poetry
# (which would allow us to use `--without-pip` here to skip the pip install), since it
# requires using the `--python` option, which was only added in pip v22.3. And whilst
# all major Python versions we support now bundled a newer pip than that, some apps
# are still using outdated patch releases of those Python versions, whose bundled pip
# can be older (for example Python 3.9.0 ships with pip v20.2.1). Once Python 3.10 EOLs
# we can switch back to the previous approach since Python 3.11.0 ships with pip v22.3.
python -m venv "${poetry_venv_dir}"

if ! {
python "${bundled_pip_module_path}" \
--python "${poetry_venv_dir}" \
"${poetry_venv_dir}/bin/pip" \
install \
--disable-pip-version-check \
--no-cache-dir \
Expand Down
1 change: 1 addition & 0 deletions spec/fixtures/poetry_oldest_python/.python-version
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
3.9.0
17 changes: 17 additions & 0 deletions spec/fixtures/poetry_oldest_python/poetry.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 6 additions & 0 deletions spec/fixtures/poetry_oldest_python/pyproject.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
[tool.poetry]
package-mode = false

[tool.poetry.dependencies]
python = "^3.9"
typing-extensions = "*"
29 changes: 29 additions & 0 deletions spec/hatchet/poetry_spec.rb
Original file line number Diff line number Diff line change
Expand Up @@ -185,6 +185,35 @@
end
end

# This checks that the Poetry bootstrap works even with older bundled pip, and that
# our chosen Poetry version also supports our oldest supported Python version.
context 'when using the oldest supported Python version' do
let(:app) { Hatchet::Runner.new('spec/fixtures/poetry_oldest_python') }

it 'installs successfully' do
app.deploy do |app|
expect(clean_output(app.output)).to include(<<~OUTPUT)
remote: -----> Python app detected
remote: -----> Using Python 3.9.0 specified in .python-version
remote: -----> Installing Python 3.9.0
remote:
remote: ! Warning: A Python security update is available!
remote: !
remote: ! Upgrade as soon as possible to: Python #{LATEST_PYTHON_3_9}
remote: ! See: https://devcenter.heroku.com/articles/python-runtimes
remote:
remote: -----> Installing Poetry #{POETRY_VERSION}
remote: -----> Installing dependencies using 'poetry install --sync --only main'
remote: Installing dependencies from lock file
remote:
remote: Package operations: 1 install, 0 updates, 0 removals
remote:
remote: - Installing typing-extensions (4.12.2)
OUTPUT
end
end
end

context 'when poetry.lock is out of sync with pyproject.toml' do
let(:app) { Hatchet::Runner.new('spec/fixtures/poetry_lockfile_out_of_sync', allow_failure: true) }

Expand Down

0 comments on commit 981ba20

Please sign in to comment.