-
Notifications
You must be signed in to change notification settings - Fork 1.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add support for Poetry #1682
Merged
Add support for Poetry #1682
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
After many refactoring/preparation PRs, we're now ready to add support for the package manager Poetry: https://python-poetry.org To use Poetry, apps must have a `poetry.lock` lockfile, which can be created by running `poetry lock` locally, after adding Poetry config to `pyproject.toml` (which can be done either manually or by using `poetry init`). For now, if a `requirements.txt` or `Pipfile` is found it will take precedence over `poetry.lock` for backwards compatibility (in the future this will become a warning then an error). This means users of the third-party `python-poetry-buildpack` will need to remove that buildpack in order to use the new native Poetry support, since it exports a `requirements.txt` file during the build. Poetry is installed into the build cache rather than the slug, so is not available at run-time (since it's not typically needed at run-time and doing so reduces the slug size). The entrypoints of installed dependencies are available on `PATH`, so use of `poetry run` or `poetry shell` is not required at run-time to use dependencies in the environment. When using Poetry, pip is not installed since Poetry includes its own internal vendored copy that it will use instead (for the small number of Poetry operations for which it still calls out to pip, such as package uninstalls). During normal (non-CI) builds, the `poetry install --sync` command is run using `--only main` so as to only install the main `[tool.poetry.dependencies]` dependencies group from `pyproject.toml` and not any of the app's other dependency groups (such as test/dev/... groups, eg `[tool.poetry.group.test.dependencies]`). On Heroku CI, all default Poetry dependency groups are installed (i.e. all groups minus those marked as `optional = true`). Relevant Poetry docs: - https://python-poetry.org/docs/cli/#install - https://python-poetry.org/docs/configuration/ - https://python-poetry.org/docs/managing-dependencies/#dependency-groups See also the Python CNB equivalent of this PR: - heroku/buildpacks-python#261 Note: We don't support controlling the Python version via Poetry's `tool.poetry.dependencies.python` field, since that field typically contains a version range, which is not safe to use. Use the newly added `.python-version` file support instead. For more on this, see the longer explanation over in the Python CNB repo: heroku/buildpacks-python#260 Refs #796. GUS-W-16810914.
runesoerensen
approved these changes
Nov 1, 2024
Merged
edmorley
added a commit
that referenced
this pull request
Nov 6, 2024
The existing Poetry bootstrap process added in #1682 used the pip wheel bundled within the Python stdlib. This required use of pip's `--python` option, which was added to pip in v22.3 in 2022. All of the major Python versions we support have been updated to that pip version or newer, the older patch releases of some of those major Python versions can contain pip versions that are older. Whilst we strongly recommend upgrading to newer patch releases (since older versions are missing security updates and so likely insecure), we still want to support using Poetry on these versions, so the I've adjusted the bootstrap process to no longer use `--python`.
edmorley
added a commit
that referenced
this pull request
Nov 6, 2024
The existing Poetry bootstrap process added in #1682 used the pip wheel bundled within the Python stdlib. This required use of pip's `--python` option, which was added to pip in v22.3 in 2022. All of the major Python versions we support have been updated to that pip version or newer, however, the older patch releases of some of those major Python versions can contain pip versions that are older (for example, whilst latest Python 3.9.x bundles pip v23.0.1, Python 3.9.0 bundles pip v20.2.1). Previously, using those older patch versions would result in: ``` -----> Installing Poetry 1.8.4 Usage: pip <command> [options] no such option: --python ! Error: Unable to install Poetry. ``` Whilst we strongly recommend users upgrade to newer patch releases (since older versions are missing security updates and so likely insecure), we still want to support using Poetry on these versions, so the I've adjusted the bootstrap process to no longer use `--python` to prevent that error.
edmorley
added a commit
to edmorley/python-poetry-buildpack
that referenced
this pull request
Nov 18, 2024
This buildpack is no longer required now that the Heroku Python buildpack supports Poetry natively: - https://devcenter.heroku.com/changelog-items/3050 - heroku/heroku-buildpack-python#1682 As such, a deprecation warning has been added to both the build log and the README. The warning borrows from the implementation here (which uses colour, and also makes sure to ANSI wrap each line individually to prevent issues with streaming logs from `git push` etc): https://github.com/heroku/heroku-buildpack-python/blob/ca99c39f9e3627fff727788206f7fcc4ff064b7f/lib/output.sh#L50-L66 Closes moneymeets#75.
edmorley
added a commit
to edmorley/python-poetry-buildpack
that referenced
this pull request
Nov 18, 2024
This buildpack is no longer required now that the Heroku Python buildpack supports Poetry natively: - https://devcenter.heroku.com/changelog-items/3050 - heroku/heroku-buildpack-python#1682 As such, a deprecation warning has been added to both the build log and the README. The warning borrows from the implementation here (which uses colour, and also makes sure to ANSI wrap each line individually to prevent issues with streaming logs from `git push` etc): https://github.com/heroku/heroku-buildpack-python/blob/ca99c39f9e3627fff727788206f7fcc4ff064b7f/lib/output.sh#L50-L66 Closes moneymeets#75.
edmorley
added a commit
to edmorley/python-poetry-buildpack
that referenced
this pull request
Nov 18, 2024
…ired This buildpack is no longer required now that the Heroku Python buildpack supports Poetry natively: - https://devcenter.heroku.com/changelog-items/3050 - heroku/heroku-buildpack-python#1682 As such, a deprecation warning has been added to both the build log and the README. The warning borrows from the implementation here (which uses colour, and also makes sure to ANSI wrap each line individually to prevent issues with streaming logs from `git push` etc): https://github.com/heroku/heroku-buildpack-python/blob/ca99c39f9e3627fff727788206f7fcc4ff064b7f/lib/output.sh#L50-L66 Closes moneymeets#75.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
After many refactoring/preparation PRs, we're now ready to add support for the package manager Poetry 🎉
https://python-poetry.org
To use Poetry, apps must have a
poetry.lock
lockfile, which can be created by runningpoetry lock
locally, after adding Poetry config topyproject.toml
(which can be done either manually or by usingpoetry init
).For now, if a
requirements.txt
orPipfile
is found it will take precedence overpoetry.lock
for backwards compatibility (in the future this will become a warning then an error). This means users of the third-party python-poetry-buildpack will need to remove that buildpack in order to use the new native Poetry support, since it exports arequirements.txt
file during the build.Poetry is installed into the build cache rather than the slug, so is not available at run-time (since it's not typically needed at run-time and doing so reduces the slug size). The entrypoints of installed dependencies are available on
PATH
, so use ofpoetry run
orpoetry shell
is not required at run-time to use dependencies in the environment.When using Poetry, pip is not installed since Poetry includes its own internal vendored copy that it will use instead (for the small number of Poetry operations for which it still calls out to pip, such as package uninstalls).
During normal (non-CI) builds, the
poetry install --sync
command is run using--only main
so as to only install the main[tool.poetry.dependencies]
dependencies group frompyproject.toml
and not any of the app's other dependency groups (such as test/dev/... groups, eg[tool.poetry.group.test.dependencies]
).On Heroku CI, all default Poetry dependency groups are installed (i.e. all groups minus those marked as
optional = true
).Relevant Poetry docs:
See also the Python CNB equivalent of this PR:
Note: We don't support controlling the Python version via Poetry's
tool.poetry.dependencies.python
field, since that field typically contains a version range, which is not safe to use. Use the newly added.python-version
file support instead. For more on this, see the longer explanation over in the Python CNB repo:heroku/buildpacks-python#260
Closes #796.
Closes #835.
GUS-W-16810914.