Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add initial support for Poetry #261

Merged
merged 1 commit into from
Sep 4, 2024
Merged

Add initial support for Poetry #261

merged 1 commit into from
Sep 4, 2024

Conversation

edmorley
Copy link
Member

@edmorley edmorley commented Sep 3, 2024

The Python package manager Poetry is now supported for installing app dependencies:
https://python-poetry.org

To use Poetry, apps must have a poetry.lock lockfile, which can be created by running poetry lock locally, after adding Poetry config to pyproject.toml (which can be done either manually or by using poetry init). Apps must only have one package manager file (either requirements.txt or poetry.lock, but not both) otherwise the buildpack will abort the build with an error (which will help prevent some of the types of support tickets we see in the classic buildpack with users unknowingly mixing and matching pip + Pipenv).

Poetry is installed into a build-only layer (to reduce the final app image size), so is not available at run-time. The app dependencies are installed into a virtual environment (the same as for pip after #257, for the reasons described in #253), which is on PATH so does not need explicit activation when using the app image. As such, use of poetry run or poetry shell is not required at run-time to use dependencies in the environment.

When using Poetry, pip is not installed (possible thanks to #258), since Poetry includes its own internal vendored copy that it will use instead (for the small number of Poetry operations for which it still calls out to pip, such as package uninstalls).

Both the Poetry and app dependencies layers are cached, however, the Poetry download/wheel cache is not cached, since using it is slower than caching the dependencies layer (for more details see the comments on poetry_dependencies::install_dependencies).

The poetry install --sync command is run using --only main so as to only install the main [tool.poetry.dependencies] dependencies group from pyproject.toml, and not any of the app's other dependency groups (such as test/dev/... groups, eg [tool.poetry.group.test.dependencies]).

I've marked this semver: major since in the (probably unlikely) event there are any early-adopter projects using this CNB that have both a requirements.txt and poetry.lock then this change will cause them to error (until one of the files is deleted).

Relevant Poetry docs:

Work that will be handled later:

Closes #7.
GUS-W-9607867.
GUS-W-9608286.
GUS-W-9608295.

@edmorley edmorley added enhancement New feature or request semver: major labels Sep 3, 2024
@edmorley edmorley self-assigned this Sep 3, 2024
@edmorley edmorley marked this pull request as ready for review September 3, 2024 16:36
@edmorley edmorley requested a review from a team as a code owner September 3, 2024 16:36
The Python package manager Poetry is now supported for installing app
dependencies:
https://python-poetry.org

To use Poetry, apps must have a `poetry.lock` lockfile, which can be
created by running `poetry lock` locally, after adding Poetry config to
`pyproject.toml` (which can be done either manually or by using
`poetry init`). Apps must only have one package manager file (either
`requirements.txt` or `poetry.lock`, but not both) otherwise the
buildpack will abort the build with an error (which will help prevent
some of the types of support tickets we see in the classic buildpack
with users unknowingly mixing and matching pip + Pipenv).

Poetry is installed into a build-only layer (to reduce the final app
image size), so is not available at run-time. The app dependencies are
installed into a virtual environment (the same as for pip after #257,
for the reasons described in #253), which is on `PATH` so does not need
explicit activation when using the app image. As such, use of
`poetry run` or `poetry shell` is not required at run-time to use
dependencies in the environment.

When using Poetry, pip is not installed (possible thanks to #258), since
Poetry includes its own internal vendored copy that it will use instead
(for the small number of Poetry operations for which it still calls out
to pip, such as package uninstalls).

Both the Poetry and app dependencies layers are cached, however, the
Poetry download/wheel cache is not cached, since using it is slower than
caching the dependencies layer (for more details see the comments on
`poetry_dependencies::install_dependencies`).

The `poetry install --sync` command is run using `--only main` so as to
only install the main `[tool.poetry.dependencies]` dependencies group
from `pyproject.toml`, and not any of the app's other dependency groups
(such as test/dev groups, eg `[tool.poetry.group.test.dependencies]`).

I've marked this `semver: major` since in the (probably unlikely) event
there are any early-adopter projects using this CNB that have both a
`requirements.txt` and `poetry.lock` then this change will cause them to
error (until one of the files is deleted).

Relevant Poetry docs:
- https://python-poetry.org/docs/cli/#install
- https://python-poetry.org/docs/configuration/
- https://python-poetry.org/docs/managing-dependencies/#dependency-groups

Work that will be handled later:
- Support for selecting Python version via `tool.poetry.dependencies.python`:
  #260
- Build output and error messages polish/CX review (this will be performed
  when switching the buildpack to the new logging style).
- More detailed user-facing docs:
  #11

Closes #7.
GUS-W-9607867.
GUS-W-9608286.
GUS-W-9608295.
@edmorley edmorley enabled auto-merge (squash) September 4, 2024 08:42
@edmorley edmorley merged commit e691799 into main Sep 4, 2024
7 checks passed
@edmorley edmorley deleted the poetry-support branch September 4, 2024 08:43
heroku-linguist bot added a commit that referenced this pull request Sep 4, 2024
## heroku/python

### Added

- Added initial support for the Poetry package manager. ([#261](#261))
@heroku-linguist heroku-linguist bot mentioned this pull request Sep 4, 2024
heroku-linguist bot added a commit that referenced this pull request Sep 4, 2024
## heroku/python

### Added

- Added initial support for the Poetry package manager. ([#261](#261))

Co-authored-by: heroku-linguist[bot] <136119646+heroku-linguist[bot]@users.noreply.github.com>
heroku-linguist bot added a commit to heroku/cnb-builder-images that referenced this pull request Sep 4, 2024
## heroku/python

### Added

- Added initial support for the Poetry package manager. ([#261](heroku/buildpacks-python#261))
edmorley added a commit to heroku/heroku-buildpack-python that referenced this pull request Nov 6, 2024
After many refactoring/preparation PRs, we're now ready to add support
for the package manager Poetry:
https://python-poetry.org

To use Poetry, apps must have a `poetry.lock` lockfile, which can be
created by running `poetry lock` locally, after adding Poetry config to
`pyproject.toml` (which can be done either manually or by using
`poetry init`).

For now, if a `requirements.txt` or `Pipfile` is found it will take
precedence over `poetry.lock` for backwards compatibility (in the future
this will become a warning then an error). This means users of the
third-party `python-poetry-buildpack` will need to remove that buildpack
in order to use the new native Poetry support, since it exports a
`requirements.txt` file during the build.

Poetry is installed into the build cache rather than the slug, so is not
available at run-time (since it's not typically needed at run-time and
doing so reduces the slug size). The entrypoints of installed
dependencies are available on `PATH`, so use of `poetry run` or
`poetry shell` is not required at run-time to use dependencies in the
environment.

When using Poetry, pip is not installed since Poetry includes its own
internal vendored copy that it will use instead (for the small number
of Poetry operations for which it still calls out to pip, such as
package uninstalls).

During normal (non-CI) builds, the `poetry install --sync` command is
run using `--only main` so as to only install the main
`[tool.poetry.dependencies]` dependencies group from `pyproject.toml`
and not any of the app's other dependency groups (such as test/dev/...
groups, eg `[tool.poetry.group.test.dependencies]`).

On Heroku CI, all default Poetry dependency groups are installed (i.e.
all groups minus those marked as `optional = true`).

Relevant Poetry docs:
- https://python-poetry.org/docs/cli/#install
- https://python-poetry.org/docs/configuration/
- https://python-poetry.org/docs/managing-dependencies/#dependency-groups

See also the Python CNB equivalent of this PR:
- heroku/buildpacks-python#261

Note: We don't support controlling the Python version via Poetry's
`tool.poetry.dependencies.python` field, since that field typically
contains a version range, which is not safe to use. Use the newly
added `.python-version` file support instead. For more on this, see
the longer explanation over in the Python CNB repo:
heroku/buildpacks-python#260

Closes #796.
Closes #835.
GUS-W-16810914.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request semver: major
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support the Poetry package manager
2 participants