Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: PEP-621 standard Python project metadata #143

Open
wants to merge 19 commits into
base: main
Choose a base branch
from

Conversation

bolinocroustibat
Copy link
Contributor

@bolinocroustibat bolinocroustibat commented Aug 26, 2024

Closes #145 by:

  • Refactor pyproject.toml to become a package-manager agnostic project file
  • Generate locked dependencies files as requirements.txt and requirements-dev.txt files from pyproject.toml Python project file
    Question: do we want to include dependencies hashes in the lock dependencies files? Can be done with pip-compile --generate-hashes
  • Refactor CI not to use Poetry files
  • Change version number in pyproject.tomlso that when CI adds dev version, it is a PEP440 compatible
  • Update README and documentation to explain how to install and generate lock files

@bolinocroustibat bolinocroustibat self-assigned this Aug 26, 2024
@bolinocroustibat bolinocroustibat force-pushed the pep-standard-pyproject branch 6 times, most recently from e3051e3 to 949a72a Compare August 27, 2024 11:38
@bolinocroustibat bolinocroustibat force-pushed the pep-standard-pyproject branch 2 times, most recently from 2ba6f13 to 1ffe9e2 Compare August 27, 2024 12:03
@bolinocroustibat bolinocroustibat force-pushed the pep-standard-pyproject branch 7 times, most recently from 4748c9e to 0faa942 Compare August 27, 2024 13:03
@bolinocroustibat bolinocroustibat changed the title feat: PEP standard pyproject.toml feat: PEP-621 standard pyproject.toml Aug 27, 2024
@bolinocroustibat bolinocroustibat force-pushed the pep-standard-pyproject branch 3 times, most recently from 9fdd413 to 7637389 Compare August 27, 2024 13:42
@bolinocroustibat bolinocroustibat changed the title feat: PEP-621 standard pyproject.toml feat: PEP-621 standard Python project metadata Aug 27, 2024
Copy link
Contributor

@magopian magopian left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks NICE! Disclaimer, I didn't double check the requirements nor the install of the dependencies, I can do it if needed, if we don't trust the CI ;)

.circleci/config.yml Outdated Show resolved Hide resolved
README.md Outdated Show resolved Hide resolved
README.md Outdated Show resolved Hide resolved
README.md Outdated Show resolved Hide resolved
pyproject.toml Show resolved Hide resolved
Copy link
Contributor

@maudetes maudetes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for this standardisation! 🚀

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After this PR, we need to check if there is any usage of poetry in our current infra since it may have been used to install the hydra.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We may want to add a pre-commit to generate the requirements lock files.
See for example the logic on udata side: https://github.com/opendatateam/udata/blob/a83b29a6516b3ededda7d797fb004f4d1eede2fa/.pre-commit-config.yaml#L3

CHANGELOG.md Outdated Show resolved Hide resolved
...or `pip-sync requirements-dev.txt` with [pip-tools](https://pip-tools.readthedocs.io/en/stable/).

To update the lock files, you can use any modern Python package manager (except Poetry) like [pip-tools](https://pip-tools.readthedocs.io/en/stable/), [PDM](https://pdm.fming.dev/) or [uv](https://uv.readthedocs.io/en/latest/), while defining `requirement.txt` and `requirement-dev.txt` as the output lock files.
With [pip-tools](https://pip-tools.readthedocs.io/en/stable/), the commands are `pip-compile` for requirements.txt, and `pip-compile --extra dev -o requirements-dev.txt` for requirements-dev.txt.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to specify the constraint to compile dev deps with requirements.txt? Or is it already implicit with the way --extra works? Ex adding -c requirements.txt.
We don't want a version to diverge between requirements.txt and requirements-dev.txt.

Comment on lines -88 to -89
udata-hydra = "udata_hydra.cli:run"
udata-hydra-crawl = "udata_hydra.crawl:run"
udata-hydra-app = "udata_hydra.app:run"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After this PR, we must check if these scripts were used in our infra

README.md Outdated Show resolved Hide resolved
README.md Outdated Show resolved Hide resolved
@@ -84,17 +89,29 @@ jobs:
steps:
- attach_workspace:
at: .
- run:
name: Install build dependencies, not including dev ones
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need to reinstall venv here?

pip install --upgrade pip
python3 -m venv .venv_build && source .venv_build/bin/activate
pip install -r requirements.txt
pip install build
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we prefer using build or pip-wheel? I'm unsure which one to prefer reading https://pip.pypa.io/en/stable/cli/pip_wheel/#differences-to-build.

# Conflicts:
#	pyproject.toml
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

hydra: Convert to a package-manager-agnostic, PEP-621 standard Python project metadata
3 participants