Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Parse pyproject.toml statically where possible #1964

Merged
merged 19 commits into from
Mar 25, 2024
Merged
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
74 changes: 74 additions & 0 deletions piptools/build.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,13 @@
import pyproject_hooks
from pip._internal.req import InstallRequirement
from pip._internal.req.constructors import install_req_from_line, parse_req_from_line
from pip._vendor.packaging.markers import Marker
from pip._vendor.packaging.requirements import Requirement

if sys.version_info >= (3, 11):
import tomllib
else:
import tomli as tomllib
hauntsaninja marked this conversation as resolved.
Show resolved Hide resolved

PYPROJECT_TOML = "pyproject.toml"

Expand Down Expand Up @@ -41,6 +48,67 @@ class ProjectMetadata:
build_requirements: tuple[InstallRequirement, ...]


def maybe_statically_parse_project_metadata(
src_file: pathlib.Path,
) -> ProjectMetadata | None:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's better not to use None as a magic sentinel value that the caller shouldn't forget to check for.
Instead, how about raising a LookupError. The handling would be cleaner, then.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given that we're using a static type checker, I think that would actually be less safe. mypy will complain if you do not handle None, but it will not complain if you do not handle the exception

"""
Return the metadata for a project, if it can be statically parsed from pyproject.toml.
hauntsaninja marked this conversation as resolved.
Show resolved Hide resolved

This function is typically significantly faster than invoking a build backend.
Returns None if the project metadata cannot be statically parsed.
"""
if src_file.name != PYPROJECT_TOML:
return None

try:
with open(src_file, "rb") as f:
pyproject_contents = tomllib.load(f)
except tomllib.TOMLDecodeError:
pyproject_contents = {}

# Not valid PEP 621 metadata
if (
"project" not in pyproject_contents
or "name" not in pyproject_contents["project"]
):
hauntsaninja marked this conversation as resolved.
Show resolved Hide resolved
return None

# Dynamic dependencies require build invocation
hauntsaninja marked this conversation as resolved.
Show resolved Hide resolved
dynamic = pyproject_contents["project"].get("dynamic", [])
chrysle marked this conversation as resolved.
Show resolved Hide resolved
if "dependencies" in dynamic or "optional-dependencies" in dynamic:
return None

package_name = pyproject_contents["project"]["name"]
comes_from = f"{package_name} ({src_file})"

extras = pyproject_contents["project"].get("optional-dependencies", {}).keys()
install_requirements = [
InstallRequirement(Requirement(req), comes_from)
for req in pyproject_contents["project"].get("dependencies", [])
]
for extra, reqs in (
pyproject_contents.get("project", {}).get("optional-dependencies", {}).items()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we support Poetry for static metadata parsing?

Copy link
Contributor Author

@hauntsaninja hauntsaninja Mar 24, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If someone is interested in that, it can be a follow-up PR. It's been tough enough getting this one in, so I want to just stick with the standards based thing in this one

):
for req in reqs:
requirement = Requirement(req)
# Note we don't need to modify `requirement` to include this extra
marker = Marker(f"extra == '{extra}'")
install_requirements.append(
InstallRequirement(requirement, comes_from, markers=marker)
)

comes_from = f"{package_name} ({src_file}::build-system.requires)"
build_requirements = [
InstallRequirement(Requirement(req), comes_from)
for req in pyproject_contents.get("build-system", {}).get("requires", [])
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't return complete build deps. It should call the corresponding hooks or not inject the build deps at all. An incomplete list would be harmful, expectations-wise.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed!

]
return ProjectMetadata(
extras=tuple(extras),
requirements=tuple(install_requirements),
build_requirements=tuple(build_requirements),
)


def build_project_metadata(
src_file: pathlib.Path,
build_targets: tuple[str, ...],
Expand All @@ -51,6 +119,8 @@ def build_project_metadata(
"""
Return the metadata for a project.

Attempts to determine the metadata statically from the pyproject.toml file.

Uses the ``prepare_metadata_for_build_wheel`` hook for the wheel metadata
if available, otherwise ``build_wheel``.

Expand All @@ -66,6 +136,10 @@ def build_project_metadata(
:param quiet: Whether to suppress the output of subprocesses.
"""

project_metadata = maybe_statically_parse_project_metadata(src_file)
if project_metadata is not None:
return project_metadata

src_dir = src_file.parent
with _create_project_builder(src_dir, isolated=isolated, quiet=quiet) as builder:
metadata = _build_project_wheel_metadata(builder)
Expand Down
Loading