Skip to content

Commit

Permalink
Add function to monkey-patch Session.install (#108)
Browse files Browse the repository at this point in the history
* Forward keyword arguments to Session.install

* Add nox_poetry.patch()

* Rename build_package parameter {format => distribution_format}

* Ignore RST201 "Block quote ends without a blank line"

* Use nox_poetry.patch() in noxfile.py

* Rename local variable {format => distribution_format}

* Monkey-patch core.Session_install in tests

* Simplify imports in tests

* Add test case for nox_poetry.patch

* Update README.rst
  • Loading branch information
cjolowicz authored Sep 27, 2020
1 parent 4fe0119 commit 3caa2fa
Show file tree
Hide file tree
Showing 6 changed files with 108 additions and 59 deletions.
2 changes: 1 addition & 1 deletion .flake8
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[flake8]
select = B,B9,C,D,DAR,E,F,N,RST,S,W
ignore = E203,E501,RST203,RST301,W503
ignore = E203,E501,RST201,RST203,RST301,W503
max-line-length = 80
max-complexity = 10
docstring-convention = google
Expand Down
72 changes: 44 additions & 28 deletions README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -33,13 +33,19 @@ nox-poetry
:alt: Black


Helper functions for using Poetry_ inside Nox_ sessions
Use Poetry_ inside Nox_ sessions

This package provides a drop-in replacement for ``session.install`` in Nox sessions.
It modifies its behavior in two ways:

- Packages are pinned to the versions specified in Poetry's lock file.
- The argument ``"."`` is replaced by a wheel built from the package.


Installation
------------

You can install ``nox-poetry`` via pip_ from the Python Package Index:
Install ``nox-poetry`` from the Python Package Index:

.. code:: console
Expand All @@ -59,23 +65,32 @@ use the following command to install this package into the same environment:
Usage
-----

- The function ``nox_poetry.install(session, ...)`` is a drop-in replacement for ``session.install(...)``.
- Packages installed like this must be declared as dependencies using Poetry.
- Use the constants ``WHEEL`` and ``SDIST`` to build and install your own package.
Invoke ``nox_poetry.patch()`` at the top of your ``noxfile.py``, and you're good to go.
``nox-poetry`` intercepts calls to ``session.install``
and uses Poetry to export a `constraints file`_ and build the package behind the scenes.

If you prefer a less magical, more explicit approach,
you can also invoke ``nox_poetry.install(session, ...)`` instead of ``session.install(...)``.
Pass ``nox_poetry.WHEEL`` or ``nox_poetry.SDIST`` to build and install the local package
using the specified distribution format.

Packages installed in this way must be managed as dependencies in Poetry.

For example, the following Nox session runs your test suite:

.. code:: python
# noxfile.py
import nox
from nox.sessions import Session
from nox_poetry import install, WHEEL
import nox_poetry
nox_poetry.patch()
@nox.session
def tests(session: Session) -> None:
"""Run the test suite."""
install(session, WHEEL, "pytest")
session.install(".")
session.install("pytest")
session.run("pytest")
More precisely, the session builds a wheel from the local package,
Expand All @@ -86,24 +101,16 @@ invokes ``pytest`` to run the test suite against the installation.
Why?
----

Compare the session above to one written without this package:
Consider what would happen in the above session with an unpatched ``session.install``:

.. code:: python
@nox.session
def tests(session: Session) -> None:
"""Run the test suite."""
session.install(".")
session.install("pytest")
session.run("pytest")
This session has several problems:

- Poetry is installed as a build backend every time.
- Package dependencies are only constrained by the wheel metadata, not by the lock file.
In other words, their versions are not *pinned*.
- The ``pytest`` dependency is not constrained at all.
- Package dependencies would only be constrained by the wheel metadata, not by the lock file.
In other words, their versions would not be *pinned*.
- The ``pytest`` dependency would not be constrained at all.
- Poetry would be installed as a build backend every time
(although this could be avoided by passing the ``"--no-build-isolation"`` option).

Unpinned dependencies mean that your checks are not reproducible and deterministic,
which can lead to surprises in Continuous Integration and when collaborating with others.
You can solve these issues by declaring ``pytest`` as a development dependency,
and installing your package and its dependencies using ``poetry install``:

Expand All @@ -115,7 +122,7 @@ and installing your package and its dependencies using ``poetry install``:
session.run("poetry", "install", external=True)
session.run("pytest")
Unfortunately, this approach creates problems of its own:
Unfortunately, this approach comes with its own set of problems:

- Checks run against an editable installation of your package,
i.e. your local copy of the code, instead of the installed wheel your users see.
Expand All @@ -128,9 +135,10 @@ Unfortunately, this approach creates problems of its own:
Third-party packages are installed by exporting the lock file in ``requirements.txt`` format,
and passing it as a `constraints file`_ to pip.
When installing your own package, Poetry is used to build a wheel, which is then installed by pip.
This approach has some advantages:

- You can declare tools like ``pytest`` as development dependencies in Poetry.
In summary, this approach brings the following advantages:

- You can manage tools like ``pytest`` as development dependencies in Poetry.
- Dependencies are pinned by Poetry's lock file, making checks predictable and deterministic.
- You can run checks against an installed wheel, instead of your local copy of the code.
- Every tool can run in an isolated environment with minimal dependencies.
Expand All @@ -144,7 +152,14 @@ __ https://cjolowicz.github.io/posts/hypermodern-python-03-linting/#managing-dep
API
---

``nox_poetry.install(session, *args)``:
``nox_poetry.patch(*, distribution_format=nox_poetry.WHEEL)``:
Monkey-patch `nox.sessions.Session.install`_ to use ``nox_poetry.install``.
The optional ``distribution_format`` parameter determines
how to handle the special ``"."`` argument.
By default, this is replaced by a wheel built from the package.
Pass ``nox_poetry.SDIST`` to build an sdist archive instead.

``nox_poetry.install(session, *args, **kwargs)``:
Install packages into a Nox session using Poetry.

The ``nox_poetry.install`` function
Expand All @@ -156,6 +171,7 @@ API
typically just the package or packages to be installed.
The constants ``WHEEL`` and ``SDIST`` are replaced by a distribution archive
built for the local package.
Keyword arguments are the same as those for `nox.sessions.Session.run`_.


Contributing
Expand Down
28 changes: 14 additions & 14 deletions noxfile.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,10 @@
import nox
from nox.sessions import Session

from nox_poetry import export_requirements
from nox_poetry import install
from nox_poetry import WHEEL
import nox_poetry


nox_poetry.patch()


package = "nox_poetry"
Expand Down Expand Up @@ -78,8 +79,7 @@ def activate_virtualenv_in_precommit_hooks(session: Session) -> None:
def precommit(session: Session) -> None:
"""Lint using pre-commit."""
args = session.posargs or ["run", "--all-files", "--show-diff-on-failure"]
install(
session,
session.install(
"black",
"darglint",
"flake8",
Expand All @@ -100,16 +100,16 @@ def precommit(session: Session) -> None:
@nox.session(python="3.8")
def safety(session: Session) -> None:
"""Scan dependencies for insecure packages."""
install(session, "safety")
requirements = export_requirements(session)
session.install("safety")
requirements = nox_poetry.export_requirements(session)
session.run("safety", "check", f"--file={requirements}", "--bare")


@nox.session(python=python_versions)
def mypy(session: Session) -> None:
"""Type-check using mypy."""
args = session.posargs or ["src", "tests", "docs/conf.py"]
install(session, WHEEL, "mypy")
session.install(".", "mypy")
session.run("mypy", *args)
if not session.posargs:
session.run("mypy", f"--python-executable={sys.executable}", "noxfile.py")
Expand All @@ -118,7 +118,7 @@ def mypy(session: Session) -> None:
@nox.session(python=python_versions)
def tests(session: Session) -> None:
"""Run the test suite."""
install(session, WHEEL, "coverage[toml]", "pytest")
session.install(".", "coverage[toml]", "pytest")
try:
session.run("coverage", "run", "--parallel", "-m", "pytest", *session.posargs)
finally:
Expand All @@ -132,7 +132,7 @@ def coverage(session: Session) -> None:
has_args = session.posargs and len(session._runner.manifest) == 1
args = session.posargs if has_args else ["report"]

install(session, "coverage[toml]")
session.install("coverage[toml]")

if not has_args and any(Path().glob(".coverage.*")):
session.run("coverage", "combine")
Expand All @@ -143,23 +143,23 @@ def coverage(session: Session) -> None:
@nox.session(python=python_versions)
def typeguard(session: Session) -> None:
"""Runtime type checking using Typeguard."""
install(session, WHEEL, "pytest", "typeguard")
session.install(".", "pytest", "typeguard")
session.run("pytest", f"--typeguard-packages={package}", *session.posargs)


@nox.session(python=python_versions)
def xdoctest(session: Session) -> None:
"""Run examples with xdoctest."""
args = session.posargs or ["all"]
install(session, WHEEL, "xdoctest")
session.install(".", "xdoctest")
session.run("python", "-m", "xdoctest", package, *args)


@nox.session(name="docs-build", python="3.8")
def docs_build(session: Session) -> None:
"""Build the documentation."""
args = session.posargs or ["docs", "docs/_build"]
install(session, WHEEL, "sphinx")
session.install(".", "sphinx")

build_dir = Path("docs", "_build")
if build_dir.exists():
Expand All @@ -172,7 +172,7 @@ def docs_build(session: Session) -> None:
def docs(session: Session) -> None:
"""Build and serve the documentation with live reloading on file changes."""
args = session.posargs or ["--open-browser", "docs", "docs/_build"]
install(session, WHEEL, "sphinx", "sphinx-autobuild")
session.install(".", "sphinx", "sphinx-autobuild")

build_dir = Path("docs", "_build")
if build_dir.exists():
Expand Down
4 changes: 3 additions & 1 deletion src/nox_poetry/__init__.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
"""Using Poetry in Nox sessions."""
from nox_poetry.core import export_requirements
from nox_poetry.core import install
from nox_poetry.core import patch
from nox_poetry.poetry import DistributionFormat


Expand All @@ -10,6 +11,7 @@
__all__ = [
"export_requirements",
"install",
"WHEEL",
"patch",
"SDIST",
"WHEEL",
]
42 changes: 33 additions & 9 deletions src/nox_poetry/core.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
"""Core functions."""
import hashlib
from pathlib import Path
from typing import Any
from typing import List
from typing import Union

from nox.sessions import Session
Expand All @@ -9,6 +11,9 @@
from nox_poetry.poetry import Poetry


Session_install = Session.install


def export_requirements(session: Session) -> Path:
"""Export the lock file to requirements format.
Expand All @@ -32,29 +37,31 @@ def export_requirements(session: Session) -> Path:
return path


def build_package(session: Session, *, format: DistributionFormat) -> str:
def build_package(session: Session, *, distribution_format: DistributionFormat) -> str:
"""Build a distribution archive for the package.
Args:
session: The Session object.
format: The distribution format, either wheel or sdist.
distribution_format: The distribution format, either wheel or sdist.
Returns:
The file URL for the distribution package.
"""
# Provide a hash for the wheel since the constraints file uses hashes.
# https://pip.pypa.io/en/stable/reference/pip_install/#hash-checking-mode
poetry = Poetry(session)
wheel = Path("dist") / poetry.build(format=format)
wheel = Path("dist") / poetry.build(format=distribution_format)
digest = hashlib.sha256(wheel.read_bytes()).hexdigest()

return f"file://{wheel.resolve().as_posix()}#sha256={digest}"


def install(session: Session, *args: Union[DistributionFormat, str]) -> None:
def install(
session: Session, *args: Union[DistributionFormat, str], **kwargs: Any
) -> None:
"""Install packages into the session's virtual environment.
This function is a wrapper for nox.sessions.Session.install.
This function is a wrapper for ``nox.sessions.Session.install``.
The packages must be managed as dependencies in Poetry.
Expand All @@ -63,20 +70,37 @@ def install(session: Session, *args: Union[DistributionFormat, str]) -> None:
args: Command-line arguments for ``pip install``. The ``WHEEL``
and ``SDIST`` constants are replaced by a wheel or sdist
archive built from the local package.
kwargs: Keyword-arguments for ``session.install``.
"""
resolved = {
arg: (
build_package(session, format=arg)
build_package(session, distribution_format=arg)
if isinstance(arg, DistributionFormat)
else arg
)
for arg in args
}

for format in DistributionFormat:
package = resolved.get(format)
for distribution_format in DistributionFormat:
package = resolved.get(distribution_format)
if package is not None:
session.run("pip", "uninstall", "--yes", package, silent=True)

requirements = export_requirements(session)
session.install(f"--constraint={requirements}", *resolved.values())
Session_install(
session, f"--constraint={requirements}", *resolved.values(), **kwargs
)


def patch(
*, distribution_format: DistributionFormat = DistributionFormat.WHEEL
) -> None:
"""Monkey-patch nox.sessions.Session.install with nox_poetry.install."""

def patched_install(self: Session, *args: str, **kwargs: Any) -> None:
newargs: List[Union[DistributionFormat, str]] = [
distribution_format if arg == "." else arg for arg in args
]
install(self, *newargs, **kwargs)

Session.install = patched_install # type: ignore[assignment]
Loading

0 comments on commit 3caa2fa

Please sign in to comment.