Skip to content

Commit

Permalink
Add tutorial for Python and pixi build
Browse files Browse the repository at this point in the history
  • Loading branch information
Hofer-Julian committed Dec 16, 2024
1 parent 5e4c0aa commit 86a3d55
Show file tree
Hide file tree
Showing 12 changed files with 146 additions and 39 deletions.
107 changes: 107 additions & 0 deletions docs/build/python.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,107 @@
# Tutorial: Doing Python development with Pixi

In this tutorial, we will show you how to create a simple Python package with pixi.

## Why is this useful?

Pixi builds upon the conda ecosystem, which allows you to create a Python environment with all the dependencies you need.
Unlike PyPI, the conda ecosystem is cross-language and also offers packages written in Rust, R, C, C++ and many other languages.

By building a Python package with pixi, you can:

- build both conda and Python packages with the same tool
- manage Python packages and packages written in other languages in the same workspace

## Let's get started

First, we create a simple Python package with a `pyproject.toml` and a single Python file.
The package will be called `rich_example`, so we will create the following structure

```shell
├── src # (1)!
│ └── rich_example
│ └── __init__.py
└── pyproject.toml
```

1. This project uses a src-layout, but pixi supports both [flat- and src-layouts](https://packaging.python.org/en/latest/discussions/src-layout-vs-flat-layout/#src-layout-vs-flat-layout).


The Python package has a single function `main`.
Calling that, will print a table containing the name, age and city of three people.

```py title="src/rich_example/__init__.py"
--8<-- "docs/source_files/pixi_projects/pixi_build_python/src/rich_example/__init__.py"
```


The metadata of the Python package is defined in `pyproject.toml`.

```toml title="pyproject.toml"
--8<-- "docs/source_files/pixi_projects/pixi_build_python/pyproject.toml"
```

1. We use the `rich` package to print the table in the terminal.
2. By specifying a script, the executable `rich-example-main` will be available in the environment. When being called it will in return call the `main` function of the `rich_example` module.
3. One can choose multiple backends to build a Python package, we choose `hatchling` which works well without additional configuration.


### Adding a `pixi.toml`

What we have in the moment, constitutes a full Python package.
It could be uploaded to [PyPI](https://pypi.org/) as-is.

However, we still need a tool to manage our environments and if we want other pixi projects to depend on our tool, we need to include more information.
We will do exactly that by creating a `pixi.toml`.

!!! note
The pixi manifest can be in its own `pixi.toml` file or integrated in `pyproject.toml`
In this tutorial, we will use `pixi.toml`.
If you want everything integrated in `pyproject.toml` just copy the content of `pixi.toml` in this tutorial to your `pyproject.toml` and append `tool.pixi` to each table.

The file structure will then look like this:

```shell
├── src
│ └── rich_example
│ └── __init__.py
├── pixi.toml
└── pyproject.toml
```

This is the content of the `pixi.toml`:

```toml title="pixi.toml"
--8<-- "docs/source_files/pixi_projects/pixi_build_python/pixi.toml"
```

1. In `workspace` information is set that is shared across all packages in the workspace.
2. In `dependencies` you specify all of your pixi packages. Here, this includes only our own package that is defined further below under `package`
3. We define a task that runs the `rich-example-main` executable we defined earlier. You can learn more about tasks in this [section](../features/advanced_tasks.md)
4. In `package` we define the actual pixi package. This information will be used when other pixi packages or workspaces depend on our package or when we upload it to a conda channel.
5. The same way, Python uses build backends to build a Python package, pixi uses build backends to build pixi packages. `pixi-build-python` creates a pixi package out of a Python package.
6. In `host-dependencies`, we add Python dependencies that are necessary to build the Python package. By adding them here as well, the dependencies will come from the conda channel rather than PyPI.
7. In `run-dependencies`, we add the Python dependencies needed during runtime.


When we now run `pixi run start`, we get the following output:

```
┏━━━━━━━━━━━━━━┳━━━━━┳━━━━━━━━━━━━━┓
┃ Name ┃ Age ┃ City ┃
┡━━━━━━━━━━━━━━╇━━━━━╇━━━━━━━━━━━━━┩
│ John Doe │ 30 │ New York │
│ Jane Smith │ 25 │ Los Angeles │
│ Tim de Jager │ 35 │ Utrecht │
└──────────────┴─────┴─────────────┘
```

## Conclusion

In this tutorial, we created a pixi package based on Python.
It can be used as-is, to upload to a conda channel or to PyPI.
In another tutorial we will learn how to add multiple pixi packages to the same workspace and let one pixi package use another.

Thanks for reading! Happy Coding 🚀

Any questions? Feel free to reach out or share this tutorial on [X](https://twitter.com/prefix_dev), [join our Discord](https://discord.gg/kKV8ZxyzY4), send us an [e-mail](mailto:hi@prefix.dev) or follow our [GitHub](https://github.com/prefix-dev).

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Original file line number Diff line number Diff line change
@@ -1,31 +1,27 @@
[workspace]
[workspace] # (1)!
channels = ["https://prefix.dev/conda-forge"]
platforms = ["win-64", "linux-64", "osx-arm64", "osx-64"]
preview = ["pixi-build"]

[dependencies]
[dependencies] # (2)!
rich_example = { path = "." }

[tasks]
test = "rich-example-main"
[tasks] # (3)!
start = "rich-example-main"

[package]
[package] # (4)!
name = "rich_example"
version = "0.1.0"

[build-system]
[build-system] # (5)!
build-backend = { name = "pixi-build-python", version = "*" }
channels = [
"https://prefix.dev/pixi-build-backends",
"https://prefix.dev/conda-forge",
]

[host-dependencies]
# To be able to install this pyproject we need to install the dependencies of
# the python build-system defined above. Note that different from the
# pyproject build-system this refers to a conda package instead of a pypi
# package.
[host-dependencies] # (6)!
hatchling = "==1.26.3"

[run-dependencies]
[run-dependencies] # (7)!
rich = ">=13.9.4,<14"
10 changes: 10 additions & 0 deletions docs/source_files/pixi_projects/pixi_build_python/pyproject.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
[project]
dependencies = ["rich"] # (1)!
name = "rich_example"
requires-python = ">= 3.11"
scripts = { rich-example-main = "rich_example:main" } # (2)!
version = "0.1.0"

[build-system] # (3)!
build-backend = "hatchling.build"
requires = ["hatchling"]
Original file line number Diff line number Diff line change
Expand Up @@ -5,11 +5,11 @@
def main() -> None:
console = Console()

table = Table(title="Simple Rich Example")
table = Table()

table.add_column("Name", justify="right", style="cyan", no_wrap=True)
table.add_column("Age", style="magenta")
table.add_column("City", justify="right", style="green")
table.add_column("Name")
table.add_column("Age")
table.add_column("City")

table.add_row("John Doe", "30", "New York")
table.add_row("Jane Smith", "25", "Los Angeles")
Expand Down

This file was deleted.

1 change: 1 addition & 0 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -126,6 +126,7 @@ nav:
- Building Packages:
- Getting started: build/getting_started.md
- Dependency Types: build/dependency_types.md
- Building a Python package: build/python.md
- Advanced:
- Authentication: advanced/authentication.md
- Channel Logic: advanced/channel_priority.md
Expand Down
5 changes: 5 additions & 0 deletions tests/integration_python/conftest.py
Original file line number Diff line number Diff line change
Expand Up @@ -77,3 +77,8 @@ def non_self_expose_channel_1(channels: Path) -> str:
@pytest.fixture
def non_self_expose_channel_2(channels: Path) -> str:
return channels.joinpath("non_self_expose_channel_2").as_uri()


@pytest.fixture
def doc_pixi_projects() -> Path:
return Path(__file__).parents[2].joinpath("docs", "source_files", "pixi_projects")
11 changes: 6 additions & 5 deletions tests/integration_python/pixi_build/test_build.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,16 +5,17 @@
from ..common import verify_cli_command


def test_build_conda_package(pixi: Path, examples_dir: Path, tmp_pixi_workspace: Path) -> None:
def test_build_conda_package(pixi: Path, tmp_pixi_workspace: Path, doc_pixi_projects: Path) -> None:
"""
This one tries to build the rich example project
"""
pyproject = examples_dir / "rich_example"
target_dir = tmp_pixi_workspace / "pyproject"
shutil.copytree(pyproject, target_dir)

project = doc_pixi_projects / "pixi_build_python"
target_dir = tmp_pixi_workspace / "project"
shutil.copytree(project, target_dir)
shutil.rmtree(target_dir.joinpath(".pixi"), ignore_errors=True)

manifest_path = target_dir / "pyproject.toml"
manifest_path = target_dir / "pixi.toml"

# build it
verify_cli_command(
Expand Down
7 changes: 3 additions & 4 deletions tests/integration_python/test_docs.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,17 +6,16 @@


@pytest.mark.slow
def test_doc_pixi_projects(pixi: Path, tmp_pixi_workspace: Path) -> None:
def test_doc_pixi_projects(pixi: Path, tmp_pixi_workspace: Path, doc_pixi_projects: Path) -> None:
# TODO: Setting the cache dir shouldn't be necessary!
env = {"PIXI_CACHE_DIR": str(tmp_pixi_workspace.joinpath("pixi_cache"))}
pixi_project_dir = Path(__file__).parents[2].joinpath("docs", "source_files", "pixi_projects")
target_dir = tmp_pixi_workspace.joinpath("pixi_projects")
shutil.copytree(pixi_project_dir, target_dir)
shutil.copytree(doc_pixi_projects, target_dir)

for pixi_project in target_dir.iterdir():
shutil.rmtree(pixi_project.joinpath(".pixi"))
manifest = pixi_project.joinpath("pixi.toml")
# Run the test command
verify_cli_command(
[pixi, "run", "--manifest-path", manifest, "test"], ExitCode.SUCCESS, env=env
[pixi, "run", "--manifest-path", manifest, "start"], ExitCode.SUCCESS, env=env
)

0 comments on commit 86a3d55

Please sign in to comment.