diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 0f8fcb2c1..e2ac95086 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -1,15 +1,35 @@ +# Contributing -## Developing +Thanks for helping to improve conda-lock! We appreciate your time and effort. -1. Ensure that Conda, Mamba, and Micromamba are installed. Install [mambaforge](https://github.com/conda-forge/miniforge#mambaforge) if you're otherwise not sure which Conda distribution to pick. -2. `micromamba create --name=conda-lock-dev --category=main --category=dev --file=environments/conda-lock.yml` -3. `conda activate conda-lock-dev` -4. `pip install --no-deps --editable .` +## How to install a dev environment -Run the tests to ensure that everything is running correctly. Due to the nature of this project, it hits remote webservers regularly so some tests occasionally fail. This is a normal part of conda-lock development. If you're not sure if your env is borked or the remote webserver is just being flaky, run the tests again. If you're still not sure you can open an issue about. +Of course, we use conda-lock to manage our development environment. -5. `pytest` +1. Get [micromamba](https://mamba.readthedocs.io/en/latest/installation/micromamba-installation.html) -Whilst not strictly necessary; the CI run using github actions will run pre-commit in order to reduce development friction you may want to install the pre-commit hooks: +2. Install and activate the `conda-lock-dev` environment: -6. `pre-commit install` + ```shell + micromamba env create --name=conda-lock-dev --category=main --category=dev --file=environments/conda-lock.yml + micromamba activate conda-lock-dev + ``` + +3. Install `conda-lock` in editable mode. This will also install its runtime + dependencies as defined in `pyproject.toml`. + + ```shell + pip install --no-deps --editable . + ``` + +4. Check to ensure that your Python environment is consistent. + + ```shell + pip check + ``` + +5. Finally, while not strictly necessary, it's recommended to install pre-commit to reduce development friction. + + ```shell + pre-commit install + ``` diff --git a/README.md b/README.md index 9501f4918..ccc40e9b1 100644 --- a/README.md +++ b/README.md @@ -36,6 +36,10 @@ mamba install --channel=conda-forge --name=base conda-lock The first two options are recommended since they install conda-lock into an isolated environment. (Otherwise there is a risk of dependency conflicts.) +## Contributing + +If you would like to contribute to conda-lock, please refer to the [Contributing Guide](CONTRIBUTING.md) for instructions on how to set up your development environment. + ## Basic usage ```bash @@ -78,6 +82,7 @@ conda-lock --lockfile superspecial.conda-lock.yml Rendered `explicit` and `env` lockfiles will be named as `"conda-{platform}.lock"` and `"conda-{platform}.lock.yml` respectively by default. If you want to override that call conda-lock as follows. + ```bash conda-lock -k explicit --filename-template "specific-{platform}.conda.lock" ``` @@ -153,7 +158,6 @@ The default category is `main`. `conda-lock` can also lock the `dependencies.pip` section of [environment.yml][envyaml], using a vendored copy of [Poetry's][poetry] dependency solver. - ### private pip repositories Right now `conda-lock` only supports [legacy](https://warehouse.pypa.io/api-reference/legacy.html) pypi repos with basic auth. Most self-hosted repositories like Nexus, Artifactory etc. use this. You can configure private pip repositories in a similar way to channels, for example: @@ -181,8 +185,8 @@ url = "https://username:password@example.repo/simple" The location of this file can be determined with `python -c 'from conda_lock._vendor.poetry.locations import CONFIG_DIR; print(CONFIG_DIR)'` - Private repositories will be used in addition to `pypi.org`. For projects using `pyproject.toml`, it is possible to [disable `pypi.org` entirely](#disabling-pypiorg). + ### --dev-dependencies/--no-dev-dependencies By default conda-lock will include dev dependencies in the specification of the lock (if the files that the lock @@ -191,6 +195,7 @@ is being built from support them). This can be disabled easily ```bash conda-lock --no-dev-dependencies -f ./recipe/meta.yaml ``` + ### --check-input-hash Under some situation you may want to run conda lock in some kind of automated way (eg as a precommit) and want to not @@ -206,7 +211,7 @@ When the input_hash of the input files, channels match those present in a given By default `conda-lock` will leave basic auth credentials for private conda channels in the lock file. If you wish to strip authentication from the file, provide the `--strip-auth` argument. -``` +```shell conda-lock --strip-auth -f environment.yml ``` @@ -351,7 +356,7 @@ pgsql = ["psycopg2"] These can be referened as follows -``` +```shell conda-lock --extra mysql --extra pgsql -f pyproject.toml ``` @@ -412,10 +417,12 @@ ampel-ztf = {source = "pypi"} ##### Defaulting non-conda dependency sources to PyPI Alternatively, the above behavior is defaulted for all dependencies defined outside of `[tool.conda-lock.dependencies]`, i.e.: + - Default to `pip` dependencies for `[tool.poetry.dependencies]`, `[project.dependencies]`, etc. - Default to `conda` dependencies for `[tool.conda-lock.dependencies]` by explicitly providing `default-non-conda-source = "pip"` in the `[tool.conda-lock]` section, e.g.: + ```toml [tool.conda-lock] default-non-conda-source = "pip" @@ -428,6 +435,7 @@ dependency. #### Lock only conda-lock dependencies To lock only dependencies specified under `[tool.conda-lock]` (i.e. skipping all dependencies specified elsewhere), explicitly provide `skip-non-conda-lock = true` in the `[tool.conda-lock]` section, e.g.: + ```toml [tool.conda-lock] skip-non-conda-lock = true @@ -436,6 +444,7 @@ skip-non-conda-lock = true #### Disabling pypi.org When using private pip repos, it is possible to disable `pypi.org` entirely. This can be useful when using `conda-lock` behind a network proxy that does not allow access to `pypi.org`. + ```toml [tool.conda-lock] allow-pypi-requests = false @@ -447,7 +456,8 @@ In order to use conda-lock in a docker-style context you want to add the lockfil docker container. In order to refresh the lock file just run `conda-lock` again. Given a file tree like -``` + +```text Dockerfile environment.yaml * conda-linux-64.lock diff --git a/environments/README.md b/environments/README.md index 77ae95ea6..0a681f9a5 100644 --- a/environments/README.md +++ b/environments/README.md @@ -1,15 +1,9 @@ -# How to install dev environment +# Conda environment specifications -* First install dev dependencies: +Our development dependencies are specified in [`dev-environment.yaml`](dev-environment.yaml). - ``` - mamba env create -f environments/dev-environment.yaml - mamba activate conda-lock-dev - ``` - -* Then, install `conda-lock` in editable mode. This will also install its runtime - dependencies as defined in `pyproject.toml`. - - ``` - pip install --editable . - ``` +The lockfile [`conda-lock.yml`](conda-lock.yml) is regularly updated using `conda-lock` +via the [`update-lockfile.yaml`](../.github/workflows/update-lockfile.yaml) GHA workflow. +In particular, the lockfile is generated based on the project dependencies specified in +[`pyproject.toml`](../pyproject.toml) the development dependencies specified in +[`dev-environment.yml`](dev-environment.yml). diff --git a/environments/dev-environment.yaml b/environments/dev-environment.yaml index 34b7f9d77..35c67b6c2 100644 --- a/environments/dev-environment.yaml +++ b/environments/dev-environment.yaml @@ -1,3 +1,7 @@ +# Here we specify the development dependencies. + +# The main project dependencies are specified independently in pyproject.toml. + name: conda-lock-dev category: dev channels: