Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Switch project to use pyproject.toml #316

Merged
merged 6 commits into from
Jul 20, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions .flake8
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
[flake8]
exclude =
.eggs,
build,
extend-ignore =
# math, https://github.com/PyCQA/pycodestyle/issues/513
W503,
per-file-ignores =
# ignore unused imports
__init__.py: F401
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
name: Flake8
name: Linter

on:
push:
Expand Down
6 changes: 3 additions & 3 deletions .github/workflows/publish.yml
Original file line number Diff line number Diff line change
Expand Up @@ -21,15 +21,15 @@ jobs:
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install setuptools wheel twine
pip install build twine virtualenv

# PyPI package
- name: Build and publish
env:
TWINE_USERNAME: ${{ secrets.PYPI_USERNAME }}
TWINE_PASSWORD: ${{ secrets.PYPI_PASSWORD }}
run: |
python setup.py sdist bdist_wheel
python -m build
python -m twine upload dist/*

# Remove apt repos that are known to break from time to time
Expand Down Expand Up @@ -68,7 +68,7 @@ jobs:
CHANGELOG="${CHANGELOG//$'\n'/'%0A'}"
CHANGELOG="${CHANGELOG//$'\r'/'%0D'}"
echo "Got changelog: $CHANGELOG"
echo "::set-output name=body::$CHANGELOG"
echo "body=$CHANGELOG" >> $GITHUB_OUTPUT

- name: Create release on Github
id: create_release
Expand Down
16 changes: 13 additions & 3 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,18 @@
# $ pre-commit install
# $ pre-commit run --all-files
#
#
default_language_version:
python: python3.8

repos:
- repo: https://github.com/pycqa/flake8
rev: '5.0.4'
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.0.276
hooks:
- id: ruff
- repo: https://github.com/codespell-project/codespell
rev: v2.2.4
hooks:
- id: flake8
- id: codespell
additional_dependencies:
- tomli
70 changes: 41 additions & 29 deletions CONTRIBUTING.rst
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,12 @@ Contributing

Everyone is invited to contribute to this project.
Feel free to create a `pull request`_ .
If you find errors, omissions, inconsistencies or other things
that need improvement, please create an issue_.
If you find errors,
omissions,
inconsistencies,
or other things
that need improvement,
please create an issue_.

.. _issue: https://github.com/audeering/audb/issues/new/
.. _pull request: https://github.com/audeering/audb/compare/
Expand All @@ -13,30 +17,35 @@ that need improvement, please create an issue_.
Development Installation
------------------------

Instead of pip-installing the latest release from PyPI,
Instead of pip-installing the latest release from PyPI_,
you should get the newest development version from Github_::

git clone https://github.com/audeering/audb/
cd audb
# Create virutal environment for this project
# e.g.
# virtualenv --python="python3" $HOME/.envs/audb
# source $HOME/.envs/audb/bin/activate
pip install -r requirements.txt
git clone https://github.com/audeering/audb/
cd audb
# Create virtual environment for this project
# e.g.
# virtualenv --python="python3" $HOME/.envs/audb
# source $HOME/.envs/audb/bin/activate
pip install -r requirements.txt

.. _Github: https://github.com/audeering/audb

This way, your installation always stays up-to-date,
This way,
your installation always stays up-to-date,
even if you pull new changes from the Github repository.

.. _PyPI: https://pypi.org/project/audb/
.. _Github: https://github.com/audeering/audb/


Coding Convention
-----------------

We follow the PEP8_ convention for Python code
and check for correct syntax with flake8_.
Exceptions are defined under the ``[flake8]`` section
in :file:`setup.cfg`.
and check for correct syntax with ruff_.
In addition,
we check for common spelling errors with codespell_.
Both tools and possible exceptions
are defined in :file:`pyproject.toml`.

The checks are executed in the CI using `pre-commit`_.
You can enable those checks locally by executing::
Expand All @@ -45,22 +54,26 @@ You can enable those checks locally by executing::
pre-commit install
pre-commit run --all-files

Afterwards flake8_ is executed
Afterwards ruff_ and codespell_ are executed
every time you create a commit.

You can also install flake8_
You can also install ruff_ and codespell_
and call it directly::

pip install flake8 # consider system wide installation
flake8
pip install ruff codespell # consider system wide installation
ruff check .
codespell

It can be restricted to specific folders::

flake8 audfoo/ tests/
ruff check audfoo/ tests/
codespell audfoo/ tests/


.. _codespell: https://github.com/codespell-project/codespell/
.. _PEP8: http://www.python.org/dev/peps/pep-0008/
.. _flake8: https://flake8.pycqa.org/en/latest/index.html
.. _pre-commit: https://pre-commit.com
.. _ruff: https://beta.ruff.rs


Building the Documentation
Expand All @@ -70,21 +83,20 @@ If you make changes to the documentation,
you can re-create the HTML pages using Sphinx_.
You can install it and a few other necessary packages with::

pip install -r requirements.txt
pip install -r docs/requirements.txt
pip install -r docs/requirements.txt

To create the HTML pages, use::

python -m sphinx docs/ build/sphinx/html -b html
python -m sphinx docs/ build/sphinx/html -b html

The generated files will be available
in the directory :file:`build/sphinx/html/`.

It is also possible to automatically check if all links are still valid::

python -m sphinx docs/ build/sphinx/linkcheck -b linkcheck
python -m sphinx docs/ build/sphinx/html -b linkcheck

.. _Sphinx: http://sphinx-doc.org/
.. _Sphinx: http://sphinx-doc.org


Running the Tests
Expand All @@ -93,13 +105,13 @@ Running the Tests
You'll need pytest_ for that.
It can be installed with::

pip install -r tests/requirements.txt
pip install -r tests/requirements.txt

To execute the tests, simply run::

python -m pytest
python -m pytest

.. _pytest: https://pytest.org/
.. _pytest: https://pytest.org


Creating a New Release
Expand Down
2 changes: 1 addition & 1 deletion README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,7 @@ If you want to cite **audb**, you can refer to our paper_:
:target: https://audeering.github.io/audb/
:alt: audb's documentation
.. |license| image:: https://img.shields.io/badge/license-MIT-green.svg
:target: https://github.com/audeering/audb/blob/master/LICENSE
:target: https://github.com/audeering/audb/blob/main/LICENSE
:alt: audb's MIT license
.. |python-versions| image:: https://img.shields.io/pypi/pyversions/audb.svg
:target: https://pypi.org/project/audb/
Expand Down
30 changes: 13 additions & 17 deletions audb/__init__.py
Original file line number Diff line number Diff line change
@@ -1,25 +1,21 @@
from audb import info
from audb.core.api import (
available,
cached,
dependencies,
exists,
flavor_path,
latest_version,
remove_media,
repository,
versions,
)
from audb.core.api import available
from audb.core.api import cached
from audb.core.api import dependencies
from audb.core.api import exists
from audb.core.api import flavor_path
from audb.core.api import latest_version
from audb.core.api import remove_media
from audb.core.api import repository
from audb.core.api import versions
from audb.core.cache import default_cache_root
from audb.core.config import config
from audb.core.dependencies import Dependencies
from audb.core.flavor import Flavor
from audb.core.load import (
load,
load_attachment,
load_media,
load_table,
)
from audb.core.load import load
from audb.core.load import load_attachment
from audb.core.load import load_media
from audb.core.load import load_table
from audb.core.load_to import load_to
from audb.core.publish import publish
from audb.core.repository import Repository
Expand Down
12 changes: 4 additions & 8 deletions audb/core/api.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,19 +9,15 @@
import audformat

from audb.core import define
from audb.core.cache import (
database_cache_root,
default_cache_root,
)
from audb.core.cache import database_cache_root
from audb.core.cache import default_cache_root
from audb.core.config import config
from audb.core.dependencies import Dependencies
from audb.core.flavor import Flavor
from audb.core.lock import FolderLock
from audb.core.repository import Repository
from audb.core.utils import (
lookup_backend,
_lookup,
)
from audb.core.utils import _lookup
from audb.core.utils import lookup_backend


def available(
Expand Down
7 changes: 2 additions & 5 deletions audb/core/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,8 @@

import audeer

from audb.core.define import (
CONFIG_FILE,
USER_CONFIG_FILE,
)
from audb.core.define import CONFIG_FILE
from audb.core.define import USER_CONFIG_FILE
from audb.core.repository import Repository


Expand Down Expand Up @@ -79,7 +77,6 @@ class config:
can be overwritten by environment variables.

Examples:

>>> config.CACHE_ROOT
'~/audb'
>>> config.CACHE_ROOT = '~/caches/audb'
Expand Down
2 changes: 1 addition & 1 deletion audb/core/conftest.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@

@pytest.fixture(scope='package', autouse=True)
def cache(tmpdir_factory):
r"""Provide a reuseable cache for docstring tests.
r"""Provide a reusable cache for docstring tests.

As we rely on emodb from the public repo,
it makes sense to cache it
Expand Down
3 changes: 2 additions & 1 deletion audb/core/dependencies.py
Original file line number Diff line number Diff line change
Expand Up @@ -94,9 +94,10 @@ def __getitem__(self, file: str) -> typing.List:
return list(self._df.loc[file])

def __len__(self) -> int:
r"""Number of all media, table, attachment files."""
return len(self._df)

def __str__(self) -> str:
def __str__(self) -> str: # noqa: D105
return self._df.to_string()

@property
Expand Down
1 change: 0 additions & 1 deletion audb/core/flavor.py
Original file line number Diff line number Diff line change
Expand Up @@ -215,7 +215,6 @@ def _resample(
sampling_rate: int,
) -> (np.ndarray, int):
r"""Resample signal to flavor."""

if (self.sampling_rate is not None) and \
(sampling_rate != self.sampling_rate):
signal = audresample.resample(
Expand Down
8 changes: 3 additions & 5 deletions audb/core/info.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,11 +6,9 @@

from audb.core import define
from audb.core.api import dependencies
from audb.core.load import (
filtered_dependencies,
load_header,
load_table,
)
from audb.core.load import filtered_dependencies
from audb.core.load import load_header
from audb.core.load import load_table


def attachments(
Expand Down
27 changes: 9 additions & 18 deletions audb/core/load.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,21 +11,15 @@

from audb.core import define
from audb.core import utils
from audb.core.api import (
cached,
dependencies,
latest_version,
)
from audb.core.cache import (
database_cache_root,
database_tmp_root,
default_cache_root,
)
from audb.core.dependencies import (
Dependencies,
error_message_missing_object,
filter_deps,
)
from audb.core.api import cached
from audb.core.api import dependencies
from audb.core.api import latest_version
from audb.core.cache import database_cache_root
from audb.core.cache import database_tmp_root
from audb.core.cache import default_cache_root
from audb.core.dependencies import Dependencies
from audb.core.dependencies import error_message_missing_object
from audb.core.dependencies import filter_deps
from audb.core.flavor import Flavor
from audb.core.lock import FolderLock
from audb.core.utils import lookup_backend
Expand All @@ -43,7 +37,6 @@ def _cached_versions(
cache_root: typing.Optional[str],
) -> CachedVersions:
r"""Find other cached versions of same flavor."""

df = cached(cache_root=cache_root, name=name)
# If no explicit cache root is given,
# we look into the private and shared one.
Expand Down Expand Up @@ -83,7 +76,6 @@ def _cached_files(
verbose: bool,
) -> (typing.Sequence[typing.Union[str, str]], typing.Sequence[str]):
r"""Find cached files."""

cached_files = []
missing_files = []

Expand Down Expand Up @@ -422,7 +414,6 @@ def _get_media_from_backend(
verbose: bool,
):
r"""Load media from backend."""

# figure out archives
archives = set()
archive_names = set()
Expand Down
Loading