Version 0.2.0

.readthedocs.yaml

@@ -0,0 +1,25 @@
+# .readthedocs.yaml
+# Read the Docs configuration file
+# See https://docs.readthedocs.io/en/stable/config-file/v2.html for details
+
+# Required
+version: 2
+
+# Set the version of Python and other tools you might need
+build:
+  os: ubuntu-20.04
+  tools:
+    python: "3.8"
+
+# Build documentation in the docs/ directory with Sphinx
+sphinx:
+   configuration: docs/conf.py
+
+# If using Sphinx, optionally build your docs in additional formats such as PDF
+# formats:
+#    - pdf
+python:
+   install:
+    - method: pip
+      path: .
+    - requirements: docs/requirements.txt

CHANGELOG.md

@@ -0,0 +1,46 @@
+# Change log
+
+All notable changes to this project will be documented in this file.
+
+## [0.2.0] (2022-08-30)
+### Added
+- CHANGELOG file.
+- First version of the API documentation.
+- Supports slicing and list indexing for the three datasets.
+- Competence values for MACS annotators.
+- Fields scene_label and identifier from TAU Urban acoustic scene dataset in MACS.
+- Add `examples/dataloader.ipynb` notebook.
+
+### Changed
+- Update README with PyPI install and software citation.
+- Download functions returns the datasets downloaded.
+- MACS now have a subset parameter.
+- Underscores in functions names to avoid import private functions.
+- Function `aac_datasets.check.check_directory` now returns only the list of subsets loaded.
+- Replace function `torchaudio.datasets.utils.download_url` by `torch.hub.download_url_to_file` to keep compatibility with future torchaudio version v0.12.
+- Rename `get_raw` methods in datasets by `get_field` and add support for slicing and multi-indexing.
+
+### Fixed
+- LICENCE.txt and MACS_competence.yaml download for MACS dataset.
+- Clotho download archives files.
+
+### Removed
+- Transforms dictionary in datasets.
+- Argument item_type in datasets.
+- Method `get` in datasets.
+
+## [0.1.1] 2022-06-10
+### Added
+- CITATION file.
+
+### Changed
+- MACS now downloads only the required TAU Urban Sound archive files.
+- Documentation for arguments in dataset constructors.
+
+### Fixed
+- Clotho analysis subset download and preparation.
+
+## [0.1.0] 2022-06-07
+### Added
+- Initial versions of Clotho, AudioCaps and MACS pytorch dataset code.
+- Download and check scripts.

CITATION.cff

@@ -1,7 +1,7 @@
 # -*- coding: utf-8 -*-
 
 cff-version: 1.2.0
-title: aac_datasets
+title: aac-datasets
 message: 'If you use this software, please cite it as below.'
 type: software
 authors:
@@ -12,10 +12,11 @@ authors:
 repository-code: 'https://github.com/Labbeti/aac_datasets/'
 abstract: Automated Audio Captioning datasets for Pytorch.
 keywords:
-  - pytorch
-  - datasets
   - audio
+  - pytorch
   - caption
+  - datasets
+  - captioning
 license: MIT
-version: 0.1.1
-date-released: '2022-06-10'
+version: 0.2.0
+date-released: '2022-08-30'

README.md

@@ -14,14 +14,8 @@ Automated Audio Captioning Unofficial datasets source code for **AudioCaps** [1]
 </div>
 
 ## Installation
-
-```bash
-pip install git+https://github.com/Labbeti/aac_datasets
-```
-or clone the repository :
 ```bash
-git clone https://github.com/Labbeti/aac_datasets
-pip install -e aac_datasets
+pip install aac-datasets
 ```
 
 ## Examples
@@ -32,7 +26,8 @@ pip install -e aac_datasets
 from aac_datasets import Clotho
 
 dataset = Clotho(root=".", subset="dev", download=True)
-audio, captions, *_ = dataset[0]
+item = dataset[0]
+audio, captions = item["audio"], item["captions"]
 # audio: Tensor of shape (n_channels=1, audio_max_size)
 # captions: list of str captions
 ```
@@ -41,15 +36,15 @@ audio, captions, *_ = dataset[0]
 
 ```python
 from torch.utils.data.dataloader import DataLoader
-from aac_datasets import MACS
+from aac_datasets import Clotho
 from aac_datasets.utils import BasicCollate
 
-dataset = MACS(root=".", download=True)
+dataset = Clotho(root=".", download=True)
 dataloader = DataLoader(dataset, batch_size=4, collate_fn=BasicCollate())
 
 for audio_batch, captions_batch in dataloader:
-    # audio_batch: Tensor of shape (batch_size=4, n_channels=2, audio_max_size)
-    # captions_batch: list of list of str captions
+    # audio_batch: Tensor of shape (batch_size, n_channels, audio_max_size)
+    # captions_batch: list of list of str
     ...
 ```
 
@@ -73,7 +68,7 @@ Here is the **train** subset statistics for each dataset :
 | Nb captions per audio | 1 | 5 | 2-5 |
 | Nb captions | 49838 | 19195 | 17275 |
 | Total nb words<sup>2</sup> | 402482 | 217362 | 160006 |
-| Nb words range<sup>2</sup> | 1-52 | 8-20 | 5-40 |
+| Nb words range<sup>2</sup> | 2-52 | 8-20 | 5-40 |
 
 <sup>1</sup> This duration is estimated on the total duration of 46230/49838 files of 126.7h.
 
@@ -101,7 +96,7 @@ You can also override their paths for AudioCaps:
 from aac_datasets import AudioCaps
 AudioCaps.FFMPEG_PATH = "/my/path/to/ffmpeg"
 AudioCaps.YOUTUBE_DL_PATH = "/my/path/to/youtube_dl"
-_ = AudioCaps(root=".", download=True)
+dataset = AudioCaps(root=".", download=True)
 ```
 
 ## Command line download
@@ -118,3 +113,19 @@ python -m aac_datasets.download --root "./data" clotho --version "v2.1"
 [2] K. Drossos, S. Lipping, and T. Virtanen, “Clotho: An Audio Captioning Dataset,” arXiv:1910.09387 [cs, eess], Oct. 2019, Available: http://arxiv.org/abs/1910.09387
 
 [3] F. Font, A. Mesaros, D. P. W. Ellis, E. Fonseca, M. Fuentes, and B. Elizalde, Proceedings of the 6th Workshop on Detection and Classication of Acoustic Scenes and Events (DCASE 2021). Barcelona, Spain: Music Technology Group - Universitat Pompeu Fabra, Nov. 2021. Available: https://doi.org/10.5281/zenodo.5770113
+
+## Cite the aac-datasets package
+If you use this software, please consider cite it as below :
+
+```
+@software{
+    Labbe_aac-datasets_2022,
+    author = {Labbé, Etienne},
+    license = {MIT},
+    month = {6},
+    title = {{aac-datasets}},
+    url = {https://github.com/Labbeti/aac_datasets/},
+    version = {0.1.1},
+    year = {2022}
+}
+```

docs/Makefile

@@ -0,0 +1,20 @@
+# Minimal makefile for Sphinx documentation
+#
+
+# You can set these variables from the command line, and also
+# from the environment for the first two.
+SPHINXOPTS    ?=
+SPHINXBUILD   ?= sphinx-build
+SOURCEDIR     = .
+BUILDDIR      = _build
+
+# Put it first so that "make" without argument is like "make help".
+help:
+	@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
+
+.PHONY: help Makefile
+
+# Catch-all target: route all unknown targets to Sphinx using the new
+# "make mode" option.  $(O) is meant as a shortcut for $(SPHINXOPTS).
+%: Makefile
+	@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)

docs/aac_datasets.check.rst

@@ -0,0 +1,7 @@
+aac\_datasets.check module
+==========================
+
+.. automodule:: aac_datasets.check
+   :members:
+   :undoc-members:
+   :show-inheritance:

docs/aac_datasets.datasets.audiocaps.rst

@@ -0,0 +1,7 @@
+aac\_datasets.datasets.audiocaps module
+=======================================
+
+.. automodule:: aac_datasets.datasets.audiocaps
+   :members:
+   :undoc-members:
+   :show-inheritance:

docs/aac_datasets.datasets.clotho.rst

@@ -0,0 +1,7 @@
+aac\_datasets.datasets.clotho module
+====================================
+
+.. automodule:: aac_datasets.datasets.clotho
+   :members:
+   :undoc-members:
+   :show-inheritance:

docs/aac_datasets.datasets.macs.rst

@@ -0,0 +1,7 @@
+aac\_datasets.datasets.macs module
+==================================
+
+.. automodule:: aac_datasets.datasets.macs
+   :members:
+   :undoc-members:
+   :show-inheritance:

docs/aac_datasets.datasets.rst

@@ -0,0 +1,17 @@
+aac\_datasets.datasets package
+==============================
+
+.. automodule:: aac_datasets.datasets
+   :members:
+   :undoc-members:
+   :show-inheritance:
+
+Submodules
+----------
+
+.. toctree::
+   :maxdepth: 4
+
+   aac_datasets.datasets.audiocaps
+   aac_datasets.datasets.clotho
+   aac_datasets.datasets.macs

docs/aac_datasets.download.rst

@@ -0,0 +1,7 @@
+aac\_datasets.download module
+=============================
+
+.. automodule:: aac_datasets.download
+   :members:
+   :undoc-members:
+   :show-inheritance:

docs/aac_datasets.rst

@@ -0,0 +1,26 @@
+aac\_datasets package
+=====================
+
+.. automodule:: aac_datasets
+   :members:
+   :undoc-members:
+   :show-inheritance:
+
+Subpackages
+-----------
+
+.. toctree::
+   :maxdepth: 4
+
+   aac_datasets.datasets
+
+Submodules
+----------
+
+.. toctree::
+   :maxdepth: 4
+
+   aac_datasets.check
+   aac_datasets.download
+   aac_datasets.utils
+   aac_datasets.version

docs/aac_datasets.utils.rst

@@ -0,0 +1,7 @@
+aac\_datasets.utils module
+==========================
+
+.. automodule:: aac_datasets.utils
+   :members:
+   :undoc-members:
+   :show-inheritance:

docs/aac_datasets.version.rst

@@ -0,0 +1,7 @@
+aac\_datasets.version module
+============================
+
+.. automodule:: aac_datasets.version
+   :members:
+   :undoc-members:
+   :show-inheritance:

docs/conf.py

@@ -0,0 +1,89 @@
+# Configuration file for the Sphinx documentation builder.
+#
+# This file only contains a selection of the most common options. For a full
+# list see the documentation:
+# https://www.sphinx-doc.org/en/master/usage/configuration.html
+
+# -- Path setup --------------------------------------------------------------
+
+# If extensions (or modules to document with autodoc) are in another directory,
+# add these directories to sys.path here. If the directory is relative to the
+# documentation root, use os.path.abspath to make it absolute, like shown here.
+#
+import os
+import sys
+
+sys.path.insert(0, os.path.abspath(".."))
+
+import aac_datasets
+
+
+# -- Project information -----------------------------------------------------
+
+project = aac_datasets.__name__
+copyright = "MIT"
+author = aac_datasets.__author__
+
+# The short X.Y version
+version = aac_datasets.__version__
+
+# The full version, including alpha/beta/rc tags
+release = f"{aac_datasets.__status__}-{aac_datasets.__version__}"
+
+
+# -- General configuration ---------------------------------------------------
+
+# Add any Sphinx extension module names here, as strings. They can be
+# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
+# ones.
+extensions = [
+    "sphinx.ext.autodoc",
+    "sphinx.ext.viewcode",
+    "sphinx.ext.todo",
+    "sphinx.ext.autosummary",
+    "sphinx.ext.coverage",
+    "sphinx.ext.intersphinx",
+]
+
+# Add any paths that contain templates here, relative to this directory.
+templates_path = ["_templates"]
+
+# The language for content autogenerated by Sphinx. Refer to documentation
+# for a list of supported languages.
+#
+# This is also used if you do content translation via gettext catalogs.
+# Usually you set "language" from the command line for these cases.
+language = "en"
+
+# List of patterns, relative to source directory, that match files and
+# directories to ignore when looking for source files.
+# This pattern also affects html_static_path and html_extra_path.
+exclude_patterns = ["_build", "Thumbs.db", ".DS_Store"]
+
+
+# -- Options for HTML output -------------------------------------------------
+
+# The theme to use for HTML and HTML Help pages.  See the documentation for
+# a list of builtin themes.
+#
+html_theme = "sphinx_rtd_theme"
+
+# Add any paths that contain custom static files (such as style sheets) here,
+# relative to this directory. They are copied after the builtin static files,
+# so a file named "default.css" will overwrite the builtin "default.css".
+html_static_path = ["_static"]
+
+
+# -- Extension configuration -------------------------------------------------
+
+# -- Options for todo extension ----------------------------------------------
+
+# If true, `todo` and `todoList` produce output, else they produce nothing.
+todo_include_todos = True
+
+add_module_names = False
+
+intersphinx_mapping = {
+    "python": ("https://docs.python.org/", None),
+    "torch": ("https://pytorch.org/docs/master/", None),
+}