Prepare yadg-6.0 (#201)

* Docs changes part 1 * More docs changes. * Fix tests for dataschema-6.0 * bump schemas to >=121 * pydantic autodocs to _6_0 * Another docs pass.
dgbowl · Nov 14, 2024 · be95156 · be95156
1 parent 5721801
commit be95156
Show file tree

Hide file tree

Showing 48 changed files with 142 additions and 116 deletions.
diff --git a/docs/source/citing.rst b/docs/source/citing.rst
@@ -1,6 +1,6 @@
 Citing **yadg**
 ---------------
-**yadg** is currently being developed at the `ConCat Lab <https://tu.berlin/en/concat>`_ at Technische Universität Berlin, Berlin, Germany. It has been previously developed at `Materials for Energy Conversion lab <https://www.empa.ch/web/s501>`_ at Empa, Dübendorf, Switzerland. If you find **yadg** useful, please cite our recent paper [Kraus2022b]_, or our previous work where **yadg** was first introduced [Kraus2022a]_.
+**yadg** is currently being developed at the `ConCat Lab <https://tu.berlin/en/concat>`_ at Technische Universität Berlin, Berlin, Germany. It has been previously developed at `Materials for Energy Conversion lab <https://www.empa.ch/web/s501>`_ at Empa, Dübendorf, Switzerland. If you find **yadg** useful, please cite our recent paper [Kraus2022b]_. You may also want to refer to our previous work where **yadg** was first introduced [Kraus2022a]_.
 
 We extend no warranty, neither express nor implied.
 
@@ -15,6 +15,9 @@ Papers related to yadg
 ``````````````````````
 If you found **yadg** useful, and would like your publication listed, let us know!
 
+.. [#]
+  Evans, M. L.; Rignanese, G.-M.; Elbert, D.; Kraus. P; *Datatractor: Metadata, automation, and registries for extractor interoperability in the chemical and materials sciences*, arXiv:2410.18839 [physics.data-an] **2024**, DOI: https://doi.org/10.48550/arXiv.2410.18839
+
 .. [#]
   Stüble, P.; Müller, C.; Bohn, N.; Müller, M.; Hofmann, A.; Akçay, T.; Klemens, J.; Koeppe, A.; Kolli, S.; Rajagopal, D.; Geßwein, H.; Schabel, W.; Scharfer, P.; Selzer, M.; Binder, J. R.; Smith, A.; *From Powder to Pouch Cell: Setting up a Sodium-Ion Battery Reference System Based on Na3V2(PO4)3/C and Hard Carbon*, Batteries & Supercaps **2024**, e202400406, DOI: https://doi.org/10.1002/batt.202400406
 

diff --git a/docs/source/index.rst b/docs/source/index.rst
@@ -17,6 +17,7 @@ Contributors
 ````````````
 - `Peter Kraus <https://github.com/PeterKraus>`_
 - `Nicolas Vetsch <https://github.com/vetschn>`_
+- `Carla Terboven <https://github.com/carla-terboven>`_
 
 Acknowledgements
 ````````````````
@@ -33,6 +34,7 @@ The project is also part of BATTERY 2030+, the large-scale European research ini
    :caption: yadg user manual
    :hidden:
 
+   install
    usage
    features
    citing

diff --git a/docs/source/install.rst b/docs/source/install.rst
@@ -0,0 +1,24 @@
+.. _installation:
+
+How to install **yadg**
+=======================
+**yadg** is available on the `Python Package Index <https://pypi.org/project/yadg/>`_, which means the latest stable version can be installed using ``pip``:
+
+.. code-block:: bash
+
+    pip install yadg
+
+To install the latest development version of **yadg**, which is available on `the project GitHub <https://github.org/dgbowl/yadg>`, you can use the ``git+https://`` installation path:
+
+.. code-block:: bash
+
+    pip install git+https://github.org/dgbowl/yadg.git
+
+Finally, **yadg** supports editable installations. If you want to access the test suite, the easiest way to install **yadg** would be to clone the GitHub repository, install **yadg** in editable mode with ``[testing]`` optional dependencies, and run the test suite via ``pytest``:
+
+.. code-block:: bash
+
+    git clone https://github.org/dgbowl/yadg.git
+    cd yadg
+    pip install -e .[testing]
+    pytest -vvx
diff --git a/docs/source/object.dataschema.rst b/docs/source/object.dataschema.rst
@@ -5,6 +5,7 @@ A `dataschema` is an object defining the files and folders to be processed by **
 
 The current version of the `dataschema` is implemented as a Pydantic model in the class :obj:`~dgbowl_schemas.yadg.dataschema.DataSchema` of the :mod:`dgbowl_schemas.yadg` module. The following (previous) versions of the `dataschema` are available in the same repository:
 
+- :class:`dgbowl_schemas.yadg.dataschema_6_0.DataSchema`
 - :class:`dgbowl_schemas.yadg.dataschema_5_1.DataSchema`
 - :class:`dgbowl_schemas.yadg.dataschema_5_0.DataSchema`
 - :class:`dgbowl_schemas.yadg.dataschema_4_2.DataSchema`

diff --git a/docs/source/version.5_2.rst b/docs/source/version.5_2.rst
diff --git a/docs/source/version.6_0.rst b/docs/source/version.6_0.rst
@@ -0,0 +1,34 @@
+**yadg** version 6.0
+``````````````````````
+.. warning::
+
+   This version of yadg is not yet released.
+
+.. image:: https://img.shields.io/static/v1?label=yadg&message=v6.0&color=blue&logo=github
+  :target: https://github.com/PeterKraus/yadg/tree/6.0
+.. image:: https://img.shields.io/static/v1?label=yadg&message=v6.0&color=blue&logo=pypi
+  :target: https://pypi.org/project/yadg/6.0/
+.. image:: https://img.shields.io/static/v1?label=release%20date&message=2024-11-14&color=red&logo=pypi
+
+
+Developed in the `ConCat Lab <https://tu.berlin/en/concat>`_ at Technische Universität Berlin (Berlin, DE).
+
+New features in ``yadg-6.0`` are:
+
+  - Implemented support for merged/appended files in :mod:`yadg.extractors.eclab.mpr`. Test files taken from the data available at https://zenodo.org/doi/10.5281/zenodo.12165685. Thanks to Arnd Koeppe from KIT for bringing the issue up.
+  - Implemented support for the Modular Potentio technique in :mod:`yadg.extractors.eclab.mpr`. Thanks to Graham Kimbell and Clea Burgel from Empa for providing test data.
+  - Implemented support for the Constant Current and Constant Voltage techniques in :mod:`yadg.extractors.eclab.mpr`. Thanks to Carla Terboven from HZB for providing test data.
+
+Breaking changes in ``yadg-6.0`` are:
+
+  - As :class:`~xarray.DataTree` is now merged into the :mod:`xarray` module, ``yadg-6.0`` no longer depends on :mod:`xarray-datatree`. For data stored |NetCDF| files, this shouldn't be a problem, as they can be read using :func:`xarray.open_datatree`. However, if you stored a :class:`datatree.DataTree` as a ``pickle``, and attempt to load it using :class:`xarray.DataTree`, this may not work!
+  - The parameter specification within the ``original_metadata`` entries in :mod:`~yadg.extractors.eclab.mpr` should now be consistent with :mod:`~yadg.extractors.eclab.mpt`. This means renaming of most parameters in :mod:`~yadg.extractors.eclab.mpr` to match :mod:`~yadg.extractors.eclab.mpt`, and a much larger parameter map to map from the :class:`int` values stored in :mod:`~yadg.extractors.eclab.mpr` files to their :class:`str` counterparts. Thanks to Carla Terboven from HZB for the contribution!
+  - Support for ``python <= 3.9`` has been dropped. This is a consequence of ``yadg-6.0`` requiring ``xarray >= 2024.10.0``. However, support for ``python == 3.13`` is now included.
+
+Bug fixes in ``yadg-6.0`` include:
+
+  - Increased supported I-range values in :mod:`yadg.extractors.eclab` up to 193.
+  - Implemented support for a 33-parameter GCPL file in :mod:`yadg.extractors.eclab.mpr`.
+  - Fixed parsing of optional metadata in :mod:`yadg.extractors.fusion.json`. Thanks to Emiliano Dal Molin for finding the bug and providing test data.
+
+.. |NetCDF| replace:: ``NetCDF``
diff --git a/docs/source/version.rst b/docs/source/version.rst
@@ -1,7 +1,7 @@
 **yadg** version history
 ------------------------
 
-.. include:: version.5_2.rst
+.. include:: version.6_0.rst
 
 .. include:: version.5_1.rst
 

diff --git a/pyproject.toml b/pyproject.toml
@@ -40,7 +40,7 @@ dependencies = [
     "pandas >= 2.0",
     "babel >= 2.15",
     "xarray >= 2024.10.0",
-    "dgbowl-schemas >= 120",
+    "dgbowl-schemas >= 121",
 ]
 
 [project.optional-dependencies]

diff --git a/src/yadg/extractors/agilent/ch.py b/src/yadg/extractors/agilent/ch.py
@@ -10,7 +10,7 @@
 `````
 Available since ``yadg-4.0``.
 
-.. autopydantic_model:: dgbowl_schemas.yadg.dataschema_5_1.filetype.Agilent_ch
+.. autopydantic_model:: dgbowl_schemas.yadg.dataschema_6_0.filetype.Agilent_ch
 
 Schema
 ``````

diff --git a/src/yadg/extractors/agilent/csv.py b/src/yadg/extractors/agilent/csv.py
@@ -14,7 +14,7 @@
 `````
 Available since ``yadg-4.0``.
 
-.. autopydantic_model:: dgbowl_schemas.yadg.dataschema_5_1.filetype.Agilent_csv
+.. autopydantic_model:: dgbowl_schemas.yadg.dataschema_6_0.filetype.Agilent_csv
 
 Schema
 ``````

diff --git a/src/yadg/extractors/agilent/dx.py b/src/yadg/extractors/agilent/dx.py
@@ -13,7 +13,7 @@
 `````
 Available since ``yadg-4.0``.
 
-.. autopydantic_model:: dgbowl_schemas.yadg.dataschema_5_1.filetype.Agilent_dx
+.. autopydantic_model:: dgbowl_schemas.yadg.dataschema_6_0.filetype.Agilent_dx
 
 Schema
 ``````

diff --git a/src/yadg/extractors/basic/__init__.py b/src/yadg/extractors/basic/__init__.py
@@ -0,0 +1,4 @@
+"""
+A basic tabulated file extractor.
+
+"""
diff --git a/src/yadg/extractors/basic/csv.py b/src/yadg/extractors/basic/csv.py
@@ -21,7 +21,7 @@
 `````
 Available since ``yadg-4.0``.
 
-.. autopydantic_model:: dgbowl_schemas.yadg.dataschema_5_1.filetype.Basic_csv
+.. autopydantic_model:: dgbowl_schemas.yadg.dataschema_6_0.filetype.Basic_csv
 
 Schema
 ``````

diff --git a/src/yadg/extractors/drycal/csv.py b/src/yadg/extractors/drycal/csv.py
@@ -11,7 +11,7 @@
 `````
 Available since ``yadg-4.0``.
 
-.. autopydantic_model:: dgbowl_schemas.yadg.dataschema_5_1.filetype.Drycal_csv
+.. autopydantic_model:: dgbowl_schemas.yadg.dataschema_6_0.filetype.Drycal_csv
 
 Schema
 ``````

diff --git a/src/yadg/extractors/drycal/rtf.py b/src/yadg/extractors/drycal/rtf.py
@@ -12,7 +12,7 @@
 `````
 Available since ``yadg-4.0``.
 
-.. autopydantic_model:: dgbowl_schemas.yadg.dataschema_5_1.filetype.Drycal_rtf
+.. autopydantic_model:: dgbowl_schemas.yadg.dataschema_6_0.filetype.Drycal_rtf
 
 Schema
 ``````

diff --git a/src/yadg/extractors/drycal/txt.py b/src/yadg/extractors/drycal/txt.py
@@ -11,7 +11,7 @@
 `````
 Available since ``yadg-4.0``.
 
-.. autopydantic_model:: dgbowl_schemas.yadg.dataschema_5_1.filetype.Drycal_txt
+.. autopydantic_model:: dgbowl_schemas.yadg.dataschema_6_0.filetype.Drycal_txt
 
 Schema
 ``````

diff --git a/src/yadg/extractors/eclab/__init__.py b/src/yadg/extractors/eclab/__init__.py
@@ -0,0 +1,4 @@
+"""
+Extractors for data files generated by BioLogic's EC-Lab software.
+
+"""
diff --git a/src/yadg/extractors/eclab/mpr.py b/src/yadg/extractors/eclab/mpr.py
@@ -5,7 +5,7 @@
 `````
 Available since ``yadg-4.0``.
 
-.. autopydantic_model:: dgbowl_schemas.yadg.dataschema_5_1.filetype.EClab_mpr
+.. autopydantic_model:: dgbowl_schemas.yadg.dataschema_6_0.filetype.EClab_mpr
 
 Schema
 ``````
@@ -43,37 +43,8 @@
 Kerr <https://github.com/echemdata/galvani>`_, and builds on the work done
 by the previous civilian service member working on the project, Jonas Krieger.
 
-These are the implemented techniques for which the technique parameter
-sequences can be parsed:
-
-+------+-------------------------------------------------+
-| CA   | Chronoamperometry / Chronocoulometry            |
-+------+-------------------------------------------------+
-| CP   | Chronopotentiometry                             |
-+------+-------------------------------------------------+
-| CV   | Cyclic Voltammetry                              |
-+------+-------------------------------------------------+
-| GCPL | Galvanostatic Cycling with Potential Limitation |
-+------+-------------------------------------------------+
-| GEIS | Galvano Electrochemical Impedance Spectroscopy  |
-+------+-------------------------------------------------+
-| LOOP | Loop                                            |
-+------+-------------------------------------------------+
-| LSV  | Linear Sweep Voltammetry                        |
-+------+-------------------------------------------------+
-| MB   | Modulo Bat                                      |
-+------+-------------------------------------------------+
-| OCV  | Open Circuit Voltage                            |
-+------+-------------------------------------------------+
-| PEIS | Potentio Electrochemical Impedance Spectroscopy |
-+------+-------------------------------------------------+
-| WAIT | Wait                                            |
-+------+-------------------------------------------------+
-| ZIR  | IR compensation (PEIS)                          |
-+------+-------------------------------------------------+
-
 At a top level, ``.mpr`` files are made up of a number of modules,
-separated by the ``MODULE`` keyword. In all the files I have seen, the
+separated by the ``MODULE`` keyword. In all the files we have seen, the
 first module is the settings module, followed by the data module, the
 log module and then an optional loop module.
 
@@ -196,15 +167,6 @@
 include information about the technique, as well as any explicitly parsed cell
 characteristics data specified in EC-Lab.
 
-.. admonition:: TODO
-
-    https://github.com/dgbowl/yadg/issues/12
-
-    The mapping between metadata parameters between ``.mpr`` and ``.mpt`` files
-    is not yet complete. In ``.mpr`` files, some technique parameters in the settings
-    module correspond to entries in drop-down lists in EC-Lab. These values are
-    stored as single-byte values in ``.mpr`` files.
-
 The metadata also contains the infromation from the *Log module*, which contains
 more general parameters, like software, firmware and server versions, channel number,
 host address and an acquisition start timestamp in Microsoft OLE format.
@@ -216,7 +178,8 @@
     not present and therefore the full timestamp cannot be calculated.
 
 .. codeauthor::
-    Nicolas Vetsch
+    Nicolas Vetsch,
+    Peter Kraus,
 
 """
 

diff --git a/src/yadg/extractors/eclab/mpt.py b/src/yadg/extractors/eclab/mpt.py
@@ -5,7 +5,7 @@
 `````
 Available since ``yadg-4.0``.
 
-.. autopydantic_model:: dgbowl_schemas.yadg.dataschema_5_1.filetype.EClab_mpt
+.. autopydantic_model:: dgbowl_schemas.yadg.dataschema_6_0.filetype.EClab_mpt
 
 Schema
 ``````
@@ -35,7 +35,7 @@
 
 Notes on file structure
 ```````````````````````
-These human-readable files are sectioned into headerlines and datalines.
+These human-readable files are sectioned into header lines and data lines.
 The header part of the ``.mpt`` files is made up of information that can be found
 in the settings, log and loop modules of the binary ``.mpr`` file.
 
@@ -46,11 +46,6 @@
 ````````
 The metadata will contain the information from the header of the file.
 
-.. note ::
-
-    The mapping between metadata parameters between ``.mpr`` and ``.mpt`` files
-    is not yet complete.
-
 .. codeauthor::
     Nicolas Vetsch,
     Peter Kraus

diff --git a/src/yadg/extractors/eclab/techniques.py b/src/yadg/extractors/eclab/techniques.py
@@ -19,10 +19,16 @@
     - ZIR - IR compensation (PEIS)
     - MP - Modular Potentio
     - CoV - Constant Voltage
+    - CoC - Constant Current
 
 The module also implements resolution determination for parameters of techniques,
 in :func:`get_resolution`.
 
+.. codeauthor::
+    Nicolas Vetsch,
+    Peter Kraus,
+    Carla Terboven
+
 """
 
 import numpy as np

diff --git a/src/yadg/extractors/empalc/csv.py b/src/yadg/extractors/empalc/csv.py
@@ -10,7 +10,7 @@
 `````
 Available since ``yadg-4.2``.
 
-.. autopydantic_model:: dgbowl_schemas.yadg.dataschema_5_1.filetype.EmpaLC_csv
+.. autopydantic_model:: dgbowl_schemas.yadg.dataschema_6_0.filetype.EmpaLC_csv
 
 Schema
 ``````

diff --git a/src/yadg/extractors/empalc/xlsx.py b/src/yadg/extractors/empalc/xlsx.py
@@ -10,7 +10,7 @@
 `````
 Available since ``yadg-4.2``.
 
-.. autopydantic_model:: dgbowl_schemas.yadg.dataschema_5_1.filetype.EmpaLC_xlsx
+.. autopydantic_model:: dgbowl_schemas.yadg.dataschema_6_0.filetype.EmpaLC_xlsx
 
 Schema
 ``````

diff --git a/src/yadg/extractors/example/__init__.py b/src/yadg/extractors/example/__init__.py
@@ -6,7 +6,7 @@
 `````
 Available since ``yadg-4.0``.
 
-.. autopydantic_model:: dgbowl_schemas.yadg.dataschema_5_1.filetype.Example
+.. autopydantic_model:: dgbowl_schemas.yadg.dataschema_6_0.filetype.Example
 
 
 Schema

diff --git a/src/yadg/extractors/ezchrom/asc.py b/src/yadg/extractors/ezchrom/asc.py
@@ -9,7 +9,7 @@
 `````
 Available since ``yadg-4.0``.
 
-.. autopydantic_model:: dgbowl_schemas.yadg.dataschema_5_1.filetype.EZChrom_asc
+.. autopydantic_model:: dgbowl_schemas.yadg.dataschema_6_0.filetype.EZChrom_asc
 
 Schema
 ``````

diff --git a/src/yadg/extractors/ezchrom/dat.py b/src/yadg/extractors/ezchrom/dat.py
@@ -5,7 +5,7 @@
 `````
 Available since ``yadg-5.1``.
 
-.. autopydantic_model:: dgbowl_schemas.yadg.dataschema_5_1.filetype.EZChrom_dat
+.. autopydantic_model:: dgbowl_schemas.yadg.dataschema_6_0.filetype.EZChrom_dat
 
 Schema
 ``````

diff --git a/src/yadg/extractors/fhimcpt/csv.py b/src/yadg/extractors/fhimcpt/csv.py
@@ -7,7 +7,7 @@
 `````
 Available since ``yadg-3.0``. Deprecated since ``yadg-4.0``.
 
-.. autopydantic_model:: dgbowl_schemas.yadg.dataschema_5_1.filetype.FHI_csv
+.. autopydantic_model:: dgbowl_schemas.yadg.dataschema_6_0.filetype.FHI_csv
 
 Schema
 ``````