Skip to content

Commit

Permalink
Docs/add howto (#144)
Browse files Browse the repository at this point in the history
* Add more detailed docstring to cli

* Add how to to user guide

* Fix formatting and reference

* Apply suggestions from code review

Co-authored-by: Daniel Huppmann <dh@dergelbesalon.at>

* Update model registration how-to

* Fix list

* Fix list indentation

* Change formatting of region example

* Shorten user guide to be more concise

* Apply suggestions from code review

Co-authored-by: Daniel Huppmann <dh@dergelbesalon.at>

* Rename howto.rst

Co-authored-by: Daniel Huppmann <dh@dergelbesalon.at>
  • Loading branch information
phackstock and danielhuppmann committed Aug 10, 2022
1 parent f96d0ec commit 4a57d2f
Show file tree
Hide file tree
Showing 4 changed files with 109 additions and 12 deletions.
1 change: 1 addition & 0 deletions doc/source/user_guide.rst
Original file line number Diff line number Diff line change
Expand Up @@ -47,4 +47,5 @@ the RegionProcessor and validated using DataStructureDefinition.
user_guide/variable
user_guide/region
user_guide/model-mapping
user_guide/howto
user_guide/local-usage
20 changes: 10 additions & 10 deletions doc/source/user_guide/local-usage.rst
Original file line number Diff line number Diff line change
Expand Up @@ -8,11 +8,11 @@ Local usage of a project
.. attention:: This page is intended for users who are familiar with Python,
`git <https://git-scm.com>`_ and a service like `GitHub <https://github.com>`_.

You can use the **nomenclature** package locally (on your machine) for validation
and region-aggregration. This can be helpful as part of processing your model results,
or to ensure that a submission to an IIASA Scenario Explorer instance will succeed.
You can use the **nomenclature** package locally (on your machine) for validation and
region-aggregration. This can be helpful as part of processing your model results, or to
ensure that a submission to an IIASA Scenario Explorer instance will succeed.

The following **requirements** must be satisfied to use a :class:´nomenclature`-based
The following **requirements** must be satisfied to use a :class:`nomenclature`-based
project locally:

1. Install the **nomenclature** package (see the :ref:`installation` instructions)
Expand All @@ -28,13 +28,13 @@ project locally:
Validation against codelists
----------------------------

The easiest use case is to validate that a data file or an
:class:`IamDataFrame <pyam.IamDataFrame>` is compatible with the codelists
(lists of variables and regions) of a project's :class:`DataStructureDefinition`.
The easiest use case is to validate that a data file or an :class:`IamDataFrame
<pyam.IamDataFrame>` is compatible with the codelists (lists of variables and regions)
of a project's :class:`DataStructureDefinition`.

If there are inconsistencies with the codelists, the method
:meth:`validate <DataStructureDefinition.validate>` will raise an error.
If the scenario data is consistent, the method returns *True*.
If there are inconsistencies with the codelists, the method :meth:`validate
<DataStructureDefinition.validate>` will raise an error. If the scenario data is
consistent, the method returns *True*.

.. code-block:: python
Expand Down
85 changes: 85 additions & 0 deletions doc/source/user_guide/model-registration.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
.. _model-registration:

.. currentmodule:: nomenclature

Model registration
==================

This guide presents a quick instruction for "registering a model" in a project workflow.

It is still recommended to read the detailed instructions under :ref:`model_mapping` and
:ref:`region` in particular.

Region processing
-----------------

"Registering a model" for a **nomenclature**-based project workflow requires two
specifications:

* a model mapping to perform region aggregation from *native_regions* to
*common_regions* and renaming of model native regions (optional)
* a list of region names as they should appear in the processed scenario data

Model mapping
^^^^^^^^^^^^^

For this guide we will consider a model mapping as an example:

.. code-block:: yaml
model: model_a
native_regions:
- region_1: model_a|Region 1
- region_2: model_a|Region 2
- region_3: model_a|Region 3
common_regions:
- World:
- region_1
- region_2
- region_3
- Common region 1:
- region_1
- region_2
``model_a`` reports three regions natively, ``region_1``, ``region_2`` and ``region_3``.
These three are to be renamed to the ``{model}|{region}`` pattern as is common place in
many projects. For the ``common_regions``, there are ``World`` and ``Common region 1``.

Region definitions
^^^^^^^^^^^^^^^^^^

In order to constitute a valid "model registration", regions ``model_a|Region 1``,
``model_a|Region 2``, ``model_a|Region 3``, ``World`` and ``Common region 1`` **must**
be part of the region definitions.

``region_1``, ``region_2`` and ``region_3`` are **not required** since they refer to the
input names of ``model_a``'s regions and will be renamed in the processing.

In most cases, the common regions (in the above example ``World`` and ``Common region
1``) will already be defined in a file called ``definitions/region/regions.yaml``.

The model native regions, however, most likely need to be added. For this, a new yaml
file should be created, usually in ``definitions/region/model_native_regions/``. The
file does not need to have any special name but it is recommended to use the model name.

In order to complete the model registration for the example above, we would therefore
add a file called ``model_a.yaml`` to ``definitions/region/model_native_regions/`` with
the following content:

.. code-block:: yaml
model_a:
- model_a|Region 1
- model_a|Region 2
- model_a|Region 3
This combination of a model mapping and the definition of all regions that are part of
the processing output constitutes a complete model registration.

Continuous Integration
----------------------

In most cases, a model registration is submitted as a pull request to a project repository hosted on GitHub. As part
of this, :func:`assert_valid_structure` (details can be found here: :ref:`cli`) is run
automatically to ensure that the model registration is valid. Any regions that are, for
example mentioned in a mapping but not defined will raise an error.
15 changes: 13 additions & 2 deletions nomenclature/cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,13 +9,24 @@
@cli.command("validate-yaml")
@click.argument("path", type=click.Path(exists=True, path_type=Path))
def cli_valid_yaml(path: Path):
"""Assert that all yaml files in `path` can be parsed without errors"""
"""Assert that all yaml files in `path` are syntactically valid."""
assert_valid_yaml(path)


@cli.command("validate-project")
@click.argument("path", type=click.Path(exists=True, path_type=Path))
def cli_valid_project(path: Path):
"""Assert that `path` is a valid project nomenclature"""
"""Assert that `path` is a valid project nomenclature
This test includes three steps:
1. Test that all yaml files in `definitions/` and `mappings/` can be correctly read
as yaml files. This is a formal check for yaml syntax only.
2. Test that all files in `definitions/` can be correctly parsed as a
:class:`DataStructureDefinition` object comprised of individual codelists.
3. Test that all model mappings in `mappings/` can be correctly parsed as a
:class:`RegionProcessor` object. This includes a check that all regions mentioned
in a model mapping are defined in the region codelist.
"""
assert_valid_yaml(path)
assert_valid_structure(path)

0 comments on commit 4a57d2f

Please sign in to comment.