Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Docs/add howto #144

Merged
merged 11 commits into from
Aug 10, 2022
1 change: 1 addition & 0 deletions doc/source/user_guide.rst
Original file line number Diff line number Diff line change
Expand Up @@ -47,4 +47,5 @@ the RegionProcessor and validated using DataStructureDefinition.
user_guide/variable
user_guide/region
user_guide/model-mapping
user_guide/howto
phackstock marked this conversation as resolved.
Show resolved Hide resolved
user_guide/local-usage
73 changes: 73 additions & 0 deletions doc/source/user_guide/howto.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
.. _how-to:
phackstock marked this conversation as resolved.
Show resolved Hide resolved

.. currentmodule:: nomenclature

This guide is intended as a quick instruction for adding a model to comparison project.
It is still recommended to read the detailed instructions under :ref:`model_mapping` and
:ref:`region` in particular.

How to add your model mapping to a project repository
=====================================================

There are two parts to adding a new model mapping. The mapping itself and the easily
forgotten additionally required region definitions.
phackstock marked this conversation as resolved.
Show resolved Hide resolved

As outlined in :ref:`model_mapping` a model mapping holds the information about model
native an aggregation regions, for example:
phackstock marked this conversation as resolved.
Show resolved Hide resolved

.. code-block:: yaml

model: model_a
native_regions:
- region_1: model_a|Region 1
- region_2: model_a|Region 2
- region_3: model_a|Region 3
common_regions:
- World:
- region_1
- region_2
- region_3
- Common region 1:
- region_1
- region_2

``model_a`` reports three regions natively, ``region_1``, ``region_2`` and ``region_3``.
These three are to be renamed to the ``{model}|{region}`` pattern as is common place in
many projects. For the ``common_regions``, there are ``World`` and ``Common region 1``.

If this model mapping would be submitted as a pull request to a project repository, the
phackstock marked this conversation as resolved.
Show resolved Hide resolved
:func:`assert_valid_structure` (details can be found here: :ref:`cli`) test that is run
automatically would most likely fail.

The reason for this is that it is required for all regions that will be part of the
processing output according to a model mapping to be defined in the list of regions in
the ``definitions/region/`` folder.

In case of the example mapping above, regions ``model_a|Region 1, model_a|Region 2,
model_a|Region 3, World`` and ``Common region 1`` must be part of the region
definitions. ``region_1, region_2`` and ``region_3`` are not required since they will
not be part of the processing output as they will be renamed.

In most cases the common regions will already be fined. There will, most likely exist a
file ``defintions/region/regions.yaml`` which contains the project's common regions.

For the model native regions, however, the regions will most likely not exist. Therefore
phackstock marked this conversation as resolved.
Show resolved Hide resolved
a new yaml file needs to be added to the same pull request that contains the model
mapping. Per convention the ``definitions/region`` folder usually contains a subfolder
called ``model_native_regions`` which is where all files containing model native region
definitions should be put. The name of the file is not functionally relevant but it is
recommended to use the model name.

In order to make to above example model mapping work and the test run, we would
phackstock marked this conversation as resolved.
Show resolved Hide resolved
therefore add a file called ``model_a.yaml`` to
``definitions/region/model_native_regions`` with the following content:

.. code-block:: yaml

model_a:
- model_a|Region 1
- model_a|Region 2
- model_a|Region 3

assuming ``World`` and ``Common region 1`` are already defined, this should make the
phackstock marked this conversation as resolved.
Show resolved Hide resolved
tests pass.
20 changes: 10 additions & 10 deletions doc/source/user_guide/local-usage.rst
Original file line number Diff line number Diff line change
Expand Up @@ -8,11 +8,11 @@ Local usage of a project
.. attention:: This page is intended for users who are familiar with Python,
`git <https://git-scm.com>`_ and a service like `GitHub <https://github.com>`_.

You can use the **nomenclature** package locally (on your machine) for validation
and region-aggregration. This can be helpful as part of processing your model results,
or to ensure that a submission to an IIASA Scenario Explorer instance will succeed.
You can use the **nomenclature** package locally (on your machine) for validation and
region-aggregration. This can be helpful as part of processing your model results, or to
ensure that a submission to an IIASA Scenario Explorer instance will succeed.

The following **requirements** must be satisfied to use a :class:´nomenclature`-based
The following **requirements** must be satisfied to use a :class:`nomenclature`-based
project locally:

1. Install the **nomenclature** package (see the :ref:`installation` instructions)
Expand All @@ -28,13 +28,13 @@ project locally:
Validation against codelists
----------------------------

The easiest use case is to validate that a data file or an
:class:`IamDataFrame <pyam.IamDataFrame>` is compatible with the codelists
(lists of variables and regions) of a project's :class:`DataStructureDefinition`.
The easiest use case is to validate that a data file or an :class:`IamDataFrame
<pyam.IamDataFrame>` is compatible with the codelists (lists of variables and regions)
of a project's :class:`DataStructureDefinition`.

If there are inconsistencies with the codelists, the method
:meth:`validate <DataStructureDefinition.validate>` will raise an error.
If the scenario data is consistent, the method returns *True*.
If there are inconsistencies with the codelists, the method :meth:`validate
<DataStructureDefinition.validate>` will raise an error. If the scenario data is
consistent, the method returns *True*.

.. code-block:: python
Expand Down
15 changes: 13 additions & 2 deletions nomenclature/cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,13 +9,24 @@
@cli.command("validate-yaml")
@click.argument("path", type=click.Path(exists=True, path_type=Path))
def cli_valid_yaml(path: Path):
"""Assert that all yaml files in `path` can be parsed without errors"""
"""Assert that all yaml files in `path` are syntactically valid."""
assert_valid_yaml(path)


@cli.command("validate-project")
@click.argument("path", type=click.Path(exists=True, path_type=Path))
def cli_valid_project(path: Path):
"""Assert that `path` is a valid project nomenclature"""
"""Assert that `path` is a valid project nomenclature
This test includes three steps:
1. Test that all yaml files in `definitions/` and `mappings/` can be correctly read
as yaml files. This is a formal check for yaml syntax only.
2. Test that all files in `definitions/` can be correctly parsed as a
:class:`DataStructureDefinition` object comprised of individual codelists.
3. Test that all model mappings in `mappings/` can be correctly parsed as a
:class:`RegionProcessor` object. This includes a check that all regions mentioned
in a model mapping are defined in the region codelist.
"""
assert_valid_yaml(path)
assert_valid_structure(path)