diff --git a/doc/source/user_guide.rst b/doc/source/user_guide.rst index c455a661..43edea91 100644 --- a/doc/source/user_guide.rst +++ b/doc/source/user_guide.rst @@ -47,4 +47,5 @@ the RegionProcessor and validated using DataStructureDefinition. user_guide/variable user_guide/region user_guide/model-mapping + user_guide/howto user_guide/local-usage diff --git a/doc/source/user_guide/local-usage.rst b/doc/source/user_guide/local-usage.rst index 0874063b..775d6bc6 100644 --- a/doc/source/user_guide/local-usage.rst +++ b/doc/source/user_guide/local-usage.rst @@ -8,11 +8,11 @@ Local usage of a project .. attention:: This page is intended for users who are familiar with Python, `git `_ and a service like `GitHub `_. -You can use the **nomenclature** package locally (on your machine) for validation -and region-aggregration. This can be helpful as part of processing your model results, -or to ensure that a submission to an IIASA Scenario Explorer instance will succeed. +You can use the **nomenclature** package locally (on your machine) for validation and +region-aggregration. This can be helpful as part of processing your model results, or to +ensure that a submission to an IIASA Scenario Explorer instance will succeed. -The following **requirements** must be satisfied to use a :class:´nomenclature`-based +The following **requirements** must be satisfied to use a :class:`nomenclature`-based project locally: 1. Install the **nomenclature** package (see the :ref:`installation` instructions) @@ -28,13 +28,13 @@ project locally: Validation against codelists ---------------------------- -The easiest use case is to validate that a data file or an -:class:`IamDataFrame ` is compatible with the codelists -(lists of variables and regions) of a project's :class:`DataStructureDefinition`. +The easiest use case is to validate that a data file or an :class:`IamDataFrame +` is compatible with the codelists (lists of variables and regions) +of a project's :class:`DataStructureDefinition`. -If there are inconsistencies with the codelists, the method -:meth:`validate ` will raise an error. -If the scenario data is consistent, the method returns *True*. +If there are inconsistencies with the codelists, the method :meth:`validate +` will raise an error. If the scenario data is +consistent, the method returns *True*. .. code-block:: python diff --git a/doc/source/user_guide/model-registration.rst b/doc/source/user_guide/model-registration.rst new file mode 100644 index 00000000..f7125cbc --- /dev/null +++ b/doc/source/user_guide/model-registration.rst @@ -0,0 +1,85 @@ +.. _model-registration: + +.. currentmodule:: nomenclature + +Model registration +================== + +This guide presents a quick instruction for "registering a model" in a project workflow. + +It is still recommended to read the detailed instructions under :ref:`model_mapping` and +:ref:`region` in particular. + +Region processing +----------------- + +"Registering a model" for a **nomenclature**-based project workflow requires two +specifications: + +* a model mapping to perform region aggregation from *native_regions* to + *common_regions* and renaming of model native regions (optional) +* a list of region names as they should appear in the processed scenario data + +Model mapping +^^^^^^^^^^^^^ + +For this guide we will consider a model mapping as an example: + +.. code-block:: yaml + + model: model_a + native_regions: + - region_1: model_a|Region 1 + - region_2: model_a|Region 2 + - region_3: model_a|Region 3 + common_regions: + - World: + - region_1 + - region_2 + - region_3 + - Common region 1: + - region_1 + - region_2 + +``model_a`` reports three regions natively, ``region_1``, ``region_2`` and ``region_3``. +These three are to be renamed to the ``{model}|{region}`` pattern as is common place in +many projects. For the ``common_regions``, there are ``World`` and ``Common region 1``. + +Region definitions +^^^^^^^^^^^^^^^^^^ + +In order to constitute a valid "model registration", regions ``model_a|Region 1``, +``model_a|Region 2``, ``model_a|Region 3``, ``World`` and ``Common region 1`` **must** +be part of the region definitions. + +``region_1``, ``region_2`` and ``region_3`` are **not required** since they refer to the +input names of ``model_a``'s regions and will be renamed in the processing. + +In most cases, the common regions (in the above example ``World`` and ``Common region +1``) will already be defined in a file called ``definitions/region/regions.yaml``. + +The model native regions, however, most likely need to be added. For this, a new yaml +file should be created, usually in ``definitions/region/model_native_regions/``. The +file does not need to have any special name but it is recommended to use the model name. + +In order to complete the model registration for the example above, we would therefore +add a file called ``model_a.yaml`` to ``definitions/region/model_native_regions/`` with +the following content: + +.. code-block:: yaml + + model_a: + - model_a|Region 1 + - model_a|Region 2 + - model_a|Region 3 + +This combination of a model mapping and the definition of all regions that are part of +the processing output constitutes a complete model registration. + +Continuous Integration +---------------------- + +In most cases, a model registration is submitted as a pull request to a project repository hosted on GitHub. As part +of this, :func:`assert_valid_structure` (details can be found here: :ref:`cli`) is run +automatically to ensure that the model registration is valid. Any regions that are, for +example mentioned in a mapping but not defined will raise an error. diff --git a/nomenclature/cli.py b/nomenclature/cli.py index 6a73ba60..b0162015 100644 --- a/nomenclature/cli.py +++ b/nomenclature/cli.py @@ -9,13 +9,24 @@ @cli.command("validate-yaml") @click.argument("path", type=click.Path(exists=True, path_type=Path)) def cli_valid_yaml(path: Path): - """Assert that all yaml files in `path` can be parsed without errors""" + """Assert that all yaml files in `path` are syntactically valid.""" assert_valid_yaml(path) @cli.command("validate-project") @click.argument("path", type=click.Path(exists=True, path_type=Path)) def cli_valid_project(path: Path): - """Assert that `path` is a valid project nomenclature""" + """Assert that `path` is a valid project nomenclature + + This test includes three steps: + + 1. Test that all yaml files in `definitions/` and `mappings/` can be correctly read + as yaml files. This is a formal check for yaml syntax only. + 2. Test that all files in `definitions/` can be correctly parsed as a + :class:`DataStructureDefinition` object comprised of individual codelists. + 3. Test that all model mappings in `mappings/` can be correctly parsed as a + :class:`RegionProcessor` object. This includes a check that all regions mentioned + in a model mapping are defined in the region codelist. + """ assert_valid_yaml(path) assert_valid_structure(path)