Building a new parser

A parser is a python3 script that defines the method fetch_production and returns the production mix at current time, in this format:

def fetch_production(zone_key='FR', session=None, target_datetime=None, logger=None):
    return {
      'zoneKey': 'FR',
      'datetime': '2017-01-01T00:00:00Z',
      'production': {
          'biomass': 0.0,
          'coal': 0.0,
          'gas': 0.0,
          'hydro': 0.0,
          'nuclear': None,
          'oil': 0.0,
          'solar': 0.0,
          'wind': 0.0,
          'geothermal': 0.0,
          'unknown': 0.0
      },
      'storage': {
          'hydro': -10.0,
      },
      'source': 'mysource.com'
    }

It contains the following objects:

• session: a python request session that you can re-use to make HTTP requests.
• target_datetime: used to fetch historical data when available.
• logger: a logging.Logger whose output is publicly available for anyone to monitor correct functioning of the parsers.

The production values should never be negative. Use None, or omit the key if a specific production mode is not known. Storage values can be both positive (when storing energy) or negative (when the storage is emptied).

The parser can also return an array of objects if multiple time values can be fetched. The backend will automatically update past values properly.

Once you're done, add your parser to the zones.json and exchanges.json configuration files. Finally update the real-time sources.

Run all of the parser tests with the following command from the root directory:

python -m unittest discover parsers/test/

For more info, check out the example parser or browse existing parsers.

Test parsers locally

1. Make sure you have installed the required modules as described in parsers/requirements.txt, and are using python 3.6 (consider using a virtual environment for python). To confirm this, run:

   pip install -r parsers/requirements.txt
   

2. From the root folder, use the test_parser.py command line utility:

   python test_parser.py FR price  # get latest price parser for France
   python test_parser.py FR  # defaults to production if no data type is given
   # test a specific datetime (parser needs to be able to fetch past datetimes)
   python test_parser.py DE --target_datetime 2018-01-01T08:00

Many of the tests require API keys of the data or web service providers, and therefore fail with an error message like

Exception: No ENTSOE_TOKEN found! Please add it into secrets.env!

In such cases, please browse the website related to the provider and ask for an API key. Once you get hold of the API key, make it an environment variable. This fixes the error.