Skip to content

Add your own diagnostic to the C ESM EP_v1

Jump to bottom
jservonnat edited this page Aug 1, 2019 · 5 revisions

The C-ESM-EP offers the possibility to add your own personal diagnostic (either written with CliMAF or from a personal script) to the existing battery of diagnostics.

This way you can create your own html atlas that fits your personal needs, taking advantage of the C-ESM-EP framework (looping on datasets, easy pre-treatments, building an html page…).

The minimum you need to do to add your own diagnostic

To build a new atlas (new html page), you need to go through the following steps:

  • In your comparison directory (say, toy_comparison), create a new directory for your atlas (ex: MyOwnDiag in standard_comparison in C-ESM-EP_v2) so that you have a parameter file for your atlas. Name your parameter file after the name used for your atlas (component) : toy_comparison/MyOwnDiag/params_MyOwnDiag.py
  • Develop the code of your diagnostic:
    • in C-ESM-EP_v1: in main_C-ESM-EP.py
    • in C-ESM-EP_v2: in a diagnostic file: follow the example in standard_comparison/MyOwnDiag/diagnostics_MyOwnDiag.py ; eventually your can move your diagnostic file in share/cesmep_diagnostics if you want to use only one version for different comparisons (not having to maintain multiple copies in different comparison directories)

Run it interactively to correct what needs to be corrected

And lastly, enjoy using it routinely!

The role of the atlas directory is two-fold:

  • It contains the parameter file of the atlas (mandatory to run an atlas)
  • It will receive the output files of the execution of your atlas when you submit a job with run_C-ESM-EP.py

We create a new atlas directory like this: In your comparison directory (say, toy_comparison), create a new directory for your atlas; to do this, copy an existing atlas directory (MyOwnDiag for instance) and name it after your own choice: cd standard_comparison cp –r MyOwnDiag my_own_atlas

The name of the parameter file has to follow the name of the directory: ${my_own_atlas}/params_${my_own_atlas}.py

Ex:

mv my_own_atlas/params_MyOwnDiag.py my_own_atlas/params_my_own_atlas.py

C-ESM-EP_v1: Instruction variable to activate/deactivate the execution of the diagnostics in main_C-ESM-EP.py

The first thing to do is to add your own ‘instruction variable’: the variable that will authorize the execution of your code in main_C-ESM-EP.py . Those instruction variables are set to True (execute the code) or False (do nothing).

In the MyOwnDiag parameter file, this instruction is ‘do_atlas_explorer‘. Rename with an explicit name:

do_my_own_diag = True

(You can also leave do_atlas_explorer and set it to True if you want to add climatology maps at the beginning of your atlas. Go to section 2 of the documentation to see how to use Atlas Explorer)

Then, edit the variable ‘atlas_head_title’ to provide a meaningful title to your atlas html page. This character string will also be used in the front page of the C-ESM-EP.

Lastly, you can add all the variables, python dictionaries or CliMAF settings (calias, project definition, derive...) that will be used by your code in main_C-ESM-EP.py and still have them easily accessible in the parameter file.

Develop your diagnostic using the CliMAF functionalities

The simplest and most straightforward way to develop your own diagnostic is to use CliMAF operators (see the CliMAF documentation), and do the plot with the CliMAF plotting operators (‘plot’ and ‘curves’).

You will find an example in main_C-ESM-EP.py at the section controlled by the ‘do_my_own_climaf_diag’ instruction:

  • How to use the CliMAF function to write the html index
  • Copy the ‘models’ list to do a ‘Wmodels’ list that you will work on
  • Loop on variables
  • Loop on models
  • Apply the frequency and period manager
  • Do my own analysis
  • Plot the result
  • And add the plot to the html page

Looping on the models is actually not mandatory but it is one of the interests of the C-ESM-EP

Adding your own script

CliMAF offers an easy way to plug your own script of diagnostic/plot if the existing CliMAF operators/functions do not cover your needs.

You will find an example in main_C-ESM-EP.py in the section controlled by the instruction ‘do_mse_otorres_maps’.

You can plug any script (i.e. make a CliMAF operator of it) with ‘cscript’ if this script verifies the following constraints:

  • You can execute it in one command line taking arguments
  • The name of the input netcdf and the output netcdf file or figure have to be explicitely part of the script arguments (see the cscript documentation for more details: )

The advantages of plugging your script are (notably):

  • using the climaf cache
  • be fully part of the workflow (better in terms of coding
  • Be part of the C-ESM-EP sustainably

You will find a lot of examples in ${CLIMAF}/climaf/standard_operators.py of the use of cscript to plug external scripts or command lines (cdo and nco).

Note that it is possible to call a script in a compound command line, separated with ‘; ‘. This allows for more flexibility.

Although it is not the cleanest way to do it, you can also take advantage of intermediate or auxillary files in the command line that are hard coded. In other words: better to plug a script in a not perfect fashion than not plugging it!

Clone this wiki locally