Skip to content

Adding new variables to an existing atlas (based on Atlas Explorer)

Jump to bottom
jservonnat edited this page Jan 27, 2020 · 23 revisions

Atlas Explorer allows controlling the variables that you want in your atlas. This is also true for all the atlases based on the Atlas Explorer mechanism.

Everything can be done from the parameter file.

To add a new variable to an existing atlas, you need to:

In the following we use the Atlas Explorer parameter file as an example but it works the same way with the other atlases based on the Atlas Explorer mechanism (Atmosphere_Surface, the section controlled by do_ocean_2D_maps in NEMO_main...)

1. Add your variable to the list

The list of variables of the atlas is the python list that comes with the instruction that controls the execution of the section. In this example, the instruction is do_atlas_explorer and the list is atlas_explorer_variables:

do_atlas_explorer        = True    # -> use atlas_explorer_variables to set your own selection of variables
atlas_explorer_variables = ['tas', 'tos',
                            dict(variable='ua', season='DJF') ]

Let's say that we want to add a new variable called myvar to the atlas. The first thing I need to do is to add myvar to the list:

atlas_explorer_variables = ['tas', 'tos',
                            dict(variable='ua', season='DJF'),
                            'myvar',
                           ]

Specific note for ocean variables (irregular grid): Atlas Explorer has a special treatment with ocean variables on irregular grids (like the ORCA grid). We regrid the field before plotting (also for the climatology plots, the first one from the left), and by default add a grey mask on continents and center the plot on longitude 200°E. If you want to take advantage of this, add this to your parameter file (after the variables list):

from climaf.plot.ocean_plot_params import dict_plot_params as ocean_plot_params
ocean_variables=[]
for oceanvar in ocean_plot_params: ocean_variables.append(oceanvar)
for tmpvar in my_list_of_ocean_2D_variables: ocean_variables.append(tmpvar)

with my_list_of_ocean_2D_variables being the list of new ocean variables (character string, no dictionary).

2. Add an alias for this variable

I now need to ensure access to myvar for the different projects I'm interested in. As an example, let's say that myvar is actually the SST. In my atlas I want to use IGCM_OUT datasets and CMIP5 datasets. We now have different cases:

  • If the variable name 'myvar' is explicitly contained in the path/filename pattern of my target files (for instance, tas, pr, ua are variable names that appear in the path/filename patterns of the CMIP5 and CMIP6 files, and the IGCM_OUT Analyse/TS_MO), I don't need to do anything.
  • If the variable name 'myvar' is not explicitly contained in the path/filename pattern of my target files because the files contain multiple variables (for instance, in the *histmth.nc or *grid_T.nc files in the IGCM_OUT project), I need to add an alias using **filenameVar:
calias('IGCM_OUT', 'myvar', filenameVar='grid_T')
  • If 'myvar' actually points to another variable name in the files (here tos), I also need to add an alias:
calias('IGCM_OUT', 'myvar', 'tos', filenameVar='grid_T')
calias('CMIP5', 'myvar', 'tos')

You can read this calias instruction (the first one) as: for the IGCM_OUT project, if you want to get the myvar variable, get the tos variable in the files; if you didn't find a path/filename containing tos, try to find a file containing grid_T in place of the variable name (check the CliMAF documentation on calias for more details on calias, and/or have a look at $CLIMAF/climaf/projects/igcm_out.py).

3. Specify a reference

The principle of the Atlas Explorer mechanism is do difference maps of all your datasets with a chosen reference. You thus need to specify one.

Use a simulation as reference

The simplest case is if you want to do model-model differences for all the variables of the atlas. You specify another simulation as reference in the parameter file, like this:

reference = dict(project='IGCM_OUT',
                 root='path_to_store_thredds',
                 login='p86caub',
                 model='IPSLCM6',
                 experiment='pdControl',
                 simulation='CM607-LR-pdCtrl-01',
                 clim_period='2000_2009',
                 frequency='monthly',
                 )

In this case, Atlas Explorer will just need to be able to get 'myvar' in the IGCM_OUT project.

Note: if you use your own CliMAF project (that gives access to your own simulations) as reference, be sure that your project makes use of the attribute 'model'. The C-ESM-EP will need it to identify that the reference is a model simulation and not an observational product.

Use an observational product or reanalysis as reference

The C-ESM-EP relies on a (handmade) catalog or default references for each variable. We use a function called variable2reference (defined in $CLIMAF/reference/reference.py) that returns a dictionary with the arguments to access the dataset. You can add your own datasets (or update the existing default) from the parameter file using custom_obs_dict: For example, adding this to your parameter file:

custom_obs_dict = {
        'NO3':dict(project='ref_climatos', product='WOA09', frequency='yr'),
        'Si' :dict(project='ref_climatos', product='WOA09', frequency='yr'),
        'O2' :dict(project='ref_climatos', product='WOA09', frequency='yr'),
        'PO4':dict(project='ref_climatos', product='WOA09', frequency='yr'),
        'alk':dict(project='MY_REFs', frequency='annual', product='GLODAPv2.1', resolution='r360x180xl31'),
        }

means that you will use the dataset dict(project='ref_climatos', product='WOA09', frequency='yr', variable='NO3') as a reference when working with variable NO3 (and so on for the other variables).

With this mechanism you can add any reference to your atlas. If your reference is in your own custom data trees (and has not been described by a CliMAF project yet), see how to add your datasets to CliMAF with this page.

Two simple examples of how to specify your own obs/reference for your variable:

  • First: you have to include the attribute 'product' in the definition of your CliMAF project so that the C-ESM-EP identifies that it is an observational product and not a model simulation
  • Access to my own pre-calculated climatologies: example with the LMDZ obs archive on Ciclad In this example, each obs/reference file in the archive is simply identified by the variable. If you know the name of the obs/reference, you can pass it to 'product' (product='Name_of_my_obs_product').
# -- add this in your parameter file

# -- Create project my_obs -> access to the archive
pattern = '/prodigfs/ipslfs/dods/fabric/lmdz/SE/CMOR/OBS/${variable}.nc'
cproject('my_obs',('period','fx'),('frequency','seasonal'), separator='%')
# we set period to fx and frequency to seasonal for the climatologies
dataloc(project='my_obs', url=pattern) 

# -- Add your obs to the custom_obs_dict
custom_obs_dict = {
        'tas':dict(project='my_obs', product='Name_of_my_obs_product'),
}
  • Access to a reference with time dimension: create your own obs directory and store your obs/reference files following a simple syntax ; since we include ${PERIOD} in the syntax of the file, CliMAF can extract the period you specify with the attribute 'period' in the custom_obs_dict ; you can choose another syntax but this one is simple and allows having multiple variables and products in the same directory): ${variable}${product}${PERIOD}.nc -> tas_ERAInterim_197901-201112.nc
# -- add this in your parameter file

# -- Create project my_ts_obs
pattern = '/data/jservon/Evaluation/ReferenceDatasets/example_my_obs/${variable}_${product}_${PERIOD}.nc'
cproject('my_ts_obs', 'product', separator='%') # we set period to fx and frequency to seasonal for the climatologies
dataloc(project='my_ts_obs', url=pattern) 

# -- Add your obs to the custom_obs_dict
custom_obs_dict = {
        'tas':dict(project='my_ts_obs', product='ERAInterim', period='1979-2005'),
        # Note that you can choose your period here and benefit from the automatic period selection of CliMAF
}

4. Create a derived variable

You might want to work on a variable that does not exists in your files explicitly, but which is a combination of existing variables. For instance, the cloud radiative effect on shortwave (cress) is estimated as the SW flux down at the surface (rsds) minus the SW flux down at the surface in clear-sky conditions (rsdscs). Only rsds and rsdscs are available. We can create cress on the fly using the CliMAF operator derive:

# -- Cloud radiative effect SW at surface
derive('IGCM_OUT','cress','minus','rsds','rsdscs')

You will find many examples of derived variables in $CLIMAF/climaf/derived_variables/ipsl_derived_variables.py. In this file you will see that most of the derived variables take '*' as first argument (instead of 'IGCM_OUT' in the example). It means that this definition of derived variable works for all the projects.

Here are tow other interesting examples: using ccdo to extract one vertical level from a 3D ocean variable:

derive('*','NO3_surf','ccdo','NO3_onevar',operator='sellevidx,1') # First vertical level
derive('*','NO3_300m','ccdo','NO3_onevar',operator='intlevel,300') # Interpolate at 300m

Don't forget the associated reference!

Once you have created your derived variable, do not forget to add the variables it is based on to custom_obs_dict:

custom_obs_dict = {
        'NO3':dict(project='ref_climatos', product='WOA09', frequency='yr'),
}

5. Add plot parameters

Last step: the plot parameters. Atlas Explorer uses by default pre-defined plot parameters (if they exist) that you can see in $CLIMAF/climaf/plot/atmos_plot_params.py . These plot parameters are potentially a scale and/or offset you want to apply before plotting (for instance to convert kg/m2/s to mm/day), and any argument taken by the CliMAF plot operator.

You can add your plot parameters for you new variable tu customize your plots in two different ways:

  • either you add them in custom_plot_params.py (in your main C-ESM-EP working directory, ~/C-ESM-EP/) and this way you can control the plot parameters for three plotting contexts: the climatology ('full_field'), the bias maps ('bias', if the reference contains the attribute 'product') or model-model differences ('model_model', if the reference contains the attribute 'model'); example with 'pr' taken from custom_plot_params:
       'pr' : {
              'default'  : { 'scale' : 86400. , 'color' : 'precip_11lev' , 'contours' : 1 },
            'full_field'   : {'colors':'0.5 1 2 3 4 6 8 10 12 14'  },
            'bias'        : {'color':'MPL_BrBG','colors': '-10 -8 -6 -4 -2 -1 -0.5 -0.2 0.2 0.5 1 2 4 6 8 10' },
            'model_model' : {'color':'precip_diff_12lev','colors': '-5 -2 -1 -0.5 -0.2 -0.1 0.1 0.2 0.5 1 2 5'},
        },

And if eventually you need something that you can't do from Atlas Explorer, consider adding your own diagnostic to the C-ESM-EP!

Clone this wiki locally