Support Additional FESOM2 Grid Formats

[philipc2]

Closes #1005

Overview

• Adds support for reading NetCDF and ASCII formatted FESOM2 Grids

[philipc2]

@fluidnumerics-joe

Do you have any examples of FESOM2 grids that are stored as netcdf files?

[fluidnumerics-joe]

Here is the NetCDF version of the Soufflet example (ASCII link for reference)

Here's some additional info

The NetCDF files, afaik, are generated from the ASCII files using an older version of pyfesom2; specifically, these methods

• read_fesom_ascii_grid
• write_mesh_to_netcdf

The NetCDF file is a CDO compliant NetCDF format.

[philipc2]

Here is the NetCDF version of the Soufflet example (ASCII link for reference)

Here's some additional info

The NetCDF files, afaik, are generated from the ASCII files using an older version of pyfesom2; specifically, these methods

• read_fesom_ascii_grid
• write_mesh_to_netcdf

The NetCDF file is a CDO compliant NetCDF format.

@fluidnumerics-joe

Was able to generate a grid using the pi mesh!

Although, the soufflet mesh is giving me some trouble.

Do you have any information on the soufflet grid? The triangles along the borders appear to be periodic, though I wouldn't expect them to given their longitude values.

[fluidnumerics-joe]

@philipc2 - You're right about the soufflet grid; the domain is east-west periodic.

I artificially trimmed out those connections in a rather hacky way ( See the "Purging Periodic Elements" section here ). In general that shouldn't be needed, though. Perhaps the soufflet was not the best example, but you did indeed get a similar looking graphic for this example that I started with.

[philipc2]

@fluidnumerics-joe

This should be ready for an initial review. I haven't done any testing for UxDataArray or UxDataset since I am uncertain how history/data files are stored. Let me know if you have any insight.

[rajeeja]

good first draft, thanks!

pi folder has a lot of other files that should be removed, readme is needed for all files that are contained. I think only nod2d and ele2d are needed and the rest can be deleted.

open_grid loads the ascii, how about add another test that loads a netcdf version of the same files as you check for "triag_nodes" to specify that it is FESOM2 grid format.

Is there is FESOM1?

What is soufflet?

[philipc2]

pi folder has a lot of other files that should be removed, readme is needed for all files that are contained. I think only nod2d and ele2d are needed and the rest can be deleted.

Yeah, there's a lot of files and I'm not 100% sure what each of them represents yet. @fluidnumerics-joe do you have a good reference to what these other files represent?

open_grid loads the ascii, how about add another test that loads a netcdf version of the same files as you check for "triag_nodes" to specify that it is FESOM2 grid format.

Good idea!

Is there is FESOM1?

There is, though I believe there is differences in the model output. Will do some reading.

What is soufflet?

The name of one of the diagnostic grid files used in FESOM2

[fluidnumerics-joe]

I haven't done any testing for UxDataArray or UxDataset since I am uncertain how history/data files are stored. Let me know if you have any insight.

I'm not entirely sure at the moment but @koldunovn may be able to fill in some gaps.

[fluidnumerics-joe]

Yeah, there's a lot of files and I'm not 100% sure what each of them represents yet. @fluidnumerics-joe do you have a good reference to what these other files represent?

This is the reference I've been working from : https://fesom2.readthedocs.io/en/latest/meshes/meshes.html

The aux3d.out is for the vertical grid discretzation.
I'm not sure about the dist_* subdirectories or other files outside of nod2d.out or elem2d.out either.

I'll kick the tires with this branch though, and see if I can get a hold of some sample model output to help out with the UXDataArray bits

[rajeeja]

Cool, thanks for the update. @philipc2 a new backed (ASCII) for this open_grid file, is needed?

[koldunovn]

First of all, it’s great to see that you’ve implemented this—thanks a lot! I’m really glad FESOM meshes can be supported now. I’ll try to answer some questions and also ping @dsidoren, @suvarchal, @JanStreffing, @patrickscholz, and @pgierz—they might have more information to add.

The bare minimum, as you mentioned, consists of three files: nod2d.out, elem2d.out, and aux3d.out (which was supposed to be renamed to depth.out, but never was :). All other files in the directory are computed from the mesh topology described in those files.

There are a few ways to generate additional mesh information (like edges and so on) and netCDF files from ASCII:

• FESOM2 generates some additional ASCII files during partitioning of the mesh—things like edges.out, nlvls.out, and so on.
• FESOM2 also generates a fesom.mesh.diag.nc file when running the first time step from rest.
• There is an R package that produces netCDF files that CDO can work with.
• Finally, there are pyfesom2 scripts created by @JanStreffing, which produce files suitable for CDO post-processing. This is currently the best option. Since FESOM2 has tracers on vertices (we call them nodes for historical reasons) and vector variables on the centers of triangles, two mesh descriptions are necessary for post-processing with CDO (@JanStreffing, correct me if this is no longer the case for your scripts).

The folder also contains dist_* directories. In these, you’ll find partitions needed to distribute workload between cores during a model run. They are not needed for data processing or mesh description generation, and if you don’t plan to dive into complex parallel optimization of post-processing, you can safely delete them.

soufflet is indeed a periodic channel. The name comes from the first author of this publication, where this channel was introduced.

The README for the files can point to the FESOM2 mesh documentation page.

There is also FESOM1, which has a different mesh staggering. It is used in CMIP6 data (for example here), so some people still use it. The old pyfesom used the FESOM1 mesh.

@philipc2 Not sure what you mean by “history/data files.” I’d be happy to provide information but need a bit more explanation :)

I hope this helps! Let me know if you need more input from the FESOM team!

[philipc2]

@koldunovn

Thank you so much for the detailed write up! This clarifies the majority of questions that I had.

The folder also contains dist_* directories. In these, you’ll find partitions needed to distribute workload between cores during a model run. They are not needed for data processing or mesh description generation, and if you don’t plan to dive into complex parallel optimization of post-processing, you can safely delete them.

Got it! This does sound like something I'd be interested in reading into though.

Not sure what you mean by “history/data files.” I’d be happy to provide information but need a bit more explanation :)

I was wondering how data variables (i.e. temperature) are stored/represented when running the model. The files here are sufficient for creating a Grid, however to create a UxDataset or UxDataArray we need data to map to the grid.

[koldunovn]

@philipc2

Ah, I see now :) Here is one month of a model run on the pi mesh. The folder contains all output, including various types of restart files, so a lot of it may not be very interesting for you. Just focus on the netCDF files in the main folder. I believe I saved variables with all combinations of horizontal and vertical meshes (some variables are saved at the midpoints of vertical levels, and some at the level boundaries :)).

Regarding partitioning—we use METIS to split the mesh between the cores. A very brief description of why we use it is here. For example, here is partitioning for 8 cores on the pi mesh:

Hope it helps :)

[philipc2]

#1025 has added support for reading the UGRID diagnostic grid file from FESOM output.

[fluidnumerics-joe]

@philipc2 - This looks good. I tested out both the ascii and netcdf readers for FESOM2 and all looks good. Looking forward to having this in an upcoming uxarray release .

[fluidnumerics-joe]

@philipc2 - While I am able to read in fesom2 grid files ok, I'm curious to know where things are at with the fields. Are the field files that @koldunovn what you might expect for loading into UXarray, with each field in its own netcdf file ?

[philipc2]

@philipc2 - While I am able to read in fesom2 grid files ok, I'm curious to know where things are at with the fields. Are the field files that @koldunovn what you might expect for loading into UXarray, with each field in its own netcdf file ?

For opening a single file paired with a grid:

# Directory containing ASCII files
grid_path = base_path + "pi/"

# Directory containing Data Files
data_path = base_path + "pi/data/"

uxds = ux.open_dataset(grid_path, data_path + 'sst.fesom.1985.nc')

For opening all files paired with a grid:

from glob import glob
data_paths = glob(data_path + "*.nc")
uxds = ux.open_mfdataset(grid_path, data_paths)

[rajeeja]

Do we need TestCase from pytest, like we have it for our test_grid etc.?

[philipc2]

May you elaborate?

[philipc2]

If you mean using the TestCase class, then no it's not always required.