Integration of UFS Case Studies ICs into UFS-WM

[ulmononian]

Description

As a step toward an HTF/HSD framework for the UFS-WM, enabling the WM to run using GFS ICs from the UFS Case Studies page was identified as a desirable feature. This issue discusses a possible approach toward the development of this capability.

Solution

To achieve this, ICs corresponding to the initialization date/time of a particular case need to be generated. One way to do this is with chgres_cube, which can be built as part of UFS_UTILS. The conversion can be done automatically via the gdas_init utility, also available in UFS_UTILS. gdas_init can be configured to pull raw GFS data from HPSS onto a local machine (Hera or Jet) and convert the raw data (via chgres_cube) into the cold start format (including desired vertical level # and resolution) required by the ATM component of the UFS-WM. The converted cold start ICs can then be used as input data for certain UFS-WM RTs of corresponding model resolution and configuration, wherein the user need only replace the sfc*.nc and gfs*.nc files in the INPUT directory of the RT run directory and adjust the model start time in the model_configure file. Note that the develop version (as of 6/4/24) of UFS_UTILS generates the so-called v2 sfc files via gdas_init and chgres_cube, so at this time, successful integration of the converted cold start files has only been achieved using the somewhat recently added *v2.sfc WM RTs (e.g., https://github.com/ufs-community/ufs-weather-model/blob/develop/tests/tests/control_c48.v2.sfc; see #2005 and #1977). Since the v2 surface files are significantly different from the v1 surface file format, v2 files do not seem to work with non-v2.sfc RTs.

To generate cold start ICs via UFS_UTILS/gdas_init/chgres_cube on an RDHPC with HPSS access (Hera or Jet), the user should do the following:

git clone https://github.com/ufs-community/UFS_UTILS.git
cd UFS_UTILS
cd fix
./link_fixdirs.sh emc <Hera or Jet>
cd ..
./build_all.sh
cd util
cd gdas_init
#edit the `config` file to set paths for data extraction and converted files (`EXTRACT_DIR` and `OUTDIR`, respectively); set  yy/mm/dd/hh to desired forecast start time +6 hours); set CDUMP to `gfs`  to generate GFS ICs; for the UFS-WM, set `LEVS` to 128; set `EXTRACT_DATA` to `yes`(unless data is already staged in `EXTRACT_DIR`); set `CRES_HIRES` to desired resolution for cold start files (e.g., C48)
# edit `PROJECT_CODE` in the machine-specific driver (e.g., `driver.jet.sh`) to an account user can use for submitting batch jobs; 
# run the driver script
./driver.<machine.sh

These steps will (1) pull raw GFS data from HPSS into EXTRACT_DIR and then (2) convert the raw data to cold start ICs (placed in OUTDIR, appended by gfs.YYYYMMDD/HH/model_data/atmos/input). To use the converted cold start files with the UFS-WM, copy the gfs*.nc and sfc*.nc files from the OUTDIR/gfs.YYYYMMDD/HH/model_data/atmos/input directory into the UFS-WM RT run directory's INPUT directory. Note that the resolution of the converted cold start GFS data must match the ATM resolution of the UFS-WM RT (e.g., use C48 GFS cold start data in the control_c48.v2.sfc RT). Then, edit the model start year/month/day/hour in the RT model_configure. The user can then re-run the fv3.exe via sbatch job_card. The case should now run at the new model start time with the cold start ICs.

An example can be found in /lfs4/HFIP/hfv3gfs/Cameron.Book/RT_RUNDIRS/Cameron.Book/FV3_RT/rt_1826477/control_c48.v2.sfc_intel, which used cold start ICs generated through gdas_init/chgres_cube for the 2020 CAPE case (see /lfs4/HFIP/hfv3gfs/Cameron.Book/chgres_data/converted/gfs.20200723/00/model_data/atmos/input) from the UFS Case Studies page.

Note that, in order to generate all necessary configuration/data/input/fix files to run in this method, the user first needs to run the base RT using `./rt.sh -n ' to set-up the run directory. However, this is only a temporary method to achieve integration of the UFS Case Studies into the WM for HTF/HSD. The test cases will be integrated with the YAML-based testing feature as detailed in WM issue #2182 and PR #2278 (@jkbk2004).

Alternatives

Related to

Issue #2182 and PR #2278

[DeniseWorthen]

@ulmononian I don't understand the reasoning behind this issue. The UWM Regression Test framework is for regression testing of the UWM against known input and known output. It is designed to test code changes coming in through the PR processs: do the code changes a) break anything (ie, fail debug) or b) change answers in something other than an expected and documented way?

Why are you suggesting that the RT system accomodate a user's own desired test input? A user can already do that by (as you suggest), creating a sandbox and updating w/ their own input. Why are you suggesting that this should be a functionality of the regression test system?

[ulmononian]

@DeniseWorthen i appreciate your perspective! this work is not intended to end up in the RT framework. the RT setup is just being used for now to setup the run directory and demonstrate how to use other GFS cold start ICs with the WM. it is not meant to impact or change the current RT framework

[DeniseWorthen]

@ulmononian Thanks for the update. Is there a better location for this other than as an issue/enhancement to UWM? If it isn't intended to come into the UWM code base/RT framework, perhaps opening it as a discussion would be more appropriate.

[DeniseWorthen]

@ulmononian Please consider this converting this to a discussion, since there is no intention for this to be brought in the UWM code base. At the bottom of the right most column, you'll see a 'convert to disussion' option.