Update VariableTable microphys variables from HRRR/RAP newer versions

[gthompsnWRF]

With the newer version of RAP and HRRR files from NCEP, the GRIB2 parameter codes have changed plus I have added all variables to be available related to microphysics (and aerosols). These were tested with Jan-Mar 2019 actual stream of NCEP grib2 files I used in a real-time WRF run and I can validate all variables properly pass through to wrfinput_d0x (and wrfbdy_d0x).

[davegill]

@gthompsnWRF @jimbresch
Greg,
Jim usually puts some documentation at the end of the Vtables to indicate when a change is required for using new data. Here is info that he has in the Vtable.GFS file:

#
#  Vtable for GFS pressure-level data from the ncep server.
#  This version includes fields from the Tropopause and Max Wind levels used by WRF V3.6.1 and later.
#  NCEP has used multiple definitions of the soil temperature in their output and the Vtable attempts
#  to account for these possibilities. (Definition changed 14 Jan 2015).
#
#  ftp://ftpprd.ncep.noaa.gov/pub/data/nccf/com/gfs/prod/gfs.ccyymmddhh/    (note hh at end)
#
#                          approx.    grid    hours      domain  dx       notes
#                         file size   no.
#
#  gfs.t12z.pgrb2.0p25    220000 Kb   193  3-h to 240-h  global  0.25 deg  (26 p-levels plus sfc and trop, 1000 to 10 mb).
#  gfs.t12z.pgrb2.0p50     68000 Kb     4  3-h to 240-h  global  0.5  deg  (26 p-levels plus sfc and trop, 1000 to 10 mb).
#                                          
#  Prior to 12z 14 January 2015:
#
#  gfs.t12z.pgrb2f00       56000 Kb     4  3-h to 192-h  global  0.5  deg  (26 p-levels plus sfc and trop, 1000 to 10 mb).
#  gfs.t12z.pgrbf00.grib2  18000 Kb     3  3-h to 384-h  global  1.0  deg  (26 p-levels plus sfc and trop, 1000 to 10 mb).
#
#
#  As of mid-2017 the GFS provides two land mask fields in the pressure-level output. WPS uses LANDN if available
#  and renames it LANDSEA.
#
#  As of WPS V4.1 (April 2019) the default PMSL is changed to MSLET. MSLET is an unsmoothed sea level pressure.
#  NCEP included MSLET in their GFS files beginning 12z on 10 May 2011 and in their GDAS files at 12z 14 January 2015.
#  The smoother PRMSL is in all GFS/GDAS pressure files.
#  For GFS files prior to those dates use PRMSL as shown in the following line:
#  2 | 102  |   0  |      | PMSL     | Pa      | Sea-level Pressure                      |  0  |  3  |  1  | 101 |
#
#
#   Abbreviated history of GFS changes (all starting at 12 UTC)
#
# Sometime 2019     - Update to FV3 (requires WPS V4.0 or later)
# 19 July 2017      - Change to land mask and other terrestrial variables (requires WPS V3.9.1.1)
#                   - Change missing value, removal of grid 211 (80km conus). Flag value of .468 in soil moisture.
# 11 May 2016       - Assimilate AMSU-A radiances, AVHRR winds, CRTMv2.2.1, 4d hybrid das.
#                   - Add 5 stratospheric levels to ouput. UPP V7.0. Land surface and GWD upates.
# 14 January 2015   - Add MSLET to GDAS. Change from T574 (27km) to T1534 (13km). Fcst length to 240 h.
#                   - semi-lagrangian, uses rtgsst, Z0 dependent on vegetation type, CRTMv2.1.3, 0.25 deg output
# 10 May 2011       - Add MSLET to output. New thermal roughness length. Elevation of buoys to 10m.
#                   - Improved GSI, update CRTM. Recomputed BE.
# 27 July 2010      - T382 (35km) to T574 (27km). new pbl, gravity wave drag, updated deep convective scheme
# 13 May 2008       - RRTM radiation, aerosol, cloud overlap, windsat sfc winds, flow dependent bev.
# 25 September 2007 - Implement UPP.
# 31 May 2005       - T254L64 to T382L64. Change in soil output to 3 levels.
# 29 October 2002   - Change from T170L42  to T254L64. Model top changed from 2 hPA to .2 hPa

Do you have anything you could add in this regard for the RAP / HRRR data? For example, if a user runs a historical case, would there be troubles with the new file?

[gthompsnWRF]

@davegill (FYI) I also believe the soil moisture and temperature variables were changed for the 0cm level, but I cannot say this for certain.

[gthompsnWRF]

@davegill RAPv3/HRRRv2 happened on 23Aug2016 and numerous parameters changed from earlier versions; then on 12July2018 RAPv4/HRRRv3 other changes happened. I don't have enough specifics of which variables changed, but the Rapid Refresh page probably does, although exact GRIB2 parameter table adjustments, I have no idea. Regardless, the QNC variable was lacking entirely along with the aerosol numbers. I do know the QICE, QNR, QNI grib2 param/category changed in the latest version update. Also, I suspect similar things have happened with the GSD versions of same files.

[davegill]

@jimbresch @mgduda @gthompsnWRF
Folks,
I'd prefer if this was a v4.1.1 PR. That allows us without a rush to break this into two pieces.

1. Adding brand new fields - no impact on old data.
2. Putting in the mods that renumber the grib 2 discipline, category, parameter values.

[davegill]

@jimbresch @gthompsnWRF
Greg,
Is there now a 3d RH field? If not, then other than having a surface RH laying around, we can't really use it.

[gthompsnWRF]

@davegill Wasn't my change for Vtable only the RH field at surface/2meters? In which case the 3rd column of the GRIB2 parameters and first column of GRIB1 parameters was lacking a value and put those in. I probably have to reopen my sample GRIB2 file to ensure RH was a field given at 2m. I didn't intend to edit the line for 3D RH, only the surface.

[davegill]

Greg,
The point was if we do not have a 3d RH field, then I do not see any reason to add the surface RH into the Vtable. The metgrid program will just pass t his 2d array on through and the real program will ignore.
Sorry for being confusing in the morning.

[gthompsnWRF]

@davegill Dave, actually I better have another look. Because if RAP files have RH at 2m but not SPECHUMD, then we won't get an output RH at all for that level. The hybrid level files are such a mixture of things, that I can no longer recall from memory and should do a g2print to find out for certain. I now recall that a variable listed without the parameter number itself is a cause for calculating it downstream. So that is how SPECHUMD gets used to calculate a RH field at the real.exe step.

Also, if my (vague) memory is correct, the newest RAP/HRRR files furthermore aligned the soil variables at 0cm to use the same vertical level coding as all the other levels, and you can note the existing Vtable has a different value for 0cm - because I think it was correct pre-July2018. Like I said before, these changes occur without notice by most users, but doing something with the dates in comments is probably the only viable solution.

[davegill]

Greg,
No rush. We will target this Vtable mod for v4.1.1.
That is more than 1 month away.

[mgduda]

@gthompsnWRF @davegill @jimbresch Should we pick this up again for the v4.2 release?

[davegill]

@mgduda @jimbresch @gthompsnWRF
Folks,
I blew off even looking for PRs in WPS, I dropped the ball on this, as I said we'd do 4.1.1. Let me do a test. It should be complete by this afternoon.

[davegill]

@gthompsnWRF
I have a couple of questions about this PR.

1. When I pick up the RAP/HRRR hybrid vertical coordinate data, I do not see some important fields for WRF (soil moisture and soil temp). This was not part of your PR, so you did not break anything. However, no one can actually use this data as-is. Where do you get your additional fields?

2. You included mods in this PR for new fields: QNC, QNWFA, QNIFA. The associated grib codes are not in the RAP/HRRR hybrid coordinate data set on the NCEP ftp server. Here is every 3d variable. The only two number concentration fields are QNI (0 6 29 105) and QNR (0 1 100 105), both labeled as UNKNOWN below:

   1   0    3   0     105       1       0       0     PRES     2020-04-19_00:00:00   00
   2   0    1  22     105       1       0       0     CLMR     2020-04-19_00:00:00   00
   3   0    1  82     105       1       0       0     CIMIXR   2020-04-19_00:00:00   00
   4   0    1  24     105       1       0       0     RWMR     2020-04-19_00:00:00   00
   5   0    1  25     105       1       0       0     SNMR     2020-04-19_00:00:00   00
   6   0    1  32     105       1       0       0     GRMR     2020-04-19_00:00:00   00
   7   0    6  29     105       1       0       0     UNKNOWN  2020-04-19_00:00:00   00
   8   0    1 100     105       1       0       0     UNKNOWN  2020-04-19_00:00:00   00
   9   0    3   5     105       1       0       0     HGT      2020-04-19_00:00:00   00
  10   0    0   0     105       1       0       0     TMP      2020-04-19_00:00:00   00
  11   0    1   0     105       1       0       0     SPFH     2020-04-19_00:00:00   00
  13   0    2   2     105       1       0       0     UGRD     2020-04-19_00:00:00   00
  13   0    2   3     105       1       0       0     VGRD     2020-04-19_00:00:00   00
  14   0    2   8     105       1       0       0     VVEL     2020-04-19_00:00:00   00
  15   0   19  11     105       1       0       0     TKE      2020-04-19_00:00:00   00

Did you have a special data set when using this modified Vtable?

[gthompsnWRF]

@davegill NCEP ftp servers versus what is distributed via CONDUIT may be the answer. I haven't ever touched their ftp server. I get their real-time data over LDM using CONDUIT and I assure you the files have everything I need. I can point you to a sample grb2 file. I didn't realize they might be serving different files on that circuit - which is a real DRAG.

[davegill]

@mgduda @gthompsnWRF @jimbresch
Folks,
With a little bit of help from my friends, this PR works OK for me with the referenced NCEP ftp data to create a successful WRF simulation.

1. I modified the text at the bottom of the Vtable, as the grid 252 data is no longer sufficient for WRF. The grid 130 data is OK, however. This grid 130 dataset does not include the two friendly aerosol fields, but the newly added number concentration for cloud is present.

I am OK with this PR as it now stands to get into v4.2

[mgduda]

@davegill Can you leave your approval as a GitHub review?

[davegill]

Approved