Feature/minmaxtrh #183

EdwardColon-NOAA · 2023-12-29T19:55:55Z

Additional workflow steps, ex-scripts, and j-job scripts have been added in order to facilitate the computation of min/max 2m temperatures and relative humidities for RTMA parallels.

These changes are listed below:

exrrfs_run_maxrh.ksh
exrrfs_run_minrh.ksh
exrrfs_run_mintbg.ksh
exrrfs_run_maxtbg.ksh
JRRFS_RUN_MAXTMINRH
JRRFS_RUN_MINTMAXRH

… as expanded the prdgen products to including SPC-specific fields.

guoqing-noaa · 2023-12-31T15:44:47Z

fix/upp/spc_fields_rtma.txt

@EdwardColon-NOAA We no longer use "spc_fields_rtma.txt". If you want to update fields for SPC, please go ahead to update the file fix/upp/testbed_fields_rtma.txt.

@EdwardColon-NOAA, is this file still necessary for your PR? If not, please remove this file.

guoqing-noaa · 2023-12-31T15:46:44Z

jobs/JRRFS_RUN_MAXTMINRH

+set -x
+
+if [ ${cyc} != "07" ] ; then
+    echo -e "\nWARNING: GETMINT only runs at 07z. Exiting.\n"


Suggest changing "WARNING:" to something like "INFO:" or just removing "WARG"

guoqing-noaa · 2023-12-31T15:48:44Z

jobs/JRRFS_RUN_MINTMAXRH

+set -x
+
+if [ ${cyc} != "19" ] ; then
+    echo -e "\nWARNING: GETMINT only runs at 19z. Exiting.\n"


Similar comment about "WARNING:"

guoqing-noaa · 2023-12-31T15:57:24Z

parm/FV3LAM_wflow.xml

@@ -1235,13 +1249,9 @@ MODULES_RUN_TASK_FP script.
    <envar><name>CYCLE_SUBTYPE</name><value><cyclestr>spinup</cyclestr></value></envar>
    <envar><name>SLASH_ENSMEM_SUBDIR</name><value><cyclestr>{{ slash_ensmem_subdir }}</cyclestr></value></envar>
    <envar><name>NWGES_BASEDIR</name><value>&NWGES_BASEDIR;</value></envar>
-    <envar><name>ENSCTRL_CYCLE_DIR</name><value><cyclestr>&ENSCTRL_CYCLE_BASEDIR;/@Y@m@d@H</cyclestr></value></envar>


It looks like these differences should not be here. I guess you might not merge the latest dev-sci branch to your PR correctly for FV3LAM_wflow.xml.

guoqing-noaa · 2023-12-31T16:10:19Z

scripts/exrrfs_run_maxrh.ksh

+  "WCOSS2")
+    module load wgrib2/2.0.8
+    module list
+    wgrib2=/apps/ops/prod/libs/intel/19.1.3.304/wgrib2/2.0.8_wmo/bin/wgrib2


For wcoss2, you will need to add a local.lua file under modulefiles/tasks/wcoss2
For other machines, you don't need to do anything because build_.lua is loaded in the first place for any workflow tasks. wgrib2 is already available and no extra module load is needed.

As @guoqing-noaa pointed out, the modules necessary for each task should be loaded in rrfs-workflow/modulefiles/tasks/wcoss2/[task_name].local.lua. Please add a new file for run_maxrh.

guoqing-noaa · 2023-12-31T17:00:53Z

ush/generate_FV3LAM_wflow.sh

@@ -610,6 +621,7 @@ ${EXPTDIR}/run_rocoto.sh"

 echo "#!/bin/bash" > ${EXPTDIR}/run_rocoto.sh
 if [[ "${MACHINE,,}" == "wcoss2" ]] ; then
+  rocoto_ver=1.3.5


I think the rocoto_ver env variable should be available before line 625.

@chan-hoo Could you confirm whether we need to hardwire a version information in line 624? Thanks!

To meet the NCO standards, the module version should be defined in rrfs-workflow/versions/run.ver on WCOSS2.

guoqing-noaa · 2023-12-31T17:02:01Z

ush/sample_configs/RTMA/config.sh.RTMA_CONUS_3km

@@ -21,6 +21,7 @@ if [[ -n $RESERVATION ]] ; then
  QUEUE_FCST="rth"
  QUEUE_ANALYSIS="rth"
  QUEUE_PRDGEN="rth"
+  QUEUE_MINMAXTRH="rt"   `


remove unexpected trailing sting " `"
Also, replace "rt" with "rth"

guoqing-noaa · 2023-12-31T17:02:42Z

ush/sample_configs/RTMA/config.sh.RTMA_CONUS_3km

@@ -59,6 +60,7 @@ envir="para"
 USE_CUSTOM_POST_CONFIG_FILE="TRUE"
 TESTBED_FIELDS_FN="testbed_fields_rtma.txt"
 TESTBED_FIELDS_FN2="testbed_fields_bgrd3d.txt"
+SPC_FIELDS_FN="spc_fields_rtma.txt"


remove line 63

guoqing-noaa · 2023-12-31T17:04:18Z

ush/sample_configs/RTMA/config.sh.RTMA_NA_3km

@@ -53,6 +55,7 @@ envir="para"
 USE_CUSTOM_POST_CONFIG_FILE="TRUE"
 TESTBED_FIELDS_FN="testbed_fields_rtma.txt"
 TESTBED_FIELDS_FN2="testbed_fields_bgrd3d.txt"
+SPC_FIELDS_FN="spc_fields_rtma.txt"


remove line 58

guoqing-noaa · 2023-12-31T17:08:20Z

ush/setup.sh

@@ -811,6 +816,7 @@ FIX_SMOKE_DUST=${FIX_SMOKE_DUST:-"${HOMErrfs}/fix/smoke_dust"}
 FIX_BUFRSND=${FIX_BUFRSND:-"${HOMErrfs}/fix/bufrsnd"}
 AIRCRAFT_REJECT=${AIRCRAFT_REJECT:-"${FIX_GSI}"}
 SFCOBS_USELIST=${SFCOBS_USELIST:-"${FIX_GSI}"}
+FIX_MINMAXTRH=${FIX_MINMAXTRH:-"${HOMErrfs}/fix/minmaxtrh"}


Could you make a copy of FIX_MINMAXTRH to Hera or Jet? I can facilitate syncing them to all available rrfs-workflow platforms.

We need to sync fix files in all rrfs-workflow platforms.

guoqing-noaa · 2023-12-31T17:12:35Z

ush/config_defaults.sh

@@ -1539,6 +1541,7 @@ SFC_CLIMO_DIR=""
 NCORES_PER_NODE=24 #Jet default value
 IS_RTMA="FALSE"
 FG_ROOTDIR=""
+RRFS_GRIBDIR=""


Suggest changing RRFS_GRIBDIR to RRFS_PRODDIR

jobs/JRRFS_RUN_MAXTMINRH

jobs/JRRFS_RUN_MINTMAXRH

scripts/exrrfs_run_minrh.ksh

scripts/exrrfs_run_mintbg.ksh

chan-hoo · 2024-01-02T12:31:51Z

ush/generate_FV3LAM_wflow.sh

@@ -610,6 +621,7 @@ ${EXPTDIR}/run_rocoto.sh"

 echo "#!/bin/bash" > ${EXPTDIR}/run_rocoto.sh
 if [[ "${MACHINE,,}" == "wcoss2" ]] ; then
+  rocoto_ver=1.3.5


To meet the NCO standards, the module version should be defined in rrfs-workflow/versions/run.ver on WCOSS2.

chan-hoo · 2024-01-02T12:37:26Z

@EdwardColon-NOAA, Please let us know which sample configuration you used. We'd like to test your PR with the configuration again before approving it.

Added the contents of spc_fields_rtma.txt to testbed_fields_rtma.txt in order to consolidate postprocessing.

Removed references to SPC prdgen output file.

removed reference to spc_fields_rtma.txt

Changed the RRFS model output filename from RRFS_GRIBDIR to RRFS_PRODDIR. Also removed reference to SPC output.

removed rocoto_ver module load.

guoqing-noaa · 2024-01-03T17:44:31Z

ush/set_rrfs_config_general.sh

@@ -284,7 +294,7 @@ if [[ ${PREDEF_GRID_NAME} == "RRFS_CONUS_3km" ]] ; then
    LAYOUT_X="30"
    LAYOUT_Y="40"
    PPN_RUN_FCST="40"
-    WTIME_RUN_PRDGEN="00:20:00"
+    WTIME_RUN_PRDGEN="00:20:00


Need a matching " at the end.

guoqing-noaa · 2024-01-03T17:46:38Z

fix/upp/spc_fields_rtma.txt

@@ -0,0 +1,347 @@
+PRES:surface:


Thanks for incorporating changes into testbed_fields_rtma.txt. This spc_fields_rtma.txt file may be deleted safely now.

corrected value of QUEUE_MINMAXTRH to rth,

Removed RUN_ENVIR & RUN environment variables.

Removed RUN_ENVIR and RUN environment variables.

removed machine dependencies on module load scripts and updated the executable command lines.

Removed machine dependencies in module loads and updated executable command.

Removed machine-dependent module loads and updated executable statement.

Removed machine-dependent module loads and updated executable command line.

modified RRFS_GRIBDIR to RRFS_PRODDIR

guoqing-noaa · 2024-04-15T18:37:59Z

scripts/exrrfs_bufrsnd.sh

@@ -269,12 +277,13 @@ ln -sf ./itag                              fort.11

  echo DONE $fhr at `date`

-  mv $DATA/bufrpost/profilm.c1.${tmmark} $DATA/profilm.c1.${tmmark}.f${fhr}
+  cp $DATA/profilm.c1.${tmmark} $DATA/profilm.c1.${tmmark}.f${fhr}


This PR not only changes "mv" to "cp" but also changes the directory structure (i.e. removes the "bufrpost" subdir). why do rrfs and rtma have different treatments here?

guoqing-noaa · 2024-04-15T18:39:12Z

scripts/exrrfs_bufrsnd.sh

-  mv $DATA/profilm_int $DATA/profilm.c1.${tmmark}
-
+  if [[ "${NET}" = "RTMA"* ]]; then
+    mv $DATA/profilm_int $DATA/profilm.c1.${tmmark}


To confirm, rrfs version does not need to do this "mv"?

guoqing-noaa · 2024-04-15T18:40:41Z

scripts/exrrfs_run_prdgen.sh

    for grid in ${ADDNL_OUTPUT_GRIDS[@]}
    do
-      for leveltype in prslev natlev ififip testbed
+      for leveltype in prslev natlev ififip testbed 


remove the trailing space as there is no change in this line.

Resolved DO_MINMAXTRH option list index.

Removed blank line and correct QUEUE labels.

Updated version number

Removed extraneous conditional block

Removed minmaxtrh fix file path.

Updated the node count variable for MINMAXTRH.

Updated several more variable names.

Converted cp command on line 280 back to mv.

Trailing space on line 446 removed.

edwardhartnett · 2024-09-22T08:14:57Z

scripts/exrtma_run_maxtbg.ksh

@@ -0,0 +1,243 @@
+#!/bin/bash


Is there a reason these files are named *.ksh, even though they are bash shells?

EdwardColon-NOAA · 2024-09-23T00:04:42Z

Hello Edward, This PR will be supplanted as a new code is being finalized for generating min/max T & RH calculations on the North American domain rather than on the HRRR domain. The *.ksh file suffix is a residual suffix for older versions of the min/max T & RH ex-scripts which employed the korn shell rather than bash shell. That will be rectified.

…

On Sun, Sep 22, 2024 at 4:15 AM Edward Hartnett ***@***.***> wrote: ***@***.**** commented on this pull request. ------------------------------ In scripts/exrtma_run_maxtbg.ksh <#183 (comment)> : > @@ -0,0 +1,243 @@ +#!/bin/bash Is there a reason these files are named *.ksh, even though they are bash shells? — Reply to this email directly, view it on GitHub <#183 (review)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACFXXQHPHNW75K72X4VUOYLZXZ4BPAVCNFSM6AAAAABBG6SBJOVHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMZDGMRQGM4TKOBXGA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

-- ------------------------------------------------- Edward Colón *Lynker* at NOAA/NWS/NCEP/EMC 5830 University Research Ct. Rm #2025 Riverdale, MD 20737 Office: (301) 683-3815 Cell: (301) 213-3566

edwardhartnett · 2024-10-05T14:45:29Z

Is there an issue associated with this PR?

guoqing-noaa · 2024-10-05T14:51:06Z

Is there an issue associated with this PR?

This PR needs to be updated to catch the latest rrfs.v1 version.
@EdwardColon-NOAA Do you plan to continue to work on this or close this PR and create a new one?

EdwardColon-NOAA · 2024-10-05T17:48:50Z

Hello, Yes, the PR was initially designed to introduce the capability of min/max T and RH products produced by the RRFS-based 3D-RTMA. However, since it was initially committed, the scope has changed since those products were specified on a HRRR specified domain rather than a North American domain. The update has forced significant modifications in the existing min/max T & RH executable source codes as well as the ex-scripts. There has also been a further complication with a disagreement between the first guess and analysis products. This discrepancy should be the last issue to resolve before testing, an update in existing code to match the latest rrfs.v1 version, and a recommit.

…

On Sat, Oct 5, 2024 at 10:45 AM Edward Hartnett ***@***.***> wrote: Is there an issue associated with this PR? — Reply to this email directly, view it on GitHub <#183 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACFXXQGTZFBJ2U4UA4RHFS3ZZ73SBAVCNFSM6AAAAABBG6SBJOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGOJVGA4DCMBVHE> . You are receiving this because you were mentioned.Message ID: ***@***.***>

-- ------------------------------------------------- Edward Colón *Lynker* at NOAA/NWS/NCEP/EMC 5830 University Research Ct. Rm #2025 Riverdale, MD 20737 Office: (301) 683-3815 Cell: (301) 213-3566

EdwardColon-NOAA · 2024-10-05T17:50:15Z

Hi Guoqing, I am leaning towards closing this PR and issuing a new one since there need to be updates to the min/max source codes in the rrfs_util repository as well.

…

-Edward

On Sat, Oct 5, 2024 at 10:51 AM Guoqing Ge ***@***.***> wrote: Is there an issue associated with this PR? This PR needs to be updated to catch the latest rrfs.v1 version. @EdwardColon-NOAA <https://github.com/EdwardColon-NOAA> Do you plan to continue to work on this or close this PR and create a new one? — Reply to this email directly, view it on GitHub <#183 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACFXXQBLL67FSFDAUZBZENDZZ74G7AVCNFSM6AAAAABBG6SBJOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGOJVGA4DENJSGM> . You are receiving this because you were mentioned.Message ID: ***@***.***>

-- ------------------------------------------------- Edward Colón *Lynker* at NOAA/NWS/NCEP/EMC 5830 University Research Ct. Rm #2025 Riverdale, MD 20737 Office: (301) 683-3815 Cell: (301) 213-3566

edwardhartnett · 2024-10-05T23:21:16Z

Can we please start an issue for this work?

hu5970 · 2024-10-11T17:42:38Z

@EdwardColon-NOAA Please close this PR and start an issue for this work if needed.

EdwardColon-NOAA · 2024-10-11T17:43:46Z

Will do.

…

-Edward

On Fri, Oct 11, 2024 at 1:43 PM Ming Hu ***@***.***> wrote: @EdwardColon-NOAA <https://github.com/EdwardColon-NOAA> Please close this PR and start an issue for this work if needed. — Reply to this email directly, view it on GitHub <#183 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACFXXQGNK4PBW2HVMTA6AODZ3AE2HAVCNFSM6AAAAABBG6SBJOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMBXHA3DOMJWGM> . You are receiving this because you were mentioned.Message ID: ***@***.***>

-- ------------------------------------------------- Edward Colón *Lynker* at NOAA/NWS/NCEP/EMC 5830 University Research Ct. Rm #2025 Riverdale, MD 20737 Office: (301) 683-3815 Cell: (301) 213-3566

guoqing-noaa · 2024-10-11T17:44:32Z

close as agreed.

edward colon and others added 2 commits December 28, 2023 22:17

Integrated min/max T & RH workflow steps into existing scheme as well…

1678841

… as expanded the prdgen products to including SPC-specific fields.

Minmax T2M & RH2M update.

26fb800

EdwardColon-NOAA requested review from MatthewPyle-NOAA, JacobCarley-NOAA and chan-hoo as code owners December 29, 2023 19:55

Merge branch 'dev-sci' into feature/minmaxtrh

1095704

guoqing-noaa reviewed Dec 31, 2023

View reviewed changes

chan-hoo suggested changes Jan 2, 2024

View reviewed changes

chan-hoo linked an issue Jan 2, 2024 that may be closed by this pull request

Adding capability of computing mininmum and maximum 2m temperatures and relative humidities for RTMA parallels #182

Open

EdwardColon-NOAA added 7 commits January 2, 2024 13:45

Update JRRFS_RUN_MAXTMINRH

0b54a72

Update JRRFS_RUN_MINTMAXRH

8635127

Update testbed_fields_rtma.txt

93310d6

Added the contents of spc_fields_rtma.txt to testbed_fields_rtma.txt in order to consolidate postprocessing.

Update exrrfs_run_prdgen.sh

6c5c873

Removed references to SPC prdgen output file.

Update config.sh.RTMA_NA_3km

cffcb9e

removed reference to spc_fields_rtma.txt

Update config_defaults.sh

246e2b6

Changed the RRFS model output filename from RRFS_GRIBDIR to RRFS_PRODDIR. Also removed reference to SPC output.

Update generate_FV3LAM_wflow.sh

6f996e2

removed rocoto_ver module load.

guoqing-noaa reviewed Jan 3, 2024

View reviewed changes

EdwardColon-NOAA added 10 commits January 3, 2024 13:43

Update config.sh.RTMA_CONUS_3km

ed4ea30

corrected value of QUEUE_MINMAXTRH to rth,

Update set_rrfs_config_general.sh

403d773

Update JRRFS_RUN_MAXTMINRH

9c7ecd7

Removed RUN_ENVIR & RUN environment variables.

Update JRRFS_RUN_MINTMAXRH

7ea26c8

Removed RUN_ENVIR and RUN environment variables.

Update exrrfs_run_maxrh.ksh

be97fc1

removed machine dependencies on module load scripts and updated the executable command lines.

Update exrrfs_run_maxtbg.ksh

1c60060

Removed machine dependencies in module loads and updated executable command.

Update exrrfs_run_minrh.ksh

b21c8d5

Removed machine-dependent module loads and updated executable statement.

Update exrrfs_run_mintbg.ksh

b6e1257

Removed machine-dependent module loads and updated executable command line.

Update FV3LAM_wflow.xml

3a6287c

Update FV3LAM_wflow.xml

11fddf2

modified RRFS_GRIBDIR to RRFS_PRODDIR

guoqing-noaa reviewed Apr 15, 2024

View reviewed changes

EdwardColon-NOAA and others added 14 commits April 15, 2024 18:39

Update setup.sh

86a8155

Resolved DO_MINMAXTRH option list index.

Update set_rrfs_config_general.sh

e392178

Removed blank line and correct QUEUE labels.

Update config.sh.RTMA_NA_3km

de48813

Updated version number

Update generate_FV3LAM_wflow.sh

8df3fe2

Removed extraneous conditional block

Update config.sh.RTMA_CONUS_3km

e4c8652

Removed minmaxtrh fix file path.

Update config_defaults.sh

2494a09

Updated the node count variable for MINMAXTRH.

Update config_defaults.sh

c1429e7

Updated several more variable names.

Update generate_FV3LAM_wflow.sh

3f26e56

Update exrrfs_bufrsnd.sh

166660e

Converted cp command on line 280 back to mv.

Update exrrfs_run_prdgen.sh

d64888a

Trailing space on line 446 removed.

Merge branch 'dev-sci' into feature/minmaxtrh

ed55ac0

Changes from the latest PR included.

978497f

merging recent changes.

5098a11

Merge branch 'dev-sci' into feature/minmaxtrh

c6cf2b1

edwardhartnett reviewed Sep 22, 2024

View reviewed changes

guoqing-noaa closed this Oct 11, 2024

EdwardColon-NOAA deleted the feature/minmaxtrh branch October 11, 2024 17:48

EdwardColon-NOAA mentioned this pull request Oct 11, 2024

The Feature/minmaxtrh (PR #183) was closed #526

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Feature/minmaxtrh #183

Feature/minmaxtrh #183

EdwardColon-NOAA commented Dec 29, 2023

guoqing-noaa Dec 31, 2023 •

edited

Loading

chan-hoo Jan 6, 2024

guoqing-noaa Dec 31, 2023

guoqing-noaa Dec 31, 2023

guoqing-noaa Dec 31, 2023

guoqing-noaa Dec 31, 2023

chan-hoo Jan 2, 2024

guoqing-noaa Dec 31, 2023

chan-hoo Jan 2, 2024

guoqing-noaa Dec 31, 2023

guoqing-noaa Dec 31, 2023

guoqing-noaa Dec 31, 2023

guoqing-noaa Dec 31, 2023

guoqing-noaa Jan 7, 2024

guoqing-noaa Dec 31, 2023

chan-hoo Jan 2, 2024

chan-hoo commented Jan 2, 2024 •

edited

Loading

guoqing-noaa Jan 3, 2024

guoqing-noaa Jan 3, 2024

guoqing-noaa Apr 15, 2024

guoqing-noaa Apr 15, 2024

guoqing-noaa Apr 15, 2024

edwardhartnett Sep 22, 2024

EdwardColon-NOAA commented Sep 23, 2024 via email

edwardhartnett commented Oct 5, 2024

guoqing-noaa commented Oct 5, 2024

EdwardColon-NOAA commented Oct 5, 2024 via email

EdwardColon-NOAA commented Oct 5, 2024 via email

edwardhartnett commented Oct 5, 2024

hu5970 commented Oct 11, 2024

EdwardColon-NOAA commented Oct 11, 2024 via email

guoqing-noaa commented Oct 11, 2024

Feature/minmaxtrh #183

Feature/minmaxtrh #183

Conversation

EdwardColon-NOAA commented Dec 29, 2023

guoqing-noaa Dec 31, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

chan-hoo commented Jan 2, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

EdwardColon-NOAA commented Sep 23, 2024 via email

edwardhartnett commented Oct 5, 2024

guoqing-noaa commented Oct 5, 2024

EdwardColon-NOAA commented Oct 5, 2024 via email

EdwardColon-NOAA commented Oct 5, 2024 via email

edwardhartnett commented Oct 5, 2024

hu5970 commented Oct 11, 2024

EdwardColon-NOAA commented Oct 11, 2024 via email

guoqing-noaa commented Oct 11, 2024

guoqing-noaa Dec 31, 2023 •

edited

Loading

chan-hoo commented Jan 2, 2024 •

edited

Loading