more efficient SCRIP weights computation

[jcwarner-usgs]

Dear WAVEWATCH III Group-

Recently we have been running WAVEWATCH with nesting (both structured and unstructured). During the init, there is a computation call for SCRIP interpolation weights, calling WW3/model/src/SCRIP/scrip_remap_conservative.f
This can be time consuming for multiple large grids, up to several hours.
In a similar manner, we have been using SCRIP in my COAWST world, and we already had modified that scrip code to run in MPI. I ported the changes to your scrip_remap_conservative.f and now instead of several hours, WAVEWATCH only takes a few minutes to compute the weights. If you are interested in this update i can send you the code. not sure how to work that pull request.
thanks
john warner
jcwarner@usgs.gov

[aliabdolali]

Hi @jcwarner-usgs
this is a great feature we can utilize for our large meshes. could you refer me to a repo or send me the code?
Ali

[jcwarner-usgs]

Ali- can you grab that scrip_remap_conservative from my COAWST repo and give it a try again? it works for me and i am pushing it to WW3 source, would be nice to see if it works for you. thanks.

[aliabdolali]

@jcwarner-usgs I did it once you submitted the PR for regtests/ww3_ufs1.1/input_unstr (I added SCRIP SCRIPNC to the switch). It is running way faster than develop. Thanks for doing this great development.

[MatthewMasarik-NOAA]

Dear WAVEWATCH III Group-

Recently we have been running WAVEWATCH with nesting (both structured and unstructured). During the init, there is a computation call for SCRIP interpolation weights, calling WW3/model/src/SCRIP/scrip_remap_conservative.f This can be time consuming for multiple large grids, up to several hours. In a similar manner, we have been using SCRIP in my COAWST world, and we already had modified that scrip code to run in MPI. I ported the changes to your scrip_remap_conservative.f and now instead of several hours, WAVEWATCH only takes a few minutes to compute the weights. If you are interested in this update i can send you the code. not sure how to work that pull request. thanks john warner jcwarner@usgs.gov

Hi @jcwarner-usgs,
This sounds like a real advancement for init time, and we'd definitely be interested in reviewing the code to see if we can pull it in to the develop branch. Could you let me know how to access your code? We can work with you regarding a pull request.
Best, Matt

[jcwarner-usgs]

Glad this has a chance to help others. It will most likely need some 'beautification' to more smoothly align with the rest of the WW3 code. You can grab the file from here:
https://github.com/DOI-USGS/COAWST/blob/main/WW3/model/src/SCRIP/scrip_remap_conservative.f

I tried to make a switch, but dont think that .f is run through the cpp processor. But if you look for
!#ifdef W3_SCRIP_JCW
that will show you the changes. Better yet, just do a diff to the origin scrip_remap_conservative.f
Let me know if you have any questions or find an issue with it. It seems to be working for us.
thanks
-j

[aliabdolali]

This is all i needed. Thanks

[MatthewMasarik-NOAA]

Thanks @jcwarner-usgs! I've started looking through it. Given the federal holiday tomorrow and some staff on leave, we'll plan to get back in touch with you on Thu or Fri with our next steps.

[jcwarner-usgs]

So i apparently spoke too soon! sorry.
It is failing a test i am running now. Let me dig into it a bit deeper and see what is going on.
The version of SCRIP in WAVEWATCH is different than what i am using, so there might be something i am not exchanging.

[MatthewMasarik-NOAA]

No problem. Thanks for keeping updated.

[jcwarner-usgs]

Ok. I have a version that seems to be working now. It is at the same location.
I turned on SCRIPNC and had both the original code and my code create the nc weights files. the variables were the same in both files, and the solutions i get with the original and my updated methods are the same. So it is working for my test cases now.

[MatthewMasarik-NOAA]

Hi @jcwarner-usgs, thank you for taking the time to resolve the issue with the failing test. It's great to hear you get the same solutions for both now. I'll start reviewing it again, and will touch base with you early next week. Thanks for offering to share your work!

[MatthewMasarik-NOAA]

@jcwarner-usgs, with the shorter holiday week we need a little more time to discuss internally. We'll touch base on Friday.

[jcwarner-usgs]

yes absolutely. take your time.
i have moved on to my next problem, so i am all good.
just want to provide updates back to you as we make progress. happy 4th. -j

[JessicaMeixner-NOAA]

@jcwarner-usgs Can you make a PR with your update? We're always excited to hear about speed improvements.