Cannot check out a specific tag because submodule commit is invalid

[rickgrubin]

Steps to reproduce:

$ git clone --recursive https://github.com/ufs-community/UFS_UTILS.git
Cloning into 'UFS_UTILS'...
[...]
Submodule 'ccpp' (https://github.com/ufs-community/ccpp-physics.git) registered for path 'ccpp-physics'
Cloning into '/path/to/UFS_UTILS/ccpp-physics'...
[...]
Submodule path 'ccpp-physics': checked out '3a306a493a9a0b6c3c39c7b50d356f0ddb7c5c94'
Submodule 'physics/rte-rrtmgp' (https://github.com/earth-system-radiation/rte-rrtmgp) registered for path 'ccpp-physics/physics/rte-rrtmgp'
Cloning into '/path/to/UFS_UTILS/ccpp-physics/physics/rte-rrtmgp'...
[...]
Submodule path 'ccpp-physics/physics/rte-rrtmgp': checked out '0dc54f5ecaeb1e1e342efd1e02d0bcd41737bde2'
$ cd UFS_UTILS/
$ git fetch --all --tags
$ git checkout tags/ufs_utils_1_8_0 -b ufs_utils_1_8_0
fatal: failed to unpack tree object cec1e8e12d969c3c8c76574dbe4f40b366419cc7
error: Submodule 'physics/rte-rrtmgp' could not be updated.
error: Submodule 'ccpp-physics/physics/rte-rrtmgp' cannot checkout new HEAD.
error: Submodule 'ccpp-physics' could not be updated.
error: Cannot update submodule:
	ccpp-physics

Aborting

Similar result for other tags, e.g. tag ufs_utils_1_9_0as well.

It seems that this is because the particular commit (cec1e8e12d969c3c8c76574dbe4f40b366419cc7) for submodule rte-rrtmgp cannot be resolved; this renders tags for UFS_UTILS incomplete. Recognizing that te-rrtmgp isn't your code, this impacts your repository.

Whether attempting to get a release manually or via automated tools (e.g. spack) appears broken -- an identical message appears in spack logs.

Or am I doing it wrong?

[GeorgeGayno-NOAA]

What version of 'git' are you using? Older versions caused problems. A fix was recently made in #826 and #827.

[GeorgeGayno-NOAA]

As a workaround, can you do the clone without the --recursive. Then do git submodule init and git submodule update.

[rickgrubin]

$ git --version
git version 2.27.1

$ git clone https://github.com/ufs-community/UFS_UTILS.git
Cloning into 'UFS_UTILS'...
[...]
Resolving deltas: 100% (32430/32430), done.

$ cd UFS_UTILS/
$ git submodule init
Submodule 'ccpp' (https://github.com/ufs-community/ccpp-physics.git) registered for path 'ccpp-physics'
$ git submodule update
Cloning into '/path/to/UFS_UTILS/ccpp-physics'...
Submodule path 'ccpp-physics': checked out '3a306a493a9a0b6c3c39c7b50d356f0ddb7c5c94'

$ ls -FCa ccpp-physics/physics/rte-rrtmgp/
./  ../

In this case, submodule rte-rrtmgp isn't populated (as expected) which, as I read #826, which appears to be OK if I utilize the global workflow script checkout.sh -- which I'm not, or perhaps if I use build_all.sh -- however:

Executing build_all.sh isn't an option on hosts other than those defined as NOAA Tier 1 hosts, as that script will invoke sorc/machine-setup.sh, leading to:

$ ./build_all.sh 
++ uname -s
+ [[ Linux == Darwin ]]
++++ readlink -f -n ./build_all.sh
+++ dirname /path/to/UFS_UTILS/build_all.sh
++ cd /path/to/UFS_UTILS
++ pwd -P
+ readonly DIR_ROOT=/path/to/UFS_UTILS
+ DIR_ROOT=/path/to/UFS_UTILS
+ target=NULL
+ compiler=intel
+ PW_CSP=
+ [[ NULL == \l\i\n\u\x\.\* ]]
+ [[ NULL == \m\a\c\o\s\x\.\* ]]
+ set +x
WARNING: UNKNOWN PLATFORM
Lmod has detected the following error:  The following module(s) are unknown: "build..intel"

because there does not exist UFS_UTILS/modulefiles/build.<myhost>.intel.lua as per the Lmod error noted above.

I see the change noted in #827 and that works as expected for the develop branch -- the failure noted in this issue remains when checking out a tag (e.g. ufs_utils_1_8_0) as the rte-rrtmgp commit points to a non-existent commit or (probably) branch.

Regardless, I don't think that any of this helps with respect to automated builds (spack) as it's necessary to specify getting submodules when building UFS_UTILS since ccpp-physics is required, and spack's behavior is to recurse for all submodules. Given that, do I understand correctly that if I can manage to non-recursively populate ccpp-physics -- that is, not also populate physics/rte-rrtmgp -- I can expect a build to succeed?

[GeorgeGayno-NOAA]

UFS_UTILS uses nothing from the physics/rte-rrtmgp. It uses only one file from that repository - ./physics/sfcsub.F

[rickgrubin]

UFS_UTILS uses nothing from the physics/rte-rrtmgp. It uses only one file from that repository - ./physics/sfcsub.F

Please educate me: If UFS_UTILS uses only ./physics/sfcsub.F, why is all the rest there?