igroup name change with ontap-san driver breaks existing volumes

[ghouscht]

NetApp support case 2008427384

Describe the bug
We have multiple k8s clusters which provide persistent storage through trident (nfs & iSCSI). Historically all those clusters used the same igroup name trident for iSCSI. Since trident version 20.07.0 is now able to mange igroups automatically we had to change the irgoup name prior to updating trident in order to avoid clashes between the different clusters.
Unfortunately changing the igroup name breaks existing volumes. The igroup name is not updated in the tridentvolumes CRD in k8s. Which means pods were no longer able to mount iSCSI PVCs even if the initators were added manually to the new igroup. Manually changing the igroup name in the tridentvolumes CRD is possible, but then we also observed that the lun id on the NetApp SVM changed, so we had to manually update the lun id too. After this pods were again able to mount the iSCSI PVCs.
I was not able to find anything related in the documentation regarding wether an igroup name change is supported by trident or not, that's why I consider this a bug.

Environment

• Trident version: v20.04.0
• Trident installation flags used: tridentctl -n kube-trident
• Container runtime:

Client:
 Version:           18.09.8
 API version:       1.39
 Go version:        go1.10.8
 Git commit:        0dd43dd87f
 Built:             Wed Jul 17 17:40:31 2019
 OS/Arch:           linux/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          18.09.8
  API version:      1.39 (minimum version 1.12)
  Go version:       go1.10.8
  Git commit:       0dd43dd
  Built:            Wed Jul 17 17:10:42 2019
  OS/Arch:          linux/amd64
  Experimental:     false

• Kubernetes version:

Client Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.2", GitCommit:"52c56ce7a8272c798dbc29846288d7cd9fbae032", GitTreeState:"clean", BuildDate:"2020-04-16T11:56:40Z", GoVersion:"go1.13.9", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.6", GitCommit:"dff82dc0de47299ab66c83c626e08b245ab19037", GitTreeState:"clean", BuildDate:"2020-07-15T16:51:04Z", GoVersion:"go1.13.9", Compiler:"gc", Platform:"linux/amd64"}

• Kubernetes orchestrator: -
• Kubernetes enabled feature gates: -
• OS: Red Hat Enterprise Linux Server release 7.8 (Maipo)
• NetApp backend types: ontap
• Other: -

To Reproduce

1. Create a PVC in k8s which yields a storage class which is backed by an ontap-san driver.
2. Change the iscsi igroup name in trident backend configuration to a new value.
3. Observe that the igroup name change is not reflected to previously created trident volumes.
4. Pods are now no longer able to mount iSCSI PVCs.

Expected behavior
Pods are still able to mount iSCSI PVCs after the igroup name change, without a manual intervention.

Additional context
We had to change the iSCSI igroup name prior to updating to trident version 20.07.0 in order to avoid conflicts with other trident instances running in different clusters, which were setup to use the same igroup name.

[gnarl]

Hi @ghouscht,

We will update the documentation to address the ramifications of changing the igroup name.

[gnarl]

We have made a change in Trident that uses a unique igroup name for each Trident install so that customers can avoid this issue in the future. This change will be available in the Trident 21.04 release.