tridentctl fails when deleting k8s objects

[bram-bezem]

I am unable to install trident using tridentctl due to what seems like a bug in the k8s_client. Can you please advise how I can get this working?

Command:
./tridentctl install -n trident

Actual output:
INFO Starting storage driver. backend=/home/user/trident-installer/setup/backend.json
WARN Could not determine controller serial numbers. API status: failed, Reason: Unable to find API: system-node-get-iter, Code: 13005
INFO Storage driver loaded. driver=ontap-nas
INFO Starting Trident installation. namespace=trident
WARN Could not delete cluster role binding using existing YAML file. error="exit status 1" path=/home/user/trident-installer/setup/trident-clusterrolebinding.yaml
WARN Could not delete cluster role binding. error="exit status 1"
WARN Could not delete cluster role using existing YAML file. error="exit status 1" path=/home/user/trident-installer/setup/trident-clusterrole.yaml
WARN Could not delete cluster role. error="exit status 1"
WARN Could not delete service account using existing YAML file. error="exit status 1" path=/home/user/trident-installer/setup/trident-serviceaccount.yaml
WARN Could not delete service account. error="exit status 1"
FATA Install failed; could not remove one or more previous Trident artifacts; please delete them manually and try again

Expected output:
INFO Starting storage driver. backend=setup/backend.json
INFO Storage driver loaded. driver=ontap-nas
INFO Starting Trident installation. namespace=trident
INFO Created service account.
INFO Created cluster role.
INFO Created cluster role binding.
INFO Created PVC.
INFO Created PV. pv=trident
INFO Waiting for PVC to be bound. pvc=trident
INFO Created Trident deployment.
INFO Waiting for Trident pod to start.
INFO Trident pod started. namespace=trident pod=trident-7d5d659bd7-tzth6
INFO Trident installation succeeded.

Cause:
Using strace I have discovered the cause to be the order of the arguments provided to the kubectl delete command

strace -f ./tridentctl install -n trident -d &> strace.log

[pid 24174] execve("/usr/local/bin/kubectl", ["kubectl", "--namespace=trident", "--ignore-not-found=true", "delete", "-f", "/home/user/trident-installer/s"...], [/* 31 vars */] <unfinished ...>
...
[pid 24174] write(2, "unknown flag: --ignore-not-found"..., 33 <unfinished ...>
...
[pid 24174] +++ exited with 1 +++

According to kubectl docs the --ignore-not-found flag must appear after the delete. The following command for example runs without issues
kubectl --namespace=trident delete -f /home/user/trident-installer/setup/trident-clusterrolebinding.yaml --ignore-not-found=true

The offending lines seem to be 655, 674 and 693 in https://github.com/NetApp/trident/blob/stable/v18.04/cli/k8s_client/k8s_cli_client.go

Versions:
trident: netapp/trident:18.04
tridentctl trident-installer-18.04.0.tar.gz
kubectl v1.9.6

[clintonk]

@brambezem Thanks for this analysis! I can't explain or reproduce what you are seeing, at least with K8S 1.10.1, and we do test with K8S 1.9 as well. But reordering the kubectl arguments seems trivial and harmless, so I'll try to get that into the 18.07 release.

[depatl]

Is there any solution for this?

[bram-bezem]

My workaround was to upgrade my cluster to kubernetes v1.10.4.
If you do not wish to do so you could try to only upgrade kubectl only as this is a client-side issue.

[depatl]

upgraded kubectl client it worked. Thanks

[clintonk]

Fixed in v18.07.0-beta.1