Fix leader election in deployment #428
Conversation
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
pvc and app added backend image validation for rbd Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
|
@humblec @ShyamsundarR PTAL |
|
Ci is failing with this PR haven't changes anything related to this |
| @@ -30,7 +30,8 @@ spec: | |||
| image: "{{ .Values.provisioner.image.repository }}:{{ .Values.provisioner.image.tag }}" | |||
| args: | |||
| - "--csi-address=$(ADDRESS)" | |||
| - "--leader-election-type=leases" | |||
| - "--enable-leader-election=true" | |||
| - "--leader-election-type=endpoints" | |||
There was a problem hiding this comment.
Why are we moving away from leases? Kubernetes CSI seems to be suggesting that this option itself would be deprecated in the future as the election mechanism would default to leases, so why are we starting with endpoints and what are the repercussions.
There was a problem hiding this comment.
@ShyamsundarR external-provisoner is failing with leases kubernetes-csi/external-provisioner#291
There was a problem hiding this comment.
endpoints seem to have a request flooding problem, as noted here. In consideration of this should we even proceed with endpoints or move back to Stateful sets, or should we wait for a fix/ack for the Kubernetes issue?
There was a problem hiding this comment.
seems like leases works with kube 1.14+ version. do we need to switch back to statefulset to support older kubernetes version or will use endpoint as lease?
one question needs to answer here is how much systems will be affected with throttling enabled due to the endpoint?
There was a problem hiding this comment.
CI passed with lease and kube 1.14.2 https://travis-ci.org/ceph/ceph-csi/jobs/545169705
There was a problem hiding this comment.
seems like
leasesworks withkube 1.14+version. do we need to switch back to statefulset to support older kubernetes version or will use endpoint as lease?
I think it is too early to drop v1.13 support, so either have 2 deployments (one for each version), or switch back to Stateful sets (not optimal for future requirements of upgrade etc.).
one question needs to answer here is how much systems will be affected with throttling enabled due to the endpoint?
This issue and its related discussion around 100PVCs seems to suggest this could happen at lower threasholds, I would hence be wary of using the endpoint leadership election mechanism till we know better.
I would say have different deployments for different Kubernetes versions, and hence use leases only for v1.14 and Stateful sets for v1.13 is a solution that we should go for.
Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
Other PRs are also failing here: see #427 in travis job https://travis-ci.org/ceph/ceph-csi/jobs/545156584 |
Fixed here #429 |
|
Need to enable leader election in this deploment |
|
sent a PR #432 to revert back to statefulset will revisit PR once we sort out things to use deployment |
|
will be fixed by #497 |
Syncing latest changes from upstream devel for ceph-csi
Describe what this PR does
Is there anything that requires special attention
Related issues
Fixes: #426