openEBSctl for NDM

[RealHarshThakur]

For the gRPC client, here's what I think could work:
Each NDM pod will be exposed as a headless service
We deploy a simple REST API as a pod/deployment to interact with the headless services. This REST API will be exposed outside the cluster using kube proxy in client go. Essentially, this REST API will act as a proxy between all the NDM pods and the CLI. Here are some operations REST API would do:

1. List all the node IPs which have NDM installed with gRPC
2. API would take a request with parameter as all, which in turn creates a gRPC request for the operation specified on each NDM API.
3. It could take a particular node IP as input and perform the specified operation on particular NDM API.

cc : @kmova @akhilerm

[akhilerm]

Some approaches that can be explored for accessing the grpc server

• gRPC transcoded within REST and use the kubectl plugin (ndmctl) access ndm API via REST over K8s
• kubectl wrapper for executing commands within ndmctl shipped within ndm-operator
• Explore cloud endpoints - ESP for transcoding http/json <-> grpc

Also, instead of having openebsctl interacting with ndm, a separate component / kubectl plugin ndmctl can be implemented which deals with only NDM related operations. ndmctl can be the plugin which uses the above methods to communicate with the grpc server.

[RealHarshThakur]

Found a video showing how transcoding can be done : https://www.youtube.com/watch?v=NtkTMmKberk

Problem is we're introducing an API gateway to solve this problem.
Also, we would still require a control plane for this approach since we wouldn't want to expose each node's API .
So maybe the control plane is better off being a REST API with OpenAPI spec, so it's easy to create the CLI and expose it with kube-proxy.

[RealHarshThakur]

I just found out that we could store protobuf in etcd . This implies we can actually get kubeproxy to give protobuf messages. I think the reason gRPC proxies are required is to support streaming but as long as we don't have streaming endpoints in our NDM API, we're good. So can we do something similar in our case? cc: @akhilerm @kmova
https://kubernetes.io/docs/reference/using-api/api-concepts/ .

[avishnu]

@RealHarshThakur I believe you are currently using Mayastor. There will be no more enhancements happening on the NDM project.