Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement GetFenceClients to return the ceph clusterID and the client Address #4944

Open
wants to merge 6 commits into
base: devel
Choose a base branch
from

Conversation

Madhu-1
Copy link
Collaborator

@Madhu-1 Madhu-1 commented Nov 5, 2024

This PR implements a csiaddons RPC calls to return the ceph fsID and the client IP that is used to connect to the ceph cluster from the rbd plugin daemonset pod.

closes: #4913

updating csi-addons spec to the
latest main which GetFenceClients
API.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
register Capability_NetworkFence_
GET_CLIENTS_TO_FENCE capability and
start a NetworkFence controllers
as part of rbd nodeplugin.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
updating go-ceph to the latest commit to
pull the new function to get client address

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
added GetAddrs to get the client
Adress of the rados connection
which is helpful for NetworkFencing

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
implemented GetFenceClients which
connects to the ceph cluster and
returns the ceph clusterID and the
clientaddress that is used for rados
connection.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
@Madhu-1
Copy link
Collaborator Author

Madhu-1 commented Nov 5, 2024

Test result

sh-5.1# csi-client 
Enter the path for the Unix socket: unix:///csi/csi-addons.sock
Using socket path: unix:///csi/csi-addons.sock
Server response: id:"a815fe8e-eabd-4e87-a6e8-78cebfb67d08"  addresses:{cidr:"10.244.0.1"}
sh-5.1# cat /sys/devices/rbd/0/
block/        client_id     config_info   features      major         name          pool          pool_ns       refresh       snap_id       uevent        
client_addr   cluster_fsid  current_snap  image_id      minor         parent        pool_id       power/        size          subsystem/    
sh-5.1# cat /sys/devices/rbd/0/client_addr 
10.244.0.1:0/353261872
sh-5.1# cat /sys/devices/rbd/0/cluster_fsid 
a815fe8e-eabd-4e87-a6e8-78cebfb67d08
sh-5.1# 

wrote a small client to get the details, this is how it looks like

@nixpanic
Copy link
Member

nixpanic commented Nov 5, 2024

cidr:"10.244.0.1" is not in correct CIDR notation, it is a plain IP-address. Maybe call it differently, or make it cidr:"10.244.0.1/32".

The address we get from ceph
contains the ip in the format
of 10.244.0.1:0/2686266785 we
need to extract the client IP
from this address, we already
have a helper to extract it,
This makes the helper more generic
can be reused by multiple packages
in the fence controller.

Signed-off-by: Madhu Rajanna <madhupr007@gmail.com>
@Madhu-1
Copy link
Collaborator Author

Madhu-1 commented Nov 5, 2024

cidr:"10.244.0.1" is not in correct CIDR notation, it is a plain IP-address. Maybe call it differently, or make it cidr:"10.244.0.1/32".

added /32 to make it CIDR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add annotations with ceph client address
2 participants