Skip to content

Commit

Permalink
docs: document vdpa device type
Browse files Browse the repository at this point in the history
Signed-off-by: Adrian Moreno <amorenoz@redhat.com>
  • Loading branch information
amorenoz committed Jan 7, 2021
1 parent d6fb728 commit 581c03c
Show file tree
Hide file tree
Showing 4 changed files with 66 additions and 1 deletion.
12 changes: 11 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -166,7 +166,7 @@ For example manifest objects refer to [SR-IOV demo](https://github.com/nokia/dan

### Config parameters

This plugin creates device plugin endpoints based on the configurations given in the config map associated with the SR-IOV device plugin. In json format this file appears as shown below:
This plugin creates device plugin endpoints based on the configurations given in the config map associated with the SR-IOV device plugin. In json format this file appears as shown below:

```json
{
Expand Down Expand Up @@ -206,6 +206,15 @@ This plugin creates device plugin endpoints based on the configurations given in
"isRdma": true
}
},
{
"resourceName": "ct6dx_vdpa_vhost",
"selectors": {
"vendors": ["15b3"],
"devices": ["101e"],
"drivers": ["mlx5_core"],
"vdpaType": "vhost"
}
},
{
"resourceName": "intel_fpga",
"deviceType": "accelerator",
Expand Down Expand Up @@ -258,6 +267,7 @@ This selector is applicable when "deviceType" is "netDevice"(note: this is defau
| "ddpProfiles" | N | A map of device selectors | `string` list Default: `null` | "ddpProfiles": ["GTPv1-C/U IPv4/IPv6 payload"] |
| "isRdma" | N | Mount RDMA resources | `bool` values `true` or `false` Default: `false` | "isRdma": `true` |
| "needVhostNet"| N | Share /dev/vhost-net | `bool` values `true` or `false` Default: `false` | "needVhostNet": `true` |
| "vdpaType" | N | The type of vDPA device (virtio, vhost or `nil`) | `string` values `vhost` or `virtio` Default: `null` | "vdpaType": "vhost" |


[//]: # (The tables above generated using: https://ozh.github.io/ascii-tables/)
Expand Down
1 change: 1 addition & 0 deletions docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,3 +7,4 @@ This page contains supplimentary documention that users may find useful for vari
* [Running RDMA application in Kubernetes](rdma/)
* [SR-IOV network device plugin with DDP](ddp/)
* [Using node specific config file for running device plugin DaemonSet](config-file)
* [Using vDPA devices in Kuberenets](vdpa/)
24 changes: 24 additions & 0 deletions docs/vdpa/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
# Using vDPA devices in Kubernetes

vDPA-capable devices can be used to expose an accelerated, yet standard interface to pods. These devices implement the virtIO ring layout that is used for the data-plane.
The control plane, however, is translated by the vDPA kernel framework which takes care of configuring the virtio devices and virtqueues.

The vDPA kernel framework is composed of a vdpa bus (/sys/bus/vdpa), vdpa devices (/sys/bus/vdpa/devices) and vdpa drivers (/sys/bus/vdpa/drivers). Currently, two vdpa drivers are implemented:
* virtio_vdpa: Exposes the device as a virtio-net netdev
* vhost_vdpa: Exposes the device as a vhost-vdpa device. This device uses an extension of the vhost-net protocol to allow userspace applications access the rings directly

## Supported NICs:
* Mellanox ConnectX®-6 DX

## vDPA device creation
Each NIC might require different steps to create vDPA devices on top of the VFs.

## Bind the desired vdpa driver
The vdpa bus works similar to the pci bus. To unbind a driver from a device, run:
``echo ${DEV_NAME} > /sys/bus/vdpa/devices/${DEV_NAME}/driver/unbind``

To bind a driver to a device, run:
``echo ${DEV_NAME} > /sys/bus/vdpa/drivers/${DRIVER_NAME}/bind``

## Priviledges
IPC_LOCK capability privilege is required for "vhost" mode to be used in a Kubernetes Pod.
30 changes: 30 additions & 0 deletions docs/vdpa/configMap.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
apiVersion: v1
kind: ConfigMap
metadata:
name: sriovdp-config
namespace: kube-system
data:
config.json: |
{
"resourceList": [{
{
"resourceName": "vdpa_mlx_virtio",
"selectors": {
"vendors": ["15b3"],
"devices": ["101e"],
"drivers": ["mlx5_core"],
"vdpaType": "virtio"
}
},
{
"resourceName": "vdpa_mlx_vhost",
"selectors": {
"vendors": ["15b3"],
"devices": ["101e"],
"drivers": ["mlx5_core"],
"vdpaType": "vhost"
}
}
]
}

0 comments on commit 581c03c

Please sign in to comment.