Skip to content

Commit

Permalink
document vdpa device type
Browse files Browse the repository at this point in the history
Signed-off-by: Adrian Moreno <amorenoz@redhat.com>
  • Loading branch information
amorenoz committed Dec 10, 2020
1 parent 911330a commit 069d2ac
Show file tree
Hide file tree
Showing 4 changed files with 56 additions and 0 deletions.
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -258,6 +258,7 @@ This selector is applicable when "deviceType" is "netDevice"(note: this is defau
| "ddpProfiles" | N | A map of device selectors | `string` list Default: `null` | "ddpProfiles": ["GTPv1-C/U IPv4/IPv6 payload"] |
| "isRdma" | N | Mount RDMA resources | `bool` values `true` or `false` Default: `false` | "isRdma": `true` |
| "needVhostNet"| N | Share /dev/vhost-net | `bool` values `true` or `false` Default: `false` | "needVhostNet": `true` |
| "vdpaType" | N | The type of vDPA device (virtio, vhost or `nil`) | `string` values `vhost` or `virtio` Default: `null` | "vdpaType": "vhost" |


[//]: # (The tables above generated using: https://ozh.github.io/ascii-tables/)
Expand Down
1 change: 1 addition & 0 deletions docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,3 +7,4 @@ This page contains supplimentary documention that users may find useful for vari
* [Running RDMA application in Kubernetes](rdma/)
* [SR-IOV network device plugin with DDP](ddp/)
* [Using node specific config file for running device plugin DaemonSet](config-file)
* [Using vDPA devices in Kuberenets](vdpa/)
24 changes: 24 additions & 0 deletions docs/vdpa/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
# Using vDPA devices in Kuberenets

vDPA-capable devices can be used to expose an accelerated, yet standard interface to pods. These devices implement the virtIO ring layout that is used for the data-plane.
The control plane, however, is translated by the vDPA kernel framework which takes care of configuring the virtio devices and virtqueues.

The vDPA kernel framework is composed of a vdpa bus (/sys/bus/vdpa), vdpa devices (/sys/bus/vdpa/devices) and vdpa drivers (/sys/bus/vdpa/drivers). Currently, two vdpa drivers are implemented:
* virtio_vdpa: Exposes the device as a virtio-net netdev
* virtio_vdpa: Exposes the device as a vhost-vdpa device. This device uses an extension of the vhost-net protocol to allow userspace applications access the rings directly

## Supported NICs:
* Mellanox ConnectX®-6 DX

## vDPA device creation
Each NIC might require diffent steps to create vDPA devices on top of the VFs.

## Bind the desired vdpa driver
The vdpa bus works similar to the pci bus. To unbind a driver from a device, run:
``echo ${DEV_NAME} > /sys/bus/vdpa/devices/${DEV_NAME}/driver/unbind``

To bind a driver to a device, run:
``echo ${DEV_NAME} > /sys/bus/vdpa/drivers/${DRIVER_NAME}/bind``

## Priviledges
IPC_LOCK capability privilege is required for "vhost" mode to be used in a Kubernetes Pod.
30 changes: 30 additions & 0 deletions docs/vdpa/configMap.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
apiVersion: v1
kind: ConfigMap
metadata:
name: sriovdp-config
namespace: kube-system
data:
config.json: |
{
"resourceList": [{
{
"resourceName": "vdpa_mlx_virtio",
"selectors": {
"vendors": ["15b3"],
"devices": ["101e"],
"drivers": ["mlx5_core"],
"vdpaType": "virtio"
}
},
{
"resourceName": "vdpa_mlx_vhost",
"selectors": {
"vendors": ["15b3"],
"devices": ["101e"],
"drivers": ["mlx5_core"],
"vdpaType": "vhost"
}
}
]
}

0 comments on commit 069d2ac

Please sign in to comment.