Skip to content

Commit

Permalink
Merge pull request #37 from Bolodya1997/docs
Browse files Browse the repository at this point in the history
Add documentation
  • Loading branch information
denis-tingaikin authored Jan 18, 2021
2 parents f0e2499 + 9318020 commit e107966
Show file tree
Hide file tree
Showing 2 changed files with 52 additions and 3 deletions.
51 changes: 50 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,52 @@
# Intro

This repo contains 'cmd-forwarder-sriov' a SR-IOV based forwarder application for Network Service Mesh.

This README will provide directions for building, testing, and debugging that container.

# Usage

SR-IOV forwarder binds one of the free VFs from the list of the configured PFs for each client request. It supports 2
mechanisms for the client:
* KERNEL_INTERFACE - binds VF to the kernel driver and passes kernel interface to the client net NS
* VFIO - binds VF to the vfio-pci driver and gives permissions to the client pod to work with the VF device

Also it works as a device plugin server providing resources for the client pods. Each configured PF adds _VF count_
`serviceDomain/capalility` resources for every `{ serviceDomain, capability }` pair.

## Environment config

`cmd-forwarder-sriov` accept following environment variables:

* `NSM_NAME` - A string value of forwarder network service endpoint name (default `sriov-forwarder`)
* `NSM_NS_NAME` - A string value of forwarder network service name (default `sriovns`)
* `NSM_CONNECT_TO` - A Network Service Manager connectTo URL (default `unix:///var/lib/networkservicemesh/nsm.io.sock`)
* `NSM_MAX_TOKEN_LIFETIME` - A token lifetime duration (default `24h`)
* `NSM_RESOURCE_POLL_TIMEOUT` - A timeout to poll device plugin resources usage from kubelet API (default `30s`)
* `NSM_DEVICE_PLUGIN_PATH` - Path to the device plugin directory (default `/var/lib/kubelet/device-plugins/`)
* `NSM_POD_RESOURCES_PATH` - Path to the pod resources directory (default `/var/lib/kubelet/pod-resources/`)
* `NSM_SRIOV_CONFIG_FILE` - Path to the config file (default `./pci.config`)
* `NSM_PCI_DEVICES_PATH` - Path to the PCI devices directory (default `/sys/bus/pci/devices`)
* `NSM_PCI_DRIVERS_PATH` - Path to the PCI drivers directory (default `/sys/bus/pci/drivers`)
* `NSM_CGROUP_PATH` - Path to the host `/sys/fs/cgroup/devices` directory (default `/host/sys/fs/cgroup/devices`)
* `NSM_VFIO_PATH` - Path to the host `/dev/vfio` directory (default `/host/dev/vfio`)

## Config file

`cmd-forwarder-sriov` requires configuration file in the following format:
```yaml
---
physicalFunctions: # map [PCI address -> PF] for all supported PFs
0000:01:00.0:
pfKernelDriver: pf-driver # PF kernel driver
vfKernelDriver: vf-driver # VFs kernel driver
capabilities: # list of the capabilities supported by the PF
- intel
- 10G
serviceDomains: # list of the service domains supported by the PF
- service.domain.1
```
# Build
## Build cmd binary locally
Expand Down Expand Up @@ -76,7 +125,7 @@ through to the point of running cmd, you will not be able to attach a debugger o

## A Note on Running golangci-lint

Because cmd-forwarder-vppagent is only anticipated to run in Linux, you will need to run golangci-lint run with:
Because cmd-forwarder-sriov is only anticipated to run in Linux, you will need to run golangci-lint run with:

```go
GOOS=linux golangci-lint run
Expand Down
4 changes: 2 additions & 2 deletions main.go
Original file line number Diff line number Diff line change
Expand Up @@ -61,9 +61,9 @@ import (

// Config - configuration for cmd-forwarder-sriov
type Config struct {
Name string `default:"sriov-forwarder" desc:"Name of Endpoint"`
Name string `default:"sriov-forwarder" desc:"name of Endpoint"`
NSName string `default:"sriovns" desc:"Name of Network Service to Register with Registry"`
ConnectTo url.URL `default:"unix:///connect.to.socket" desc:"url to connect to" split_words:"true"`
ConnectTo url.URL `default:"unix:///var/lib/networkservicemesh/nsm.io.sock" desc:"URL to connect to" split_words:"true"`
MaxTokenLifetime time.Duration `default:"24h" desc:"maximum lifetime of tokens" split_words:"true"`
ResourcePollTimeout time.Duration `default:"30s" desc:"device plugin polling timeout" split_words:"true"`
DevicePluginPath string `default:"/var/lib/kubelet/device-plugins/" desc:"path to the device plugin directory" split_words:"true"`
Expand Down

0 comments on commit e107966

Please sign in to comment.