Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add ddp selector #199

Merged
merged 4 commits into from
Feb 12, 2020

Conversation

ahalimx86
Copy link
Collaborator

Introducing new selector that can filter devices based on the DDP profile a SR-IOV NIC is running with.

For more information on DDP please see here.

filteredDevice = selector.Filter(filteredDevice)
}
}

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like LinkTypes selector is missed here.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch! Will fix this.

docs/ddp/README.md Outdated Show resolved Hide resolved
docs/ddp/README.md Outdated Show resolved Hide resolved


# SR-IOV network device plugin with DDP
Dynamic Device Personalization aka DDP allows dynamic reconfiguration of the packet processing pipeline of Intel Ethernet 700 Series to meet specific use case needs on demand, adding new packet processing pipeline configuration *profiles* to a network adapter at run time, without resetting or rebooting the server.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it also possible to use dynamically loaded profiles without resetting number of VFs?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it is possible, since the profile is added to the PF. And for x700 you do not need to reload the driver to load a new profile. So, existing VFs are unaffected.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In this case, will existing VFs be able to use the dynamically loaded new profile?
btw, if VFs from one PF are using different dirvers (e.g. native kernel driver or vfio-pci driver), then the profile can be used by both types of VF devices, right?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The DDP profile is firmware on the adapter itself. So, any profile that is loaded into the card that is applicable for all VF exist on that card, regardless of VF drivers.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, VF device will not be aware of the change in firmware, so if ddp profile is re-loaded, then SR-IOV device plugin will also need to be restarted in order to detect new profile names.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's right.

## 2. Load a DDP profile in to the NIC

Use Linux `ethtool` utility to load a DDP profile into the controller.
> Note: You can only load DDP profile into a controller using only first Physical Function(PF0).
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Assume a network adaptor has two ports, does this mean only port 0 can be used to load DDP profile, port 1 cannot ?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's correct. It's to do with the region of the memory where the profile is being loaded is offset from PF0.

```

## 8. Deploy workloads
Once we can verify that VFs are registered by the device plugin under correct resource pool with specific DDP profile, we can request those VFs from our workload as normal.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does the workload needs to know which DDP profile is loaded?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, workloads just request VFs from a resourcePool they need as normal. We this selector we are adding capability to group VFs by certain DDP profiles when we need that.


(ref: [Dynamic Device Personalization for Intel® Ethernet 700 Series](https://software.intel.com/en-us/articles/dynamic-device-personalization-for-intel-ethernet-700-series))

The SR-IOV network device plugin could be used to identify currently running DDP *profiles*, thus allow it to filter Virtual Functions by their DDP profile names.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: "running DDP profiles, thus allow" >>> "running DDP profiles, allowing"

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"The SR-IOV network device plugin could be used" >>> The SR-IOV network device plugin can be used

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed! Thanks

@killianmuldoon
Copy link
Collaborator

/lgtm

1 similar comment
@zshi-redhat
Copy link
Collaborator

/lgtm


FROM alpine
RUN apk add hwdata-pci
COPY --from=builder /usr/src/sriov-network-device-plugin/build/sriovdp /usr/bin/
COPY --from=builder /tmp/ddptool/ddptool /usr/bin/
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The ddptool binary is about 40K size in my local build, seems not a big addition to the image size.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a very small utility tool. For any updates we just need to update the dockerfile url

Change-Id: Ibbfcf92c1d2b551211b48bbdaa222acaec79e3ef
Change-Id: I987ad32e3aa9f9dbbcb874d87a981cf0e6a61456
Change-Id: I282358ef2fc4fec3c6431504d7c311191f8850a6
Change-Id: Ia7fd5b45fbee3058c88eb19f1b2d2fbfb634e162
@ahalimx86 ahalimx86 merged commit 30e33f1 into k8snetworkplumbingwg:master Feb 12, 2020
@ahalimx86 ahalimx86 deleted the dev/fvl_ddp_selector branch March 12, 2020 11:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants