From b789c3c1a6613456dfad023273c6b32863f85a3b Mon Sep 17 00:00:00 2001 From: OFIWG Bot Date: Thu, 21 Mar 2024 19:01:05 +0000 Subject: [PATCH] Updated nroff-generated man pages Signed-off-by: OFIWG Bot --- man/man7/fi_cxi.7 | 32 +++++++++++++++++++++++++++++++- 1 file changed, 31 insertions(+), 1 deletion(-) diff --git a/man/man7/fi_cxi.7 b/man/man7/fi_cxi.7 index 14b838824cd..c0ad9d32a0d 100644 --- a/man/man7/fi_cxi.7 +++ b/man/man7/fi_cxi.7 @@ -1,7 +1,7 @@ .\"t .\" Automatically generated by Pandoc 2.9.2.1 .\" -.TH "fi_cxi" "7" "2024\-02\-01" "Libfabric Programmer\[cq]s Manual" "#VERSION#" +.TH "fi_cxi" "7" "2024\-03\-21" "Libfabric Programmer\[cq]s Manual" "#VERSION#" .hy .SH NAME .PP @@ -269,6 +269,19 @@ application processes. .IP "7." 3 Application processes select from the list of available service IDs and VNIs to form an authorization key to use for Endpoint allocation. +.SS Endpoint Protocols +.PP +The provider supports multiple endpoint protocols. +The default protocol is FI_PROTO_CXI and fully supports the messaging +requirements of parallel applicaitons. +.PP +The FI_PROTO_CXI_RNR endpoint protocol is an optional protocol that +targets client/server environments where send-after-send ordering is not +required and messaging is generally to pre-posted buffers; FI_MULTI_RECV +is recommended. +It utilizes a receiver-not-ready implementation where +\f[I]FI_CXI_RNR_MAX_TIMEOUT_US\f[R] can be tuned to control the maximum +retry duration. .SS Address Vectors .PP The CXI provider supports both \f[I]FI_AV_TABLE\f[R] and @@ -514,6 +527,16 @@ Using Pinned mode avoids any overhead due to network page faults but requires all buffers to be backed by physical memory. Copy-on-write semantics are broken when using pinned memory. See the Fork section for more information. +.PP +The CXI provider supports DMABUF for device memory registration. +If the ROCR and CUDA libraries support it, the CXI provider will default +to use DMA-buf. +There may be situations with CUDA that may double the BAR consumption. +Until this is fixed in the CUDA stack, the environment variable +\f[I]FI_CXI_DISABLE_DMABUF_CUDA\f[R] can be used to fall back to the +nvidia peer-memory interface. +Also, \f[I]FI_CXI_DISABLE_DMABUF_ROCR\f[R] can be used to fall back to +the amdgpu peer-memory interface. .SS Translation Cache .PP Mapping a buffer for use by the NIC is an expensive operation. @@ -1300,6 +1323,13 @@ queue becomes empty. Default VNI value used only for service IDs where the VNI is not restricted. .TP +\f[I]FI_CXI_RNR_MAX_TIMEOUT_US\f[R] +When using the endpoint FI_PROTO_CXI_RNR protocol, this setting is used +to control the maximum time from the original posting of the message +that the message should be retried. +A value of 0 will return an error completion on the first RNR ack +status. +.TP \f[I]FI_CXI_EQ_ACK_BATCH_SIZE\f[R] Number of EQ events to process before writing an acknowledgement to HW. Batching ACKs amortizes the cost of event acknowledgement over multiple