From d731157564eeae50b50c59be28691304c382e689 Mon Sep 17 00:00:00 2001 From: Alexia Ingerson Date: Wed, 14 Aug 2024 10:27:37 -0700 Subject: [PATCH] include/rdma: add FI_PEER as a capability The FI_PEER bit is already defined as a flag for the calls to import shared resources into peers. Extend this flag to be defined as a primary capability which indicates that the provider supports being used as a peer through the peer API flow. The subset of the peer API defined for the FI_PEER capability is the minimum needed for peer support: CQ, counter, and shared receive queues Signed-off-by: Alexia Ingerson --- include/ofi.h | 2 +- man/fi_domain.3.md | 5 +++++ man/fi_getinfo.3.md | 8 +++++++- src/fi_tostr.c | 1 + 4 files changed, 14 insertions(+), 2 deletions(-) diff --git a/include/ofi.h b/include/ofi.h index 674f3b13928..7592281c766 100644 --- a/include/ofi.h +++ b/include/ofi.h @@ -161,7 +161,7 @@ ofi_poll_del(struct fid_poll *pollset, struct fid *event_fid, uint64_t flags) (FI_MULTI_RECV | FI_TRIGGER | FI_RMA_PMEM | FI_SOURCE | \ FI_RMA_EVENT | FI_SOURCE_ERR) -#define OFI_DOMAIN_PRIMARY_CAPS FI_AV_USER_ID +#define OFI_DOMAIN_PRIMARY_CAPS (FI_AV_USER_ID | FI_PEER) #define OFI_DOMAIN_SECONDARY_CAPS \ (FI_SHARED_AV | FI_REMOTE_COMM | FI_LOCAL_COMM) diff --git a/man/fi_domain.3.md b/man/fi_domain.3.md index 86277052d07..75509483e80 100644 --- a/man/fi_domain.3.md +++ b/man/fi_domain.3.md @@ -719,6 +719,11 @@ The following are support primary capabilities: FI_AV_USER_ID can still be supported through the AV insert calls without this domain capability set. See [`fi_av`(3)](fi_av.3.html). +*FI_PEER* +: Specifies that the domain must support importing resources to be used in the + the peer API flow. The domain must support importing owner_ops when opening + a CQ, counter, and shared receive queue. + The following are supported secondary capabilities: *FI_LOCAL_COMM* diff --git a/man/fi_getinfo.3.md b/man/fi_getinfo.3.md index 347123415f1..6219792257e 100644 --- a/man/fi_getinfo.3.md +++ b/man/fi_getinfo.3.md @@ -338,6 +338,11 @@ additional optimizations. allow an initiator to target (or name) a specific receive context as part of a data transfer operation. +*FI_PEER* +: Specifies that the provider must support being used as a peer provider in + the peer API flow. The provider must support importing owner_ops when opening + a CQ, counter, and shared receive queue. + *FI_READ* : Indicates that the user requires an endpoint capable of initiating reads against remote memory regions. This flag requires that FI_RMA @@ -457,7 +462,8 @@ may optionally report non-selected secondary capabilities if doing so would not compromise performance or security. Primary capabilities: FI_MSG, FI_RMA, FI_TAGGED, FI_ATOMIC, FI_MULTICAST, -FI_NAMED_RX_CTX, FI_DIRECTED_RECV, FI_HMEM, FI_COLLECTIVE, FI_XPU, FI_AV_USER_ID +FI_NAMED_RX_CTX, FI_DIRECTED_RECV, FI_HMEM, FI_COLLECTIVE, FI_XPU, +FI_AV_USER_ID, FI_PEER Primary modifiers: FI_READ, FI_WRITE, FI_RECV, FI_SEND, FI_REMOTE_READ, FI_REMOTE_WRITE diff --git a/src/fi_tostr.c b/src/fi_tostr.c index 8f47dda108c..3f0eb38d0cb 100644 --- a/src/fi_tostr.c +++ b/src/fi_tostr.c @@ -212,6 +212,7 @@ static void ofi_tostr_caps(char *buf, size_t len, uint64_t caps) IFFLAGSTRN(caps, FI_NAMED_RX_CTX, len); IFFLAGSTRN(caps, FI_DIRECTED_RECV, len); IFFLAGSTRN(caps, FI_AV_USER_ID, len); + IFFLAGSTRN(caps, FI_PEER, len); IFFLAGSTRN(caps, FI_HMEM, len); ofi_remove_comma(buf);