include/rdma: add FI_PEER as a capability

The FI_PEER bit is already defined as a flag for the calls to import
shared resources into peers. Extend this flag to be defined as a
primary capability which indicates that the provider supports being
used as a peer through the peer API flow. The subset of the peer API
defined for the FI_PEER capability is the minimum needed for peer support:
CQ, counter, and shared receive queues

Signed-off-by: Alexia Ingerson <[email protected]>

include/ofi.h

@@ -161,7 +161,7 @@ ofi_poll_del(struct fid_poll *pollset, struct fid *event_fid, uint64_t flags)
 	(FI_MULTI_RECV | FI_TRIGGER | FI_RMA_PMEM | FI_SOURCE | \
 	 FI_RMA_EVENT | FI_SOURCE_ERR)
 
-#define OFI_DOMAIN_PRIMARY_CAPS FI_AV_USER_ID
+#define OFI_DOMAIN_PRIMARY_CAPS (FI_AV_USER_ID | FI_PEER)
 #define OFI_DOMAIN_SECONDARY_CAPS \
 	(FI_SHARED_AV | FI_REMOTE_COMM | FI_LOCAL_COMM)
 

man/fi_domain.3.md

@@ -719,6 +719,11 @@ The following are support primary capabilities:
   FI_AV_USER_ID can still be supported through the AV insert calls without
   this domain capability set. See [`fi_av`(3)](fi_av.3.html).
 
+*FI_PEER*
+: Specifies that the domain must support importing resources to be used in the
+  the peer API flow. The domain must support importing owner_ops when opening
+  a CQ, counter, and shared receive queue.
+
 The following are supported secondary capabilities:
 
 *FI_LOCAL_COMM*

man/fi_getinfo.3.md

@@ -338,6 +338,11 @@ additional optimizations.
   allow an initiator to target (or name) a specific receive context as
   part of a data transfer operation.
 
+*FI_PEER*
+: Specifies that the provider must support being used as a peer provider in
+  the peer API flow. The provider must support importing owner_ops when opening
+  a CQ, counter, and shared receive queue.
+
 *FI_READ*
 : Indicates that the user requires an endpoint capable of initiating
   reads against remote memory regions.  This flag requires that FI_RMA
@@ -457,7 +462,8 @@ may optionally report non-selected secondary capabilities if doing so
 would not compromise performance or security.
 
 Primary capabilities: FI_MSG, FI_RMA, FI_TAGGED, FI_ATOMIC, FI_MULTICAST,
-FI_NAMED_RX_CTX, FI_DIRECTED_RECV, FI_HMEM, FI_COLLECTIVE, FI_XPU, FI_AV_USER_ID
+FI_NAMED_RX_CTX, FI_DIRECTED_RECV, FI_HMEM, FI_COLLECTIVE, FI_XPU,
+FI_AV_USER_ID, FI_PEER
 
 Primary modifiers: FI_READ, FI_WRITE, FI_RECV, FI_SEND,
 FI_REMOTE_READ, FI_REMOTE_WRITE

src/fi_tostr.c

@@ -212,6 +212,7 @@ static void ofi_tostr_caps(char *buf, size_t len, uint64_t caps)
 	IFFLAGSTRN(caps, FI_NAMED_RX_CTX, len);
 	IFFLAGSTRN(caps, FI_DIRECTED_RECV, len);
 	IFFLAGSTRN(caps, FI_AV_USER_ID, len);
+	IFFLAGSTRN(caps, FI_PEER, len);
 	IFFLAGSTRN(caps, FI_HMEM, len);
 
 	ofi_remove_comma(buf);