Skip to content

Commit

Permalink
Update GH man pages
Browse files Browse the repository at this point in the history
Signed-off-by: OFIWG Bot <[email protected]>
  • Loading branch information
ofiwg-bot authored and github-actions[bot] committed Nov 25, 2024
1 parent 0a562f6 commit 8c66648
Showing 1 changed file with 18 additions and 34 deletions.
52 changes: 18 additions & 34 deletions main/man/fi_cxi.7.md
Original file line number Diff line number Diff line change
Expand Up @@ -380,39 +380,24 @@ increase Request buffer space using the variables *FI_CXI_REQ_\**.

## Message Ordering

The CXI provider supports the following ordering rules:
Supported message ordering: FI_ORDER_SAS, FI_ORDER_WAW, FI_ORDER_RMA_WAW,
FI_ORDER_RMA_RAR, FI_ORDER_ATOMIC_WAW, and FI_ORDER_ATOMIC_RAR.

* All message Send operations are always ordered.
* RMA Writes may be ordered by specifying *FI_ORDER_RMA_WAW*.
* AMOs may be ordered by specifying *FI_ORDER_AMO_{WAW|WAR|RAW|RAR}*.
* RMA Writes may be ordered with respect to AMOs by specifying *FI_ORDER_WAW*.
Fetching AMOs may be used to perform short reads that are ordered with
respect to RMA Writes.
Note: Any FI_ORDER_*_{WAR,RAW} are not supported.

Note: Relaxing the message ordering may result in improved performance.

## Target Ordering

Ordered RMA size limits are set as follows:

* *max_order_waw_size* is -1. RMA Writes and non-fetching AMOs of any size are
ordered with respect to each other.
* *max_order_raw_size* is -1. Fetching AMOs of any size are ordered with
respect to RMA Writes and non-fetching AMOs.
* *max_order_war_size* is -1. RMA Writes and non-fetching AMOs of any size are
ordered with respect to fetching AMOs.

## PCIe Ordering

Generally, PCIe writes are strictly ordered. As an optimization, PCIe TLPs may
have the Relaxed Order (RO) bit set to allow writes to be reordered. Cassini
sets the RO bit in PCIe TLPs when possible. Cassini sets PCIe RO as follows:

* Ordering of messaging operations is established using completion events.
Therefore, all PCIe TLPs related to two-sided message payloads will have RO
set.
* Every PCIe TLP associated with an unordered RMA or AMO operation will have RO
cleared.
* PCIe TLPs associated with the last packet of an ordered RMA or AMO operation
will have RO cleared.
* PCIe TLPs associated with the body packets (all except the last packet of an
operation) of an ordered RMA operation will have RO set.
* *max_order_waw_size* is -1. RMA Writes and AMO writes of any size are ordered with
respect to each other.

Note: Due to FI_ORDER_\*\_{WAR,RAW} not being supported, max_order_{raw,war}_size
are forced to zero.

Note: Relaxing the target ordering may result in improved performance.

## Translation

Expand Down Expand Up @@ -974,6 +959,10 @@ offloading are met.

The CXI provider checks for the following environment variables:

*FI_CXI_MR_TARGET_ORDERING*
: MR target ordering (i.e. PCI ordering). Options: default, strict, or relaxed.
Recommendation is to leave at default behavior.

*FI_CXI_ODP*
: Enables on-demand paging. If disabled, all DMA buffers are pinned.
If enabled and mr_mode bits in the hints exclude FI_MR_ALLOCATED,
Expand Down Expand Up @@ -1533,11 +1522,6 @@ if (ret)
error;
```

When an endpoint does not support FI_FENCE (e.g. optimized MR), a provider
specific transmit flag, FI_CXI_WEAK_FENCE, may be specified on an alias EP
to issue a FENCE operation to create a data ordering point for the alias.
This is supported for one-sided operations only.

Alias EP must be closed prior to closing the original EP.

## PCIe Atomics
Expand Down

0 comments on commit 8c66648

Please sign in to comment.