svcrdma: Allocate recv_ctxt's on CPU handling Receives
authorChuck Lever <chuck.lever@oracle.com>
Mon, 7 May 2018 19:27:48 +0000 (15:27 -0400)
committerJ. Bruce Fields <bfields@redhat.com>
Fri, 11 May 2018 19:48:57 +0000 (15:48 -0400)
commiteb5d7a622e0bbe3fd316b2325d3840a0e030a3c4
tree1f90beaea9186a0f9d393b943be13ec12f355567
parent3316f0631139c87631f2652c118da1a0354bd40d
svcrdma: Allocate recv_ctxt's on CPU handling Receives

There is a significant latency penalty when processing an ingress
Receive if the Receive buffer resides in memory that is not on the
same NUMA node as the the CPU handling completions for a CQ.

The system administrator and the device driver determine which CPU
handles completions. This CPU does not change during life of the CQ.
Further the Upper Layer does not have any visibility of which CPU it
is.

Allocating Receive buffers in the Receive completion handler
guarantees that Receive buffers are allocated on the preferred NUMA
node for that CQ.

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
Signed-off-by: J. Bruce Fields <bfields@redhat.com>
include/linux/sunrpc/svc_rdma.h
net/sunrpc/xprtrdma/svc_rdma_recvfrom.c