eRPC
Public Member Functions | Static Public Member Functions | Static Public Attributes | Related Functions | List of all members
erpc::Rpc< TTr > Class Template Reference

An Rpc object is the main communication end point in eRPC. Applications use it to create sessions with remote Rpc objects, send and receive requests and responses, and run the event loop. More...

#include <rpc.h>

Public Member Functions

 Rpc (Nexus *nexus, void *context, uint8_t rpc_id, sm_handler_t sm_handler, uint8_t phy_port=0)
 Construct the Rpc object. More...
 
 ~Rpc ()
 Destroy the Rpc from a foreground thread.
 
MsgBuffer alloc_msg_buffer (size_t max_data_size)
 Create a hugepage-backed buffer for storing request or response messages. More...
 
void free_msg_buffer (MsgBuffer msg_buffer)
 Free a MsgBuffer created by alloc_msg_buffer()
 
int create_session (std::string remote_uri, uint8_t rem_rpc_id)
 Create a session to a remote Rpc object and initiate session connection. A session management callback of type kConnected or kConnectFailed will be invoked if this call is successful. More...
 
int destroy_session (int session_num)
 Disconnect and destroy a session. The application must not use this session number after this function is called. More...
 
void enqueue_request (int session_num, uint8_t req_type, MsgBuffer *req_msgbuf, MsgBuffer *resp_msgbuf, erpc_cont_func_t cont_func, void *tag, size_t cont_etid=kInvalidBgETid)
 Enqueue a request for transmission. This always succeeds. eRPC owns msg_buffer until it invokes the continuation callback. More...
 
void enqueue_response (ReqHandle *req_handle, MsgBuffer *resp_msgbuf)
 Enqueue a response for transmission at the server. See ReqHandle for details about creating the response. On calling this, the application loses ownership of the request and response MsgBuffer. More...
 
void run_event_loop (size_t timeout_ms)
 Run the event loop for some milliseconds.
 
void run_event_loop_once ()
 Run the event loop once.
 
MsgBuffer alloc_msg_buffer_or_die (size_t max_data_size)
 Identical to alloc_msg_buffer(), but throws an exception on failure.
 
size_t num_active_sessions ()
 
bool is_connected (int session_num) const
 
size_t get_bandwidth () const
 Return the physical link bandwidth (bytes per second)
 
size_t get_num_re_tx (int session_num) const
 Return the number of retransmissions for a connected session.
 
void reset_num_re_tx (int session_num)
 Reset the number of retransmissions for a connected session.
 
size_t get_stat_user_alloc_tot ()
 Return the total amount of huge page memory allocated to the user.
 
Timely * get_timely (int session_num)
 Return the Timely instance for a connected session. Expert use only.
 
TimingWheel * get_wheel ()
 Return the Timing Wheel for this Rpc. Expert use only.
 
void set_context (void *_context)
 Set this Rpc's context.
 
void set_pre_resp_msgbuf_size (size_t new_pre_resp_msgbuf_size)
 Change this Rpc's preallocated response message buffer size.
 
HugeAlloc * get_huge_alloc () const
 Retrieve this Rpc's hugepage allocator. For expert use only.
 
std::string get_remote_hostname (int session_num) const
 Return the hostname of the remote endpoint for a connected session.
 
uint8_t get_rpc_id () const
 Return the ID of this Rpc object.
 
bool in_background () const
 Return true iff the caller is running in a background thread.
 
size_t get_etid () const
 Return the eRPC thread ID of the caller.
 
double get_freq_ghz () const
 Return RDTSC frequency in GHz.
 
double sec_since_creation ()
 Return the number of seconds elapsed since this Rpc was created.
 
double get_avg_rx_batch ()
 Return the average number of packets received in a call to rx_burst.
 
double get_avg_tx_batch ()
 Return the average number of packets sent in a call to tx_burst.
 
void reset_dpath_stats ()
 Reset all datapath stats to zero.
 
void fault_inject_fail_resolve_rinfo_st ()
 Inject a fault that always fails all routing info resolution. More...
 
void fault_inject_set_pkt_drop_prob_st (double pkt_drop_prob)
 Set the TX packet drop probability for this Rpc. More...
 

Static Public Member Functions

static void resize_msg_buffer (MsgBuffer *msg_buffer, size_t new_data_size)
 Resize a MsgBuffer to fit a request or response. More...
 
static constexpr size_t get_max_data_per_pkt ()
 Return the maximum data size in one packet for the (private) transport.
 
static constexpr size_t get_max_num_sessions ()
 Return the maximum number of sessions supported.
 
static size_t get_max_msg_size ()
 Return the data size in bytes that can be sent in one request or response.
 

Static Public Attributes

static constexpr size_t kMaxMsgSize
 Max request or response data size, i.e., excluding packet headers. More...
 

Related Functions

(Note that these are not member functions.)

enum  ReqFuncType : uint8_t
 The possible kinds of request handlers. Foreground-mode handlers run in the thread that calls the event loop. Background-mode handlers run in background threads spawned by eRPC.
 
typedef void(* erpc_req_func_t) (ReqHandle *req_handle, void *context)
 The type of the request handler function invoked at the server when a request is received. The application owns the request handle (and therefore the request message buffer) until it calls Rpc::enqueue_response. More...
 
typedef void(* erpc_cont_func_t) (void *context, void *tag)
 The type of the continuation callback invoked at the client. This returns ownership of the request and response message buffers that the application supplied in Rpc::enqueue_request back to the application. More...
 
static constexpr size_t kMaxNumERpcProcesses = 32
 Maximum number of eRPC processes per machine.
 
static constexpr size_t kMaxRpcId = UINT8_MAX - 1
 The maximum ID of an Rpc object.
 
static constexpr uint16_t kBaseSmUdpPort = 31850
 The management port for an eRPC processes must be between kBaseSmUdpPort and (kBaseSmUdpPort + kMaxNumERpcProcesses)
 
static constexpr size_t kMaxNumaNodes = 8
 Maximum number of NUMA nodes per machine.
 
static constexpr size_t kMaxBgThreads = 8
 Maximum number of background threads per process.
 
static constexpr size_t kMaxPhyPorts = 16
 Maximum number of datapath device ports.
 
static constexpr size_t kMachineFailureTimeoutMs = 500
 If a client cannot ping a remote server for this duration, we assume that the server has failed. If a server does not hear from a client for this duration, we assume that the client has failed.
 

Detailed Description

template<class TTr>
class erpc::Rpc< TTr >

An Rpc object is the main communication end point in eRPC. Applications use it to create sessions with remote Rpc objects, send and receive requests and responses, and run the event loop.

None of the functions are thread safe. eRPC's worker (background) threads have restricted concurrent access to Rpc objects.

Template Parameters
TTrThe unreliable transport

Constructor & Destructor Documentation

◆ Rpc()

template<class TTr >
erpc::Rpc< TTr >::Rpc ( Nexus nexus,
void *  context,
uint8_t  rpc_id,
sm_handler_t  sm_handler,
uint8_t  phy_port = 0 
)

Construct the Rpc object.

Parameters
nexusThe Nexus object created by this process
contextThe context passed by the event loop to user callbacks
rpc_idEach Rpc object created by threads of one process must have a unique ID. Users create connections to remote Rpc objects by specifying the URI of the remote process, and the remote Rpc's ID.
sm_handlerThe session management callback that is invoked when sessions are successfully created or destroyed.
phy_portAn Rpc object uses one physical port on the NIC. phy_port is the zero-based index of that port among active ports, as listed by ibv_devinfo for Raw, InfiniBand, and RoCE transports; or by dpdk-devbind for DPDK transport.
Exceptions
runtime_errorif construction fails

Member Function Documentation

◆ alloc_msg_buffer()

template<class TTr >
MsgBuffer erpc::Rpc< TTr >::alloc_msg_buffer ( size_t  max_data_size)
inline

Create a hugepage-backed buffer for storing request or response messages.

Parameters
max_data_sizeIf this call is successful, the returned MsgBuffer contains space for this many application data bytes. The MsgBuffer should be resized with resize_msg_buffer() when used for smaller requests or responses.
Returns
The allocated message buffer. The returned message buffer is invalid (i.e., its MsgBuffer.buf is null) if we ran out of hugepage memory.
Exceptions
runtime_errorif size is too large for the allocator, or if hugepage reservation failure is catastrophic. An exception is not thrown if allocation fails simply because we ran out of memory.
Note
The returned MsgBuffer's buf is surrounded by packet headers for internal use by eRPC. This function does not fill in packet headers, although it sets the magic field in the zeroth header.

◆ create_session()

template<class TTr >
int erpc::Rpc< TTr >::create_session ( std::string  remote_uri,
uint8_t  rem_rpc_id 
)
inline

Create a session to a remote Rpc object and initiate session connection. A session management callback of type kConnected or kConnectFailed will be invoked if this call is successful.

Returns
The local session number (>= 0) of the session if the session is successfully created, negative errno otherwise.
Parameters
remote_uriThe remote Nexus's URI, formatted as hostname:udp_port
rem_rpc_idThe ID of the remote Rpc object

◆ destroy_session()

template<class TTr >
int erpc::Rpc< TTr >::destroy_session ( int  session_num)
inline

Disconnect and destroy a session. The application must not use this session number after this function is called.

Parameters
session_numA session number returned from a successful create_session()
Returns
0 if the session disconnect packet was sent, and the disconnect callback will be invoked later. Negative errno if the session cannot be disconnected.

◆ enqueue_request()

template<class TTr >
void erpc::Rpc< TTr >::enqueue_request ( int  session_num,
uint8_t  req_type,
MsgBuffer req_msgbuf,
MsgBuffer resp_msgbuf,
erpc_cont_func_t  cont_func,
void *  tag,
size_t  cont_etid = kInvalidBgETid 
)

Enqueue a request for transmission. This always succeeds. eRPC owns msg_buffer until it invokes the continuation callback.

Parameters
session_numThe session number to send the request on. This session must be connected.
req_typeThe type of the request. The server for this remote procedure call must have a registered handler for this request type.
req_msgbufThe MsgBuffer containing the request data
resp_msgbufThe MsgBuffer that will contain the response data when the continuation is invoked. Its allocation size be large enough to accomodate any response for this request.
cont_funcThe continuation that will be invoked when this request completes. See erpc_req_func_t.
tagA tag for this request that will be passed to the application in the continuation callback
cont_etidThe eRPC thread ID of the background thread to run the continuation on. The default value of kInvalidBgETid means that the continuation runs in the foreground. This argument is meant only for internal use by eRPC (i.e., user calls must ignore it).

◆ enqueue_response()

template<class TTr >
void erpc::Rpc< TTr >::enqueue_response ( ReqHandle *  req_handle,
MsgBuffer resp_msgbuf 
)

Enqueue a response for transmission at the server. See ReqHandle for details about creating the response. On calling this, the application loses ownership of the request and response MsgBuffer.

This can be called outside the request handler.

Parameters
req_handleThe handle passed to the request handler by eRPC
resp_msgbufThe message buffer containing the response. This must be either the request handle's preallocated response buffer or its dynamic response. The preallocated response buffer may be used for only responses that fit in one packet, in which case it is the better choice.
Note
The restriction on resp_msgbuf is inconvenient to the user because they cannot provide an arbitrary application-owned buffer. Unfortunately, supporting this feature will require passing the response MsgBuffer by value instead of reference since eRPC provides no application callback for when the response can be re-used or freed.

◆ fault_inject_fail_resolve_rinfo_st()

template<class TTr >
void erpc::Rpc< TTr >::fault_inject_fail_resolve_rinfo_st ( )

Inject a fault that always fails all routing info resolution.

Exceptions
runtime_errorif the caller cannot inject faults

◆ fault_inject_set_pkt_drop_prob_st()

template<class TTr >
void erpc::Rpc< TTr >::fault_inject_set_pkt_drop_prob_st ( double  pkt_drop_prob)

Set the TX packet drop probability for this Rpc.

Exceptions
runtime_errorif the caller cannot inject faults

◆ is_connected()

template<class TTr >
bool erpc::Rpc< TTr >::is_connected ( int  session_num) const
inline

Return true iff this session is connected. The session must not have been disconnected.

◆ num_active_sessions()

template<class TTr >
size_t erpc::Rpc< TTr >::num_active_sessions ( )
inline

Return the number of active server or client sessions. This function can be called only from the creator thread.

◆ resize_msg_buffer()

template<class TTr >
static void erpc::Rpc< TTr >::resize_msg_buffer ( MsgBuffer msg_buffer,
size_t  new_data_size 
)
inlinestatic

Resize a MsgBuffer to fit a request or response.

Parameters
msg_bufferThe MsgBuffer to resize
new_data_sizeThe new size in bytes of the application data that this MsgBuffer should contain. This must be smaller than the size used to create the MsgBuffer in alloc_msg_buffer().

note This does not modify the MsgBuffer's packet headers

Friends And Related Function Documentation

◆ erpc_cont_func_t

template<class TTr >
typedef void(* erpc_cont_func_t) (void *context, void *tag)
related

The type of the continuation callback invoked at the client. This returns ownership of the request and response message buffers that the application supplied in Rpc::enqueue_request back to the application.

Parameters
contextThe context that was used while creating the Rpc object
tagThe tag used by the application for this request

◆ erpc_req_func_t

template<class TTr >
typedef void(* erpc_req_func_t) (ReqHandle *req_handle, void *context)
related

The type of the request handler function invoked at the server when a request is received. The application owns the request handle (and therefore the request message buffer) until it calls Rpc::enqueue_response.

The application need not enqueue the response in the body of the request handler. This is true even if the request handler is foreground-mode.

Parameters
ReqHandleA handle to the received request
contextThe context that was used while creating the Rpc object

Member Data Documentation

◆ kMaxMsgSize

template<class TTr >
constexpr size_t erpc::Rpc< TTr >::kMaxMsgSize
static
Initial value:
=
HugeAlloc::kMaxClassSize -
((HugeAlloc::kMaxClassSize / TTr::kMaxDataPerPkt) * sizeof(pkthdr_t))

Max request or response data size, i.e., excluding packet headers.

◆ pre_resp_msgbuf_size

template<class TTr >
size_t erpc::Rpc< TTr >::pre_resp_msgbuf_size = TTr::kMaxDataPerPkt

Size of the preallocated response buffer. This is one packet by default, but some applications might benefit from a larger preallocated buffer, at the expense of increased memory utilization.

◆ retry_connect_on_invalid_rpc_id

template<class TTr >
bool erpc::Rpc< TTr >::retry_connect_on_invalid_rpc_id = false

Retry session connection if the remote RPC ID was invalid. This usually happens when the server RPC thread has not started.

◆ still_in_wheel_during_retx

template<class TTr >
size_t erpc::Rpc< TTr >::still_in_wheel_during_retx = 0

Total retransmissions across all sessions.

Number of times we could not retransmit a request, or we had to drop a received packet, because a request reference was still in the wheel.


The documentation for this class was generated from the following files: