An Rpc object is the main communication end point in eRPC. Applications use it to create sessions with remote Rpc objects, send and receive requests and responses, and run the event loop. More...

#include <rpc.h>

Public Member Functions
	Rpc (Nexus nexus, void context, uint8_t rpc_id, sm_handler_t sm_handler, uint8_t phy_port=0)
	Construct the Rpc object. More...

	~Rpc ()
	Destroy the Rpc from a foreground thread.

MsgBuffer	alloc_msg_buffer (size_t max_data_size)
	Create a hugepage-backed buffer for storing request or response messages. Safe to call from background threads (TS). More...

void	free_msg_buffer (MsgBuffer msg_buffer)

int	create_session (std::string remote_uri, uint8_t rem_rpc_id)
	A session is a connection between two eRPC endpoints (similar to a TCP connection). This function creates a session to a remote Rpc object and initiates session connection. A session management callback of type `kConnected` or `kConnectFailed` will be invoked after session creation completes or fails. More...

int	destroy_session (int session_num)
	Disconnect and destroy a session. The application must not use this session number after this function is called. More...

void	enqueue_request (int session_num, uint8_t req_type, MsgBuffer req_msgbuf, MsgBuffer resp_msgbuf, erpc_cont_func_t cont_func, void *tag, size_t cont_etid=kInvalidBgETid)
	Enqueue a request for transmission. This always succeeds. eRPC owns the request and response msgbufs until it invokes the continuation callback. This function is safe to call from background threads (TS). More...

void	enqueue_response (ReqHandle req_handle, MsgBuffer resp_msgbuf)
	Enqueue a response for transmission at the server. This must be either the request handle's preallocated response buffer or its dynamic response. See ReqHandle for details about creating the response. On calling this, the application loses ownership of the request and response MsgBuffer. This function is safe to call from background threads (TS). More...

void	run_event_loop (size_t timeout_ms)
	Run the event loop for some milliseconds.

void	run_event_loop_once ()
	Run the event loop once.

MsgBuffer	alloc_msg_buffer_or_die (size_t max_data_size)
	Identical to alloc_msg_buffer(), but throws an exception on failure.

size_t	num_active_sessions ()

bool	is_connected (int session_num) const

size_t	get_bandwidth () const
	Return the physical link bandwidth (bytes per second)

size_t	get_num_re_tx (int session_num) const
	Return the number of retransmissions for a connected session.

void	reset_num_re_tx (int session_num)
	Reset the number of retransmissions for a connected session.

size_t	get_stat_user_alloc_tot ()
	Return the total amount of huge page memory allocated to the user.

Timely *	get_timely (int session_num)
	Return the Timely instance for a connected session. Expert use only.

TimingWheel *	get_wheel ()
	Return the Timing Wheel for this Rpc. Expert use only.

void	set_context (void *_context)

void	set_pre_resp_msgbuf_size (size_t new_pre_resp_msgbuf_size)
	Change this Rpc's preallocated response message buffer size.

HugeAlloc *	get_huge_alloc () const
	Retrieve this Rpc's hugepage allocator. For expert use only.

std::string	get_remote_hostname (int session_num) const
	Return the hostname of the remote endpoint for a connected session.

uint8_t	get_rpc_id () const
	Return the ID of this Rpc object.

bool	in_background () const
	Return true iff the caller is running in a background thread.

size_t	get_etid () const
	Return the eRPC thread ID of the caller.

double	get_freq_ghz () const
	Return RDTSC frequency in GHz.

double	sec_since_creation ()
	Return the number of seconds elapsed since this Rpc was created.

double	get_avg_rx_batch ()
	Return the average number of packets received in a call to rx_burst.

double	get_avg_tx_batch ()
	Return the average number of packets sent in a call to tx_burst.

void	reset_dpath_stats ()
	Reset all datapath stats to zero.

void	fault_inject_fail_resolve_rinfo_st ()
	Inject a fault that always fails all routing info resolution. More...

void	fault_inject_set_pkt_drop_prob_st (double pkt_drop_prob)
	Set the TX packet drop probability for this Rpc. More...

Static Public Member Functions
static void	resize_msg_buffer (MsgBuffer *msg_buffer, size_t new_data_size)
	Resize a MsgBuffer to fit a request or response. Safe to call from background threads (TS). More...

static constexpr size_t	get_max_data_per_pkt ()
	Return the maximum data size in one packet for the (private) transport.

static constexpr size_t	get_max_num_sessions ()
	Return the maximum number of sessions supported.

static size_t	get_max_msg_size ()
	Return the data size in bytes that can be sent in one request or response.

Static Public Attributes
static constexpr size_t	kMaxMsgSize
	Max request or response data size, i.e., excluding packet headers. More...

Related Functions
(Note that these are not member functions.)
enum	ReqFuncType : uint8_t
	The possible kinds of request handlers. Foreground-mode handlers run in the thread that calls the event loop. Background-mode handlers run in background threads spawned by eRPC.

typedef void(*	erpc_req_func_t) (ReqHandle req_handle, void context)
	The type of the request handler function invoked at the server on receiving a request. More...

typedef void(*	erpc_cont_func_t) (void context, void tag)
	The type of the continuation callback invoked at the client. This returns ownership of the request and response message buffers that the application supplied in Rpc::enqueue_request back to the application. More...

static constexpr size_t	kMaxNumERpcProcesses = 32
	Maximum number of eRPC processes per machine.

static constexpr size_t	kMaxRpcId = UINT8_MAX - 1
	The maximum ID of an Rpc object.

static constexpr uint16_t	kBaseSmUdpPort = 31850
	The management port for an eRPC processes must be between kBaseSmUdpPort and (kBaseSmUdpPort + kMaxNumERpcProcesses)

static constexpr size_t	kMaxNumaNodes = 8
	Maximum number of NUMA nodes per machine.

static constexpr size_t	kMaxBgThreads = 8
	Maximum number of background threads per process.

static constexpr size_t	kMaxPhyPorts = 16
	Maximum number of datapath device ports.

static constexpr size_t	kMachineFailureTimeoutMs = 500
	If a client cannot ping a remote server for this duration, we assume that the server has failed. If a server does not hear from a client for this duration, we assume that the client has failed.

Detailed Description

template<class TTr>
class erpc::Rpc< TTr >

An Rpc object is the main communication end point in eRPC. Applications use it to create sessions with remote Rpc objects, send and receive requests and responses, and run the event loop.

None of the functions are thread safe for user threads.

eRPC's worker (background) threads have limited; concurrent access to Rpc objects. The functions with the _st can be called from only the foreground thread that owns the Rpc object.

Template Parameters

TTr	The unreliable transport

Constructor & Destructor Documentation

◆ Rpc()

template<class TTr >

erpc::Rpc< TTr >::Rpc	(	Nexus *	nexus,
		void *	context,
		uint8_t	rpc_id,
		sm_handler_t	sm_handler,
		uint8_t	phy_port = `0`
	)

Construct the Rpc object.

Parameters

nexus	The Nexus object created by this process
context	The context passed by the event loop to user callbacks
rpc_id	Each Rpc object created by threads of one process must have a unique ID. Users create connections to remote Rpc objects by specifying the URI of the remote process, and the remote Rpc's ID.
sm_handler	The session management callback that is invoked when sessions are successfully created or destroyed.
phy_port	An Rpc object uses one physical port on the NIC. phy_port is the zero-based index of that port among active ports, as listed by `ibv_devinfo` for Raw, InfiniBand, and RoCE transports; or by `dpdk-devbind` for DPDK transport.

Exceptions

runtime_error if construction fails

Member Function Documentation

◆ alloc_msg_buffer()

template<class TTr >

MsgBuffer erpc::Rpc< TTr >::alloc_msg_buffer ( size_t max_data_size )

inline

Create a hugepage-backed buffer for storing request or response messages. Safe to call from background threads (TS).

Parameters

max_data_size If this call is successful, the returned MsgBuffer contains space for this many application data bytes. The MsgBuffer should be resized with resize_msg_buffer() when used for smaller requests or responses.

Returns: The allocated message buffer. The returned message buffer is invalid (i.e., its MsgBuffer.buf is null) if we ran out of hugepage memory.

Exceptions

runtime_error if size is too large for the allocator, or if hugepage reservation failure is catastrophic. An exception is not thrown if allocation fails simply because we ran out of memory.

Note: The returned MsgBuffer's buf is surrounded by packet headers for internal use by eRPC. This function does not fill in packet headers, although it sets the magic field in the zeroth header.

◆ create_session()

template<class TTr >

int erpc::Rpc< TTr >::create_session	(	std::string	remote_uri,
		uint8_t	rem_rpc_id
	)

inline

A session is a connection between two eRPC endpoints (similar to a TCP connection). This function creates a session to a remote Rpc object and initiates session connection. A session management callback of type kConnected or kConnectFailed will be invoked after session creation completes or fails.

Session establisment in eRPC is not optimized, and should be avoided on the datapath.

Returns: The local session number (>= 0) of the session if the session handshake is successfully initiated, negative errno otherwise.

Parameters

remote_uri	The remote Nexus's URI, formatted as hostname:udp_port. The hostname (or IP address) should be under the remote kernel's control. For example, with DPDK, the userspace driver takes full control of a port. That port's IP address is meaningful only prior to its binding with DPDK, so it won't work in create_session.
rem_rpc_id	The ID of the remote Rpc object

◆ destroy_session()

template<class TTr >

int erpc::Rpc< TTr >::destroy_session ( int session_num )

inline

Disconnect and destroy a session. The application must not use this session number after this function is called.

Parameters

session_num A session number returned from a successful create_session()

Returns: 0 if the session disconnect packet was sent, and the disconnect callback will be invoked later. Negative errno if the session cannot be disconnected.

◆ enqueue_request()

template<class TTr >

void erpc::Rpc< TTr >::enqueue_request	(	int	session_num,
		uint8_t	req_type,
		MsgBuffer *	req_msgbuf,
		MsgBuffer *	resp_msgbuf,
		erpc_cont_func_t	cont_func,
		void *	tag,
		size_t	cont_etid = `kInvalidBgETid`
	)

Enqueue a request for transmission. This always succeeds. eRPC owns the request and response msgbufs until it invokes the continuation callback. This function is safe to call from background threads (TS).

Parameters

session_num	The session number to send the request on. This session must be connected.
req_type	The type of the request. The server for this remote procedure call must have a registered handler for this request type.
req_msgbuf	The MsgBuffer containing the request data
resp_msgbuf	The MsgBuffer that will contain the response data when the continuation is invoked. Its allocation size be large enough to accomodate any response for this request.
cont_func	The continuation that will be invoked when this request completes. See erpc_req_func_t.
tag	A tag for this request that will be passed to the application in the continuation callback
cont_etid	The eRPC thread ID of the background thread to run the continuation on. The default value of `kInvalidBgETid` means that the continuation runs in the foreground. This argument is meant only for internal use by eRPC (i.e., user calls must ignore it).

◆ enqueue_response()

template<class TTr >

void erpc::Rpc< TTr >::enqueue_response	(	ReqHandle *	req_handle,
		MsgBuffer *	resp_msgbuf
	)

Enqueue a response for transmission at the server. This must be either the request handle's preallocated response buffer or its dynamic response. See ReqHandle for details about creating the response. On calling this, the application loses ownership of the request and response MsgBuffer. This function is safe to call from background threads (TS).

This can be called outside the request handler.

Parameters

req_handle	The handle passed to the request handler by eRPC
resp_msgbuf	The message buffer containing the response. This must be either the request handle's preallocated response msgbuf or the request handle's dynamic response msgbuf. The preallocated response msgbuf may be used for only responses that fit in one packet, in which case it is the better choice.

Note: The restriction on resp_msgbuf is inconvenient to the user because they cannot provide an arbitrary application-owned buffer. Unfortunately, supporting this feature will require passing the response MsgBuffer by value instead of reference since eRPC provides no application callback for when the response can be re-used or freed.

◆ fault_inject_fail_resolve_rinfo_st()

template<class TTr >

void erpc::Rpc< TTr >::fault_inject_fail_resolve_rinfo_st ( )

Inject a fault that always fails all routing info resolution.

Exceptions

runtime_error if the caller cannot inject faults

◆ fault_inject_set_pkt_drop_prob_st()

template<class TTr >

void erpc::Rpc< TTr >::fault_inject_set_pkt_drop_prob_st ( double pkt_drop_prob )

Set the TX packet drop probability for this Rpc.

Exceptions

runtime_error if the caller cannot inject faults

◆ free_msg_buffer()

template<class TTr >

void erpc::Rpc< TTr >::free_msg_buffer ( MsgBuffer msg_buffer )

inline

Free a MsgBuffer created by alloc_msg_buffer(). Safe to call from background threads (TS).

◆ is_connected()

template<class TTr >

bool erpc::Rpc< TTr >::is_connected ( int session_num ) const

inline

Return true iff this session is connected. The session must not have been disconnected.

◆ num_active_sessions()

template<class TTr >

size_t erpc::Rpc< TTr >::num_active_sessions ( )

inline

Return the number of active server or client sessions. This function can be called only from the creator thread.

◆ resize_msg_buffer()

template<class TTr >

static void erpc::Rpc< TTr >::resize_msg_buffer	(	MsgBuffer *	msg_buffer,
		size_t	new_data_size
	)

inlinestatic

Resize a MsgBuffer to fit a request or response. Safe to call from background threads (TS).

Parameters

msg_buffer	The MsgBuffer to resize
new_data_size	The new size in bytes of the application data that this MsgBuffer should contain. This must be smaller than the size used to create the MsgBuffer in alloc_msg_buffer().

note This does not modify the MsgBuffer's packet headers

◆ set_context()

template<class TTr >

void erpc::Rpc< TTr >::set_context ( void * _context )

inline

Set this Rpc's optaque context, which is passed to request handlers and continuations.

Friends And Related Function Documentation

◆ erpc_cont_func_t

template<class TTr >

typedef void(* erpc_cont_func_t) (void *context, void *tag)

The type of the continuation callback invoked at the client. This returns ownership of the request and response message buffers that the application supplied in Rpc::enqueue_request back to the application.

Parameters

context	The context that was used while creating the Rpc object
tag	The tag used by the application for this request

◆ erpc_req_func_t

template<class TTr >

typedef void(* erpc_req_func_t) (ReqHandle *req_handle, void *context)

The application need not enqueue the response inside the request handler. It can do so later.

Request buffer ownership: The application owns the request message buffer until it enqueues the response, except in one common case. If zero-copy RX is enabled (i.e., kZeroCopyRx is true) and the request message fits in one packet, the application owns the request message buffer for only the duration of the request handler.

Parameters

ReqHandle	A handle to the received request
context	The context that was used while creating the Rpc object

Member Data Documentation

◆ kMaxMsgSize

template<class TTr >

constexpr size_t erpc::Rpc< TTr >::kMaxMsgSize

static

Initial value:

=
      HugeAlloc::k_max_class_size -
      ((HugeAlloc::k_max_class_size / TTr::kMaxDataPerPkt) * sizeof(pkthdr_t))

Max request or response data size, i.e., excluding packet headers.

◆ pre_resp_msgbuf_size_

template<class TTr >

size_t erpc::Rpc< TTr >::pre_resp_msgbuf_size_ = TTr::kMaxDataPerPkt

Size of the preallocated response buffer. This is one packet by default, but some applications might benefit from a larger preallocated buffer, at the expense of increased memory utilization.

◆ retry_connect_on_invalid_rpc_id_

template<class TTr >

bool erpc::Rpc< TTr >::retry_connect_on_invalid_rpc_id_ = false

Retry session connection if the remote RPC ID was invalid. This usually happens when the server RPC thread has not started.

◆ still_in_wheel_during_retx_

template<class TTr >

size_t erpc::Rpc< TTr >::still_in_wheel_during_retx_ = 0

Total retransmissions across all sessions.

Number of times we could not retransmit a request, or we had to drop a received packet, because a request reference was still in the wheel.

The documentation for this class was generated from the following files:

Public Member Functions

Static Public Member Functions

Static Public Attributes

Related Functions

Detailed Description

template<class TTr> class erpc::Rpc< TTr >

Constructor & Destructor Documentation

◆ Rpc()

Member Function Documentation

◆ alloc_msg_buffer()

◆ create_session()

◆ destroy_session()

◆ enqueue_request()

◆ enqueue_response()

◆ fault_inject_fail_resolve_rinfo_st()

◆ fault_inject_set_pkt_drop_prob_st()

◆ free_msg_buffer()

◆ is_connected()

◆ num_active_sessions()

◆ resize_msg_buffer()

◆ set_context()

Friends And Related Function Documentation

◆ erpc_cont_func_t

◆ erpc_req_func_t

Member Data Documentation

◆ kMaxMsgSize

◆ pre_resp_msgbuf_size_

◆ retry_connect_on_invalid_rpc_id_

◆ still_in_wheel_during_retx_

template<class TTr>
class erpc::Rpc< TTr >