Interprocess RPC generation tool

Introduction

This post discusses a methodology to create a Remote Procedure Call interface for Process-to-Process intercommunication. That is, to communicate between two processes in the same host.
The general solution for the problem is summarized here as design pattern. A tool to automatically generate the RPC stubs code, called irpcgen, is presented as well.

The Problem

We have two processes in an embedded system, lets call them H and C.

• H - is a hard real-time process with strict deadlines. This can be one or more control loops or an acquisition system for example.
• I - controls the operation of H and perform other non time sensitive tasks. It may implement the user interface, operational logging etc... but it's main task is create and watchdog H.

The question that arises is, what's the best approach to create a communication channel between H and I? More specifically, we want to answer two questions:

1. which IPC mechanism will be best suited for the task?
2. how to send and receive structured information over these channels?

The Solution

To answer 1. I created a small set of programs to benchmark several alternative IPC mechanisms in Linux. See my previous posts on the subject: Embedded Linux Interprocess Communication Mechanisms Benchmark - Part 2

With these data in hand it occurred to me that a natural channel will be two pipes,  I have used this approach in other opportunities, but I never considered before of using unnamed pipes for the task. That's what I propose here, to used a pair of unnamed pipes connected to the stdin and stdout. Which was the best thing to do as our controlling process I is the one forking H. And H have only one single controller attached to it. This way we created a two-way Process to Process IPC channel. Now we have to be able to send and receive structured data trough it.

My requisites for the data exchange mechanisms were:

• Simple to program and extend
• The communication has to be synchronous
• The programming interface has to be at high level, RPC like

First thing to do was to transform an asynchronous channel into a synchronous one. To do this a small overhead protocol was introduced. It just defines a framing structure, to delimit the message boundaries and a scheme to multiplex different message types, also it introduced control messages for synchronization an link management.

Next step was to create a way of encapsulate C structures into the messages and to label them in order to be able to demultiplex on reception. No marshalling is necessary because both processes are in the same host. This involved in defining, for each message, a function to be called to transmit it and a corresponding callback to be invoked on reception.

This can be done manually. As a matter of fact I just did it, in the first product developed with this approach. It was also a way of validating the strategy without incurring in too much tooling effort.

But for this to be generally useful a tool to automatic generate the code was needed.

The irpcgen tool

To make the development easy I created a tool to generate the stubs for the server and client as well as sample server service calls. The program works pretty much like the SUN RPC rpgen tool, except that instead of reading a RPCL input file (.x) it reads a standard C header (.h) file. This is to super simplify the things. You just need to write your API in a header file an use the functions in the client's side. The implementation of the functions will be at the server's side.

The rpcgen will read the header file and will create stubs for all functions declared that can be used as RPC. This functions represent the server's API and have to follow some rules:

1 - The return has to be a bool type;
2 - There must be at most 2 arguments to the function;
3 - It cannot be declared static;
4 - If a second argument is provided it has to be a pointer to something except a void pointer;

Functions that fail to conform to any of these rules are not considered IRPC and no stub will be generated for them.

Furthermore the direction of the data transmission will be derived by the position and type of the arguments. The following cases are possible:

No arguments

Sends nothing returns nothing (but invokes the corresponding callback on the server side) .

bool my_rpc(void);

Single argument passed by value.

This is an server input value. This is more or less obvious as the client cannot read anything back. 

bool my_rpc_set(int val);
bool my_rpc_set(struct my_req req);

Single argument passed by constant reference.

This is similar as the previous case, the single argument is a server input value.

bool my_rpc_set(const struct my_req * req);

Single argument passed by reference.

This case the argument is a return value from the server. The client should provide a pointer to a variable that will receive the data.

bool my_rpc_get(int * val);
bool my_rpc_get(struct my_rsp * rsp);

Two arguments

This case the first argument is an input value and the second a return value from the server. The client should provide a pointer to a variable that will receive the data. Note that the second argument must be a non constant reference. The first argument can be any, except a void pointer (void *); 

bool my_rpc_set_and_get(struct my_req * req, struct my_rsp * rsp);
bool my_rpc_set_and_get(const struct my_req * req, struct my_rsp * rsp);
bool my_rpc_set_and_get(struct my_req req, struct my_rsp * rsp);
bool my_rpc_set_and_get(int req, int * rsp);

Strings

If any argument is passed as a char pointer (char *) it will be treated as a NULL terminated string. The rule for a single argument is the same as for reference. I.e. if it's declared as const it represents a client to server message and will be reverse for non const strings.

bool my_rpc_set_and_get(char * req, char * rsp);
bool my_rpc_set_and_get(const char * req, char * rsp);


bool my_rpc_set(const char * req);
bool my_rpc_get(char * rsp);

Service calls

The irpcgen tool will optionally create a ".h" file with "_svc" appended to the input file name. E.g. if the input is "my_rpc.h" the generated file will be "my_rpc_svc.h". The file will contains the signature for the services to be implemented.

The file:

bool my_rpc_get(int * val);
bool my_rpc_set(int val);

Will produce:

bool my_rpc_get_svc(int * val);
bool my_rpc_set_svc(int val);

The "_svc" functions must implement the server behaivour. Optionally a "*_svc.c" can be created with dummy functions. All you need to do is to fill this functions body to have a functional RPC system.

The libirpc

The libirpc is the companion of the irpcgen. The generated code depends on this library to run.

Source code

The irpcgen tool is GPL open-source and can be downloaded from: irpcgen.tar.gz

The package also contains the libirpc and a sample. The library is LGPL licensed.

There is a Makefile in the directories irpcgen, libirpc and sample. You need to compile irpcgen and libirpc before compiling the test.

If you want do cross-compile the library and the sample to an embedded platform, set the environment variable CROSS_COMPILE to the prefix of your tool-chain e.g. export CROSS_COMPILE=arm-gnu-linux-.