Embedded Linux Interprocess Communication Mechanisms Benchmark 2nd Part

This is the second part of the benchmark of some IPC on Embedded Linux.  See the previous post: Embedded Linux Interprocess Communication Mechanisms Benchmark - Part 1.


Source Code
The source code with the tests can be downloaded from here: ipc_bm.tar.gz.
To compile just adjust the variable CROSS_COMPILE in the main Makefile and do make all.

Results
The listings bellow show the output of the tests
POSIX Message Queue

* IPC Benchmark start
 - POSIX mq server...
 - POSIX mq client...
 - Large message send test:
      0.52 secs,  3870.7 Msgs/s,  3870.7 KiB/s
 - Large message receive test:
      0.52 secs,  3858.2 Msgs/s,  3858.2 KiB/s
 - Medium message send test:
      0.50 secs,  4019.3 Msgs/s,   502.4 KiB/s
 - Medium message receive test:
      0.46 secs,  4386.2 Msgs/s,   548.3 KiB/s
 - Small message send test:
      0.50 secs,  4021.6 Msgs/s,    62.8 KiB/s
 - Small message receive test:
      0.45 secs,  4426.0 Msgs/s,    69.2 KiB/s
 - Event posting test:
      0.34 secs,  5918.5 Msgs/s,    23.1 KiB/s
* IPC Benchmark end.

Shared Memory

* IPC Benchmark start
 - POSIX shared memory server...
 - POSIX shared memory client...
 - Large message send test:
      0.82 secs,  2453.5 Msgs/s,  2453.5 KiB/s
 - Large message receive test:
      0.82 secs,  2447.1 Msgs/s,  2447.1 KiB/s
 - Medium message send test:
      0.80 secs,  2503.6 Msgs/s,   313.0 KiB/s
 - Medium message receive test:
      0.80 secs,  2494.6 Msgs/s,   311.8 KiB/s
 - Small message send test:
      0.79 secs,  2519.9 Msgs/s,    39.4 KiB/s
 - Small message receive test:
      0.80 secs,  2515.5 Msgs/s,    39.3 KiB/s
 - Event posting test:
      0.79 secs,  2540.8 Msgs/s,     9.9 KiB/s
* IPC Benchmark end.

ONC RPC

* IPC Benchmark start
 - ONC RPC Server...
 - ONC RPC Client...
 - Large message send test
   -    1.73 secs,  1156.8 Msgs/s,  1156.8 KiB/s
 - Large message receive test
   -    1.74 secs,  1148.4 Msgs/s,  1148.4 KiB/s
 - Medium message send test
   -    1.65 secs,  1209.0 Msgs/s,   151.1 KiB/s
 - Medium message receive test
   -    1.65 secs,  1211.2 Msgs/s,   151.4 KiB/s
 - Small message send test
   -    1.64 secs,  1219.2 Msgs/s,    19.0 KiB/s
 - Small message receive test
   -    1.66 secs,  1202.1 Msgs/s,    18.8 KiB/s
 - Event posting test
   -    0.25 secs,  7904.5 Msgs/s,    30.9 KiB/s
* IPC Benchmark end.

Comments
In order to run all the tests you must be sure that the following options are enabled in the kernel:

General Setup:
  [*] POSIX Message Queues 
  ...
  Configure standard kernel features (for small systems) :
    [*]   Use full shmem filesystem  

File systems:
  ...
  Pseudo filesystems:
    ...
    [*] Virtual memory file system support (former shm fs) 

<< Embedded Linux Interprocess Communication Mechanisms Benchmark - Part 1

Embedded Linux Interprocess Communication Mechanisms Benchmark

Hi there,

this is my first attempt to blog, so please excuse me for not having or following any stylistic conventions for this kind of writing. As a matter of fact writing is not something I do often. That being said, I'm open to any criticism regarding either misuses of the English language or errors/omissions in the information content I will eventually present. So fell free to post comments and such.

This first post is intended to be the initial part of a benchmark test on some IPC (Interprocess Communication) mechanisms that I'm evaluating to implement in a commercial product. I will not going to disclose what the project or product is about, but I will outline the requirements of the subsystem involved in the test.

Overview

Before we start dealing with the problem, I would like to make some comments regarding the usefulness of the results I intend to achieve. For most of the reviews or comparisons out there there is a lack of information regarding the platform on which the tests where performed. In my opinion, this makes things a little confusing when you try to figure out whether such a solution will be appropriate or not for your system. Because differences in cache sizes and architectures, memory bandwidth, library implementation and other factors may affect the results, favoring one or another solution will depend on taking this conditions into account. This way whatever the results may be in my particular tests, I will only recommend the use of a particular approach to someone who have a similar platform.

The system I'm working right now is based on an ARM9926EJ processor with 8KiB of data cache and 16KiB of instruction cache. The processor clock is close to 300MHz and the system memory is a 128MiB 16bits DDR2 type running roughly at 540MHz.
The Linux kernel version is 2.6.32 and the C library is glibc version 2.8.

The testing setup will consists of two process: a server and a client that will perform 3 type of conversation:

1 - Synchronous request - the client issue a request to the server, which will perform some tasks and return some data as result. The size of the reply may vary between 4 to 1024 bytes, depending on the service being requested. The client will wait (block) for the server to reply.
2 - Synchronous send - the client send some data to the server (4 to 1024 bytes) and waits for the server to process it and reply back with a status.
3 - Asynchronous notification - the client send a notification (event) to the server without waiting for acknowledgment.


The mechanisms being tested are:

1 - POSIX Message Queues (mq): in this case the messages will be send, received and synchronized trough the mechanism itself. It's a very straightforward implementation.
2 - POSIX Shared Memory + POSIX Semaphores: this will be a little more evolving as we need to have a mechanism to send the data (Shared Memory) and another for synchronization and mutual exclusion ( in case we have more than one client accessing the server's shared memory resources).
3 - ONC RPC (Open Network Computing Remote Procedure Call - aka SUN RPC) :  it may seems a little odd why I want even to consider this but some of the reasons are:

• It will simplify the interface creation trough the use of the XDR (kind of IDL)
• It will enable the same API to be used for remote access which is another requirement of the product.
• It has provision for UNIX Sockets for local transactions (although I, myself, never used it) 
• It will be fun to do it.

Embedded Linux Interprocess Communication Mechanisms Benchmark - Part 2 >>