Kamailio (OpenSER) 1.2.0 - Performance Tests

Authors:
  Bogdan-Andrei Iancu
  Daniel-Constantin Mierla

1. Overview
2. Transaction Module Performance Tests
  2.1. Transaction-oriented tests
  2.2. Call-oriented tests
3. User Location Performance Tests
  3.1. New registrations
  3.2. Update registrations
  3.3. User location lookup
4. Conclusions
5. Notes
6. References

Overview

Scope

These tests were intended to show the performances of Transaction Module (tm module) and User Location (registrar + usrloc modules) implementations in Kamailio (OpenSER) v1.2.0.

TM Tests

All tests were focus on the relaying capabilities of the TM module. The general design for test scenarios was based on a middle proxy (tested object) between UAC and UAS entities ( UAC <-> proxy <-> UAS )

Two types of tests were performed:

• transaction oriented - scenarios involving simple relay of transactions. The target is to collect data about the performance of basic transaction operations (simple requests like MESSAGE were used)
• call oriented - scenarios involving more complex processing (call specific). The target is to collect data about the performance of call-related operations (full callflows -INVITE+ACK+BYE- were used)

UL Tests

First set of tests was intended to show performances in updating user location table. usrloc was configured with "db_mode=2", meaning that will store new user location records back in MySQL database every 60seconds.

Second set of tests targeted the performances of looking up a record in location table. The location table is cached in memory, there is no database interaction.

Software

As load generator (UAC side) it was used sipp-1.1rc8 (sipp is available at: http://sipp.sourceforge.net)

For Kamailio (OpenSER) proxy, the SVN version (branch 1.2) from 12 March 2007 was used. No packages or binaries were used, sources being locally compiled on the test machine. Kamailio (OpenSER) was compiled with default options (no custom changes at compile time), all configuration being done at runtime. For all tests, the config file is attached. Only exception was the compile time "tinfo=yes" for enabling timestamp logging from script.

For UAS side, another Kamailio (OpenSER) was used (same version) for the simple purpose of "200 OK" replying all the received replies. See here the configuration file.

All used SIP-related software was configured to use UDP as transport protocol for SIP.

Platform

A three hardware servers platform was used for the all the tests. Two identical servers (S1) were used to generate load in parallel via a UAC-like software The third server (S2) was running the openser proxy and a UAS-like software.

Description of the used servers:

• S1 - i386 32bits arch, AMD Sempron processor (more details here) with Debian testing.
• S2 - i386 64bits arch, Intel Core 2 processor (more details here) with Debian testing.

TM - Transaction-oriented tests

Kamailio (OpenSER) was started with 768MB of share memory (parameter: -m 768).

SIPP was used to generate 400 000 MESSAGE requests via the tested proxy. The results reflects the capacity (number of transactions per second) and speed (average response time).

Flow

SIP entities definition:

• UAC 192.168.2.100:5060 and 192.168.2.101:5060
• Proxy - 192.168.2.102:5060
• UAS - 192.168.2.102:5070

SIP messages flow:

In this tests, two parallel UACs were used to generated a high volume of SIP traffic.

Configuration

SIPP command:

Kamailio (OpenSER) Proxy config file:

Simple statefull relay configuration file

Results

Results reported by SIPP on the UAC servers:

By cumulating the results provided by both UACs, it results an
overall throughput of about 15000 transactions per second
for the SIP proxy.

Based on direct information provided by the openser proxy, this is the graphic representation of execution time for t_relay() in Kamailio (OpenSER): openser.cfg

As observation, most of execution times are below 0.1 milliseconds.

Based on direct information provided by the SIPP application (extra -trace_rtt parameter), these are the graphic representations for the response time (the time required - from UAC point of view- to complete a transaction)

As the response time is from UAC perspective, it includes the network delays and the required time to respond for UAS, so from openser testing point of view it is not so relevant.

What is interesting here to observe are the peaks which shows the openser internal timers activity. Requests processed in the same time with the timer hits are delayed due synchronization (locking) on timer lists.

TM - Call-oriented tests

Kamailio (OpenSER) was started with 768MB of share memory (parameter: -m 768).

SIPP was used to generate 400 000 complete calls (setup and termination) via the tested proxy. The results reflects the capacity (number of calls per second) and speed (average response time).

Flow

SIP entities definition:

• UAC 192.168.2.100:5060 and 192.168.2.101:5060
• Proxy - 192.168.2.102:5060
• UAS - 192.168.2.102:5070

SIP messages flow for one call:

In this tests, two parallel UACs were used to generated a high volume of SIP traffic.

Configuration

SIPP command:

Kamailio (OpenSER) Proxy config file:

Simple statefull relay configuration file

Results

Results reported by SIPP on the UAC servers:

By cumulating the results provided by both UACs, it results an
overall throughput of about 8060 complete calls per second
for the SIP proxy.

Based on direct information provided by the SIPP application (extra -trace_rtt parameter), these are the graphic representations for the response time (the time required - from UAC point of view- to initiate and terminate a call)

As the response time is from UAC perspective, it includes the network delays and the required time to respond for UAS, so from openser testing point of view it is not so relevant.

What is interesting here to observe are the peaks which shows the openser internal timers activity. Requests processed in the same time with the timer hits are delayed due synchronization (locking) on timer lists.

Another quite interesting think is if we roughly consider the call as composed from 2 transactions, we get a 16000 transactions per second, which is approximately the same with the result from the transaction test.

UL - Registrations tests

Kamailio (OpenSER) was started with 512MB of share memory (parameter: -m 512).

The requests were initiated only by a single instance of sipp. The CPU usage on SIP registrar was low -- using a distributed requests initiator would reflect better registrar's capacity.

New registrations

sipp was used to inject 100 000 location contacts from a file containing 8-digits numbers, starting with 91, in linear order. The location table was empty, trying to catch here the time required to create location record.

Flow

SIP entities definition:

• UAC 192.168.2.102:random
• Proxy - 192.168.2.102:5060

SIP messages flow:

Configuration

Results

Results reported by SIPP:

overall throughput of over 7600 complete new registrations per second

SIPP Response Time

To measure the response time of SIPP, we deactivated the logging of execution time, since syslog daemon was eating lot of CPU.

Simple registrar with no execution time logging openser.cfg

Graphic representation of SIPP response time:

NOTE: the response time reflects full procesing operations, including generation of SIP reply.

Execution Time

Simple registrar with execution time logging: openser.cfg

Graphic representation of execution time for save("location") in Kamailio (OpenSER):

From observations, response time get's higher when timer fires to check expired contacts. Most of execution times are below 100 micro seconds.

Update registrations

sipp was used to inject 100 000 location contacts from a file containing 8-digits numbers, starting with 91, in random order, for a more realistic simulation. uac-reg.xml from previous test was edited and CSeq was incremented by 1 to simulate a registration update. All SIP accounts used in REGISTERs were already in location table. In this case the location records was not created, but just updated with new values.

Flow

SIP entities definition:

• UAC 192.168.2.102:random
• Proxy - 192.168.2.102:5060

SIP messages flow:

Configuration

Results

Results reported by SIPP:

overall throughput of over 10000 complete updated registrations per second

SIPP Response Time

To measure the response time of SIPP, we deactivated the logging of execution time, since syslog daemon was eating lot of CPU.

Simple registrar with no execution time logging: openser.cfg

Graphic representation of SIPP response time:

NOTE: the response time reflects full procesing operations, including generation of SIP reply.

It can be observed a significant improvement in performances comparing with previous test, due to less memory operations.

UL - Location lookup tests

sipp was used to send 100 000 MESSAGE from a file containing 8-digits numbers, starting with 91, in random order. All users in MESSAGEs' R-URI were in location table.

Flow

SIP entities definition:

• UAC 192.168.2.102:random
• Proxy - 192.168.2.102:5060

SIP messages flow:

Configuration

Results

Results reported by SIPP:

overall throughput of about 10500 user location lookups per second

SIPP Response Time

To measure the response time of SIPP, we deactivated the logging of execution time, since syslog daemon was eating lot of CPU.

Simple registrar with no execution time logging: openser.cfg

Graphic representation of SIPP response time:

NOTE: the response time reflects full procesing operations, including generation of SIP reply.

Execution time

Configuration file openser.cfg

Graphic representation of execution time for lookup("location") in Kamailio (OpenSER), for a set of 100 000 operations:

Higher values are expected to be hash entry collisions together with timer firing for expired contacts cleanup.

Conclusions

The above results are quite technical, so we try to present them in a realistic mode.

Transaction module

When comes to transaction module, I think the relevant tests are the ones for complete calls -- how many calls the system can handle (see tests here). Of course, the tests cover the TM performance only, but in case of a load balancer, the overhead added by extra processing should be quite minimal comparing with TM processing.

The results showed ability to handle about 8000 complete calls per seconds, than means about 480 000 complete calls per minute and 28 800 000 complete calls per hour.

Given these numbers, we consider that processing of transactions becomes prety much insignificant. What will matter will be database responsiveness, and call routing logic in configuration file.

User location

The average expires time for a registration is about 10 min (600sec), considering that, the system would be able to handle at least the registrations comming from 4 020 000 (600 * 6700) subscribers at run time. The number is pretty much over the population someone will assign to a SIP Registrar server in real-world case.

As aside note, Kamailio (OpenSER) 1.2.0 user location uses DB fetch support to load entries from MySQL database at startup, getting rid of "out of private memory" issue present in all old versions when dealing with large online subscriber population (old workaround was to recompile Kamailio (OpenSER) with higher value for private memory). Another feature is that only one child process is loading the records from database, while the others become ready to process SIP traffic (old behaviour was to have all processes waiting to load location database).

Notes

It is important to understand that these tests actually measure the performance of the testbed and not only the performance of the Kamailio (OpenSER) proxy itself (as a separate component).

There a lot of other factors (non-openser related) that are reflected in the results:

• hardware performance (results are proportional to the hardware capacity of the testing machine; important is the number of processor, as openser is multi-process application with a high level of parallel computation)
• network performance (architecture and link/equipment quality)
• test tools performance (limitations of the testing tools)
• test data sampling (a lot of overhead is introduced just by the sampling process)

References

Kamailio (OpenSER) - Open Source SIP Server

SIPP - Open Source SIP Testing Tool