Benchmarks avec le serveur OpenERP hebergé sur Internet

As Akretion starts to host some OpenERP servers for its SMB customers, we wanted to know which protocol is best for accessing an OpenERP server hosted on the Internet. So we made a comparision of the performances of Net-RPC, XML-RPC, XML-RPC SSL, HTTP, HTTPS, etc… Of course, only secure protocols should be selected when using an OpenERP server hosted on the Internet.

=========


## Benchmarks with OpenERP server hosted on the Internet

October 27th, 2011

As Akretion starts to host some OpenERP servers for its SMB customers, we want to propose them a solution that is both fast and secure. So far, the majority of our customers host the OpenERP server on their LAN and access it via the Gtk Client through the Net-RPC protocol. The Net-RPC protocol is fast but not secure : the passwords and all the data are sent in clear, but our customers consider that, as the connection is made over their LAN, it’s not a problem. By the way, it is probably a bad assumption, because a lot of IT attacks against companies come from the inside ! And when our customers need to access OpenERP from outside the office, they connect by VPN to their LAN and then they use OpenERP as usual with Gtk Client and Net-RPC protocol.

In order to select the best option for our customers that access an OpenERP server over the Internet, we made a real-life benchmark with an OpenERP server running on a machine in a datacenter, and elaborated a precise usage scenario. Basically, the scenario consist in a full purchase workflow (PO, reception of goods, supplier invoice) and a full sale workflow (SO, delivery order, customer invoice). For each protocol, we executed this scenario manually multiple times. We kept the fastest timing that was measured. As we ran the scenario manually, we estimate that the results are valid with a 10 seconds accuracy.

#### Bench with OpenERP server hosted on the Internet

<table>
<tr>
<th style='border:1px solid #CAD0D8; padding:0.3em; background:#dbf2fc; text-align:center; color:#666;'></th>
<th style='border:1px solid #CAD0D8; padding:0.3em; background:#dbf2fc; text-align:center; color:#666;'>Net-RPC</th>
<th style='border:1px solid #CAD0D8; padding:0.3em; background:#dbf2fc; text-align:center; color:#666;'>Net-RPC inside SSH tunnel</th>
<th style='border:1px solid #CAD0D8; padding:0.3em; background:#dbf2fc; text-align:center; color:#666;'>XML-RPC</th>
<th style='border:1px solid #CAD0D8; padding:0.3em; background:#dbf2fc; text-align:center; color:#666;'>XML-RPC SSL</th>
<th style='border:1px solid #CAD0D8; padding:0.3em; background:#dbf2fc; text-align:center; color:#666;'>HTTP access with Firefox 7</th>
<th style='border:1px solid #CAD0D8; padding:0.3em; background:#dbf2fc; text-align:center; color:#666;'>HTTP access with Chrome 14</th>
<th style='border:1px solid #CAD0D8; padding:0.3em; background:#dbf2fc; text-align:center; color:#666;'>HTTPS access through an Nginx proxy with Chrome 14</th>
</tr>
<tr>
<th style='border:1px solid #CAD0D8; background:#dbf2fc; padding:0.3em; color:#666;'>Secure ?</th>
<td style='border:1px solid #CAD0D8; background:#ff2e24; padding:0.3em; text-align:center;'>no</td>
<td style='border:1px solid #CAD0D8; background:#50d023; padding:0.3em; text-align:center;'>yes</td>
<td style='border:1px solid #CAD0D8; background:#ff2e24; padding:0.3em; text-align:center;'>no</td>
<td style='border:1px solid #CAD0D8; background:#50d023; padding:0.3em; text-align:center;'>yes</td>
<td style='border:1px solid #CAD0D8; background:#ff2e24; padding:0.3em; text-align:center;'>no</td>
<td style='border:1px solid #CAD0D8; background:#ff2e24; padding:0.3em; text-align:center;'>no</td>
<td style='border:1px solid #CAD0D8; background:#50d023; padding:0.3em; text-align:center;'>yes</td>
</tr>
<tr>
<th style='border:1px solid #CAD0D8; background:#dbf2fc; padding:0.3em; color:#666;'>OpenERP 6.0.3</th>
<td style='border:1px solid #CAD0D8; background:#50d023; padding:0.3em; text-align:center;'>1 min 49</td>
<td style='border:1px solid #CAD0D8; background:#50d023; padding:0.3em; text-align:center;'>1 min 47</td>
<td style='border:1px solid #CAD0D8; background:#ffc038; padding:0.3em; text-align:center;'>2 min 11</td>
<td style='border:1px solid #CAD0D8; background:#ff2e24; padding:0.3em; text-align:center;'>3 min 08</td>
<td style='border:1px solid #CAD0D8; background:#ffc038; padding:0.3em; text-align:center;'>2 min 04</td>
<td style='border:1px solid #CAD0D8; background:#ffc038; padding:0.3em; text-align:center;'>2 min 11</td>
<td style='border:1px solid #CAD0D8; background:#ffc038; padding:0.3em; text-align:center;'>2 min 05</td>
</tr>
<tr>
<th style='border:1px solid #CAD0D8; background:#dbf2fc; padding:0.3em; color:#666;'>OpenERP 6.1 (trunk)</th>
<td style='border:1px solid #CAD0D8; background:#50d023; padding:0.3em; text-align:center;'>1 min 42</td>
<td style='border:1px solid #CAD0D8; background:#50d023; padding:0.3em; text-align:center;'>1 min 40</td>
<td style='border:1px solid #CAD0D8; background:#ffc038; padding:0.3em; text-align:center;'>1 min 52</td>
<td style='border:1px solid #CAD0D8; background:#fff; padding:0.3em; text-align:center;'>not available ?</td>
<td style='border:1px solid #CAD0D8; background:#fff; padding:0.3em; text-align:center;'>buggy</td>
<td style='border:1px solid #CAD0D8; background:#50d023; padding:0.3em; text-align:center;'>1 min 43</td>
<td style='border:1px solid #CAD0D8; background:#50d023; padding:0.3em; text-align:center;'>1 min 40</td>
</tr>
</table>

#### Bench with an OpenERP server hosted on the LAN

<table>
<tr>
<th style='border:1px solid #CAD0D8; padding:0.3em; background:#dbf2fc; text-align:center; color:#666;'></th>
<th style='border:1px solid #CAD0D8; padding:0.3em; background:#dbf2fc; text-align:center; color:#666;'>Net-RPC</th>
</tr>
<tr>
<th style='border:1px solid #CAD0D8; background:#dbf2fc; padding:0.3em; color:#666;'>OpenERP 6.0.3</th>
<td style='border:1px solid #CAD0D8; background:#50d023; padding:0.3em; text-align:center;'>1 min 15</td>
</tr>
<tr>
<th style='border:1px solid #CAD0D8; background:#dbf2fc; padding:0.3em; color:#666;'>OpenERP 6.1 (trunk)</th>
<td style='border:1px solid #CAD0D8; background:#50d023; padding:0.3em; text-align:center;'>1 min 08</td>
</tr>
</table>


Here is our analysis of the results of this benchmark :

* OpenERP 6.1 is faster than OpenERP 6.0, thanks to the optimisations in the code of the server and addons (and also thanks to an ergonomy improvement on invoice generation from delivery orders).

* XML-RPC SSL is not an option because it’s too slow. If you activate the debug logs on the OpenERP server, you will see that the SSL session is re-established for every request, which explains the bad performances.

* on OpenERP 6.0, Net-RPC is the fastest, but the Web interface is not too far behind : only 15% slower.

* on OpenERP 6.1, the Web interface is as fast as Net-RPC !

The developers of the new Web interface of OpenERP 6.1 made a great job ! Thumbs up !

As a conclusion, for those who plan to use an OpenERP server over the Internet, the best option is probably to upgrade to version 6.1 when it will be released and use the new OpenERP Web interface via HTTPS through a reverse proxy (more explainations about reverse proxy on Wikipedia).