Stress testing Porcupine v0.6

Posted by tkouts on 15 November 2009 | 2 Comments

Tags: porcupine, performance, pylot

Before releasing Porcupine v0.6 I have conducted a series of stress tests using Pylot. The aim of one of these stress tests was to benchmark Porcupine under extreme race conditions.

For this reason I created a test scenario with only two requests. The first request authenticates the "admin" user against the Porcupine database whereas the second request calls a remote web method that creates a common container object (folder) inside the root container object.
The test scenario is the following:

<testcases>
   <!-- login -->
   <case>
      <url>http://192.168.233.115/porcupine.py/</url>
      <method>POST</method>
      <body><![CDATA[<?xml version="1.0"?>
         <methodCall>
            <methodName>login</methodName>
            <params>
               <param><value><string>admin</string></value></param>
               <param><value><string>admin</string></value></param>
            </params>
         </methodCall>
       ]]></body>
      <add_header>User-agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.8) Gecko/2009032609 Firefox/3.0.8</add_header>
      <add_header>Content-type: text/xml</add_header>
   </case>
   <!-- create object -->
   <case>
      <url>http://192.168.233.115/porcupine.py/</url>
      <method>POST</method>
      <body><![CDATA[<?xml version="1.0"?>
         <methodCall>
            <methodName>create_object</methodName>
            <params></params>
         </methodCall>
      ]]></body>
      <add_header>User-agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.8) Gecko/2009032609 Firefox/3.0.8</add_header>
      <add_header>Content-type: text/xml</add_header>
   </case>
</testcases>


The "create_object" web method is as simple as:

from porcupine.utils import misc
from org.innoscript.desktop.schema.common import Folder

@webmethods.remotemethod(of_type=RootFolder)
@db.transactional(auto_commit=True)
def create_object(self):
   f = Folder()
   f.displayName.value = misc.generate_oid()
   f.append_to('')
   return True


The stress test's duration was 10mins. It initiates 100 agents applying requests every 80ms with a warm-up time of 100secs. The command line is:

$ python run.py -a 100 -d 1200 -i 80 -r 100 -x porcupine_txn_test.xml -o results -n "Porcupine 0.6 Performance Results"


Due to the fact that that the root folder in this scenario is considered a highly demanded resource since all objects created are created inside this container the maximum number of concurrent transaction is lowered to 8 (max_tx setting).
The test's highlights are:
- Porcupine was able to serve more than 160K database requests in 10mins
- Half of the above requests were transactional which means more than 80K full ACID transactions where committed successfully (more than 65 transactions per second).
- Only 17 transactional requests could not be completed (exceeded maximum retries)
- The response time of 90% of the requests was less than 1sec.

The machine used for the test was a virtual ESX server with dual core processor and Ubuntu installed.

The detailed results can be found here.


Clicky Web Analytics