Using Tsung for Load Testing Rails or Web Apps

« Problems with Siege, Sessions, and Rails 2

Using Tsung for Load Testing Rails or Web Apps

September 12, 2008 by Austin Mills

After trying some other load-testing tools, I found Tsung, a load-testing application written in Erlang to take advantage of that language’s concurrency support. It scales well (it’s been used to simulate tens of thousands of users), it supports forms and HTTP sessions, and includes some niceties like proxy recording, ‘thinktime’ support, and a choice of random or ordered traffic.

Not having a physical server handy to run Tsung isn’t a big obstacle; I just used an Amazon EC2 instance (an xlarge turned out to be more than enough, you could probably get 100s of users on a medium), which due to being rentable by the hour is great for a short-term need like occasional load-testing. I put one of Alestic’s Ubuntu images on the EC2 instance, and then it’s just a few short steps to install Tsung on the machine (we’ll call it ‘tsunghost’):

sudo apt-get install erlang
sudo apt-get install erlang-src
sudo apt-get install gnuplot-nox sudo apt-get install libtemplate-perl libhtml-template-perl libhtml-template-expr-perl
Download the latest from http://tsung.erlang-projects.org/dist/ and install it (the usual configure, make, make install). I prefixed it into /opt/tsung just to keep it a little cleaner, so the following will reflect that.

Update: Ben says in the comments, “Tsung now has Ubuntu packages, so you don’t need to compile. You can grab the latest deb at http://tsung.erlang-projects.org/dist/ubuntu/ . Then just sudo dpkg -i on it.”

The first thing you’ll want to do is record a sample web session. Start Tsung in record mode with ‘/opt/bin/tsung -L 9000 recorder’ (the -L bit indicates the port number that the proxy will listen on). Now, you can set tsunghost:9000 as the HTTP proxy in your web browser and all the web activity will be recorded by Tsung for playback later.

With Tsung recording, exercise your web application — but keep in mind that all of your actions will be repeated later, so for example, don’t make comments on a web page that only allows a max of 10 comments, or else you’ll get a high percentage of errors when it fails every time it’s hit after the 10th.

When you’re done with your web session, stop Tsung with ‘/opt/bin/tsung stop_recorder’. If you look in your home directory, you should now have a .tsung directory with a log of your session, something like ~/.tsung/tsung_recorderYYYMMDD-HH:MM.xml. That file contains the log of all the activity you did while sending traffic through the Tsung proxy, and will be used to drive the simulated user sessions during the load testing.

With the session file in hand, we can create a very simplified file to drive Tsung’s testing. There are some examples that were installed for you at /opt/tsung/share/doc/tsung/examples, but even the simplest example was more complicated than what I ended up using for my testing.

The only thing you must change here is the path to the session file that you recorded earlier, and possibly the hostname — although in my case it appeared to be overridden by the host defined in the session file. Apart from that, you might want to configure some details of the load testing, and set a maxusers value that’s somewhat reasonable for the test you have in mind.

  
  &lt;!DOCTYPE tsung SYSTEM &quot;/opt/tsung/share/tsung/tsung-1.0.dtd&quot; [  ] &gt;

  <!-- set dumptraffic="true" to dump all received and sent packets -->
  <!-- set loglevel="debug" for maximum verbosity -->
  

    <!-- Client side setup -->
    
      <!-- maxusers is the max number of simultaneous clients. Don't set it too high because you can run out of file descriptors. Remember that ssl_esock use 2 fds by connection. -->
      
    

    <!-- Server side setup -->
    
      
    

    <!-- several arrival phases can be set: for each phase, you can set the mean inter-arrival time between new clients and the phase duration -->
    
      
        
      
    

    <!-- Options. -->
     <!-- Thinktime value overrides those set in the session nodes if override=true. -->
      

      <!-- HTTP parameters -->
      
        Mozilla/5.0 (Windows; U; Windows NT 5.2; fr-FR; rv:1.7.8) Gecko/20050511
          Firefox/1.0.4
        
      
    
    <!-- ********************************************** --> <!-- start your recorded session --> <!-- ********************************************** -->
     &amp;mysession1;

In this case, we're using clients on the local machine only, with a new user arriving every 5 seconds for 10 minutes, and with each user pausing for 3 seconds between requests to simulate thinking time. All clients claim to be Firefox, but that didn't really matter to me -- you may want to change it depending on how much browser-specific code you have.

When you have your tsung.xml file the way you like it, start Tsung with /opt/tsung/bin/tsung start. It will start and run until the last user session has completed (so a test defined to run for 10 minutes might take much longer than 10 minutes to complete if the server was heavily loaded). When the test is completed, Tsung will exit and write out a log file in ~/.tsung/log with the timestamp included in the filename.

Tsung has a tool included to generate HTML formatted reports that summarize the test, but they have to be generated by running /opt/tsung/lib/tsung/bin/lib_stats.pl in that test run's directory. It will generate several html files and charts in png and ps format. Open up report.html or graphs.html in a browser to view an array of data from the test run. The Tsung home page has some samples of what the report will look like.

One last thing -- by default, the charts are pretty low-res PNG files, but if you want to, you can create higher-resolution images from the PS (PostScript) files that each chart links to using a tool like GSView.

As resources, both the Tsung documentation and this post by 21 croissants were very useful to me as I was getting started.

Posted in Coding, Coding Tips | 14 Comments

14 Responses

on September 22, 2008 at 1:02 am | Reply Amandeep

very useful information indeed
on October 23, 2008 at 2:37 pm | Reply Dmitry

Thank you for a great run through, very helpful. I had a question for you though; when I’m running an HTTP benchmark test and the users are logging into the system and performing what I did all over again, how come after I reset my questionnaire on the web app and after tsung is done with it’s http test, I log back into my questionnaire it’s still reset and the questionnaire has not been run. I was under the impression that when the http test is completed those calls should have been made and my webapp should show me a questionnaire that’s been already answered by tsungs http calls?

Thank you again.
Dmitry
on October 23, 2008 at 2:53 pm | Reply Austin Mills

Dmitry: Ideally, the web requests from Tsung should be very similar to the web requests made by a user using your web app. I would recommend a couple of things:
1. Read through your session file (if you used the recorder, it’s the output from the recorder). That will give you a step-by-step view of what Tsung is doing on a per-session basis and let you see if perhaps a step is being missed.
2. Set up Tsung so that it only runs a single session, using 1 client (which can be done easily by setting the interarrival higher than the duration in the ‘load’ section of the XML) and look at the logs on your webserver. This will let you trace through on the server side a single session (I usually turn on request logging so I can see each request as it happens) and can help verify that Tsung is doing what you expect it to.

Hope that helps,
–Austin
on October 23, 2008 at 3:00 pm | Reply Dmitry

Awesome, thanks Austin.

~Dmitry
on October 23, 2008 at 3:11 pm | Reply Dmitry

Also I’m using a GUI plug-in for tsung on called Sing-Tsung and it’s making things quite difficult on me :) My Record session xml reflects the correct steps but they’re not show correctly on my sing-tsung GUI it’s weird and I can’t figure out what’s making it do that. Wheh, is there like a tsung community where people help? I’ve went onto the process-one forum but not much help on there. :) Gald I found you.

Thanks a million
~Dmitry
on October 23, 2008 at 3:14 pm | Reply Dmitry

BTW sorry, but where do I turn on request logging?

~Dmitry
on October 23, 2008 at 3:35 pm | Reply Austin Mills

I’m not familiar with Sing-Tsung — looks interesting, but it might be introducing a layer of abstraction that’s making it difficult to troubleshoot. It might be worth trying it from the command line and seeing if there’s a difference.

As for a community, there’s a mailing list here which gets a decent amount of traffic, that’s probably the best place to look or ask: https://lists.process-one.net/mailman/listinfo/tsung-users

And sorry for not being more clear, but when I referred to request logging, I meant logging on the app server side (that is, not in Tsung), which almost all servers support, but the method of turning it on would be very server-dependent.
on October 23, 2008 at 3:46 pm | Reply Dmitry

Yeah I figured the request logging would be on the server side, I’ll give it a shot to try everything at command line.

Thanks agian.
Dmitry
on January 12, 2009 at 2:25 pm | Reply Chris

Silly question maybe, but how does Tsung work with the cloud of servers EC2 gives you? How exactly does the load get spread across all of them and your results collated?
on January 12, 2009 at 3:42 pm | Reply Austin Mills

Chris,

It depends on what your setup is with EC2. If you only have a single EC2 server, you just point Tsung to the external IP of that instance. If you have a loadbalancer and want to test the whole cluster, then you could point at the loadbalancer. EC2 by itself doesn’t provide load-balancing (until later this year, but there are open-source projects for this), so you’ll be using something external anyway, or running something like Pound/etc. on one of your instances.

In our case, instead of going through the loadbalancer like the one that we have set up for our production cluster, I went directly to a single EC2 instance. For us it’s reasonable to say that the performance of our system scales linearly (within certain bounds), so I can take the results from load-testing a single instance and feel pretty good that our N-instance cluster will provide N times that performance (up to a certain point, at least).

YMMV, based on the details of your setup, but you should definitely start with testing a single instance before looking at the performance of your whole system, so you have a context for the results. Then if the cluster results aren’t linearly improving over the single-node results, you have some things to look at. :)
on August 20, 2009 at 12:18 pm | Reply Ben

Great writeup, thank you.
Tsung now has Ubuntu packages, so you don’t need to compile. You can grab the latest deb at http://tsung.erlang-projects.org/dist/ubuntu/

Then just sudo dpkg -i on it.
on August 20, 2009 at 1:15 pm | Reply Austin Mills

Ben:

Great, thanks for the update!
on February 1, 2011 at 9:55 am | Reply Kevin Blaxland

Hello, I think your blog might be having browser compatibility issues. When I look at your blog site in Opera, it looks fine but when opening in Internet Explorer, it has some overlapping. I just wanted to give you a quick heads up! Other then that, fantastic blog!
on March 4, 2011 at 2:58 am | Reply Selvam

When I tried installing erlang using ‘sudo apt-get install erlang’, I get the following error.

Reading package lists… Done
Building dependency tree
Reading state information… Done
erlang is already the newest version.
The following packages will be upgraded:
ubuntu-sso-client
1 upgraded, 0 newly installed, 0 to remove and 130 not upgraded.
28 not fully installed or removed.
Need to get 0B/29.2kB of archives.
After this operation, 0B of additional disk space will be used.
(Reading database … 131079 files and directories currently installed.)
Preparing to replace ubuntu-sso-client 1.0.7-0ubuntu1 (using …/ubuntu-sso-client_1.0.8-0ubuntu1_all.deb) …
/var/lib/dpkg/info/ubuntu-sso-client.prerm: 6: update-python-modules: not found
dpkg: warning: subprocess old pre-removal script returned error exit status 127
dpkg – trying script from the new package instead …
/var/lib/dpkg/tmp.ci/prerm: 6: update-python-modules: not found
dpkg: error processing /var/cache/apt/archives/ubuntu-sso-client_1.0.8-0ubuntu1_all.deb (–unpack):
subprocess new pre-removal script returned error exit status 127
/var/lib/dpkg/info/ubuntu-sso-client.postinst: 28: update-python-modules: not found
dpkg: error while cleaning up:
subprocess installed post-installation script returned error exit status 127
Errors were encountered while processing:
/var/cache/apt/archives/ubuntu-sso-client_1.0.8-0ubuntu1_all.deb
E: Sub-process /usr/bin/dpkg returned an error code (1)

How can I resolve this?

Thank you