Archive for September, 2008

After trying some other load-testing tools, I found Tsung, a load-testing application written in Erlang to take advantage of that language’s concurrency support. It scales well (it’s been used to simulate tens of thousands of users), it supports forms and HTTP sessions, and includes some niceties like proxy recording, ‘thinktime’ support, and a choice of random or ordered traffic.

Not having a physical server handy to run Tsung isn’t a big obstacle; I just used an Amazon EC2 instance (an xlarge turned out to be more than enough, you could probably get 100s of users on a medium), which due to being rentable by the hour is great for a short-term need like occasional load-testing. I put one of Alestic’s Ubuntu images on the EC2 instance, and then it’s just a few short steps to install Tsung on the machine (we’ll call it ‘tsunghost’):

  1. sudo apt-get install erlang
  2. sudo apt-get install erlang-src
  3. sudo apt-get install gnuplot-nox sudo apt-get install libtemplate-perl libhtml-template-perl libhtml-template-expr-perl
  4. Download the latest from http://tsung.erlang-projects.org/dist/ and install it (the usual configure, make, make install). I prefixed it into /opt/tsung just to keep it a little cleaner, so the following will reflect that.

Update: Ben says in the comments, “Tsung now has Ubuntu packages, so you don’t need to compile. You can grab the latest deb at http://tsung.erlang-projects.org/dist/ubuntu/ .  Then just sudo dpkg -i on it.”

The first thing you’ll want to do is record a sample web session. Start Tsung in record mode with ‘/opt/bin/tsung -L 9000 recorder’ (the -L bit indicates the port number that the proxy will listen on). Now, you can set tsunghost:9000 as the HTTP proxy in your web browser and all the web activity will be recorded by Tsung for playback later.

With Tsung recording, exercise your web application — but keep in mind that all of your actions will be repeated later, so for example, don’t make comments on a web page that only allows a max of 10 comments, or else you’ll get a high percentage of errors when it fails every time it’s hit after the 10th.

When you’re done with your web session, stop Tsung with ‘/opt/bin/tsung stop_recorder’. If you look in your home directory, you should now have a .tsung directory with a log of your session, something like ~/.tsung/tsung_recorderYYYMMDD-HH:MM.xml. That file contains the log of all the activity you did while sending traffic through the Tsung proxy, and will be used to drive the simulated user sessions during the load testing.

With the session file in hand, we can create a very simplified file to drive Tsung’s testing. There are some examples that were installed for you at /opt/tsung/share/doc/tsung/examples, but even the simplest example was more complicated than what I ended up using for my testing.

The only thing you must change here is the path to the session file that you recorded earlier, and possibly the hostname — although in my case it appeared to be overridden by the host defined in the session file. Apart from that, you might want to configure some details of the load testing, and set a maxusers value that’s somewhat reasonable for the test you have in mind.

  <!DOCTYPE tsung SYSTEM "/opt/tsung/share/tsung/tsung-1.0.dtd" [  ] >

  <!-- set dumptraffic="true" to dump all received and sent packets -->
  <!-- set loglevel="debug" for maximum verbosity -->

    <!-- Client side setup -->
      <!-- maxusers is the max number of simultaneous clients. Don't set it too high because you can run out of file descriptors. Remember that ssl_esock use 2 fds by connection. -->

    <!-- Server side setup -->

    <!-- several arrival phases can be set: for each phase, you can set the mean inter-arrival time between new clients and the phase duration -->

    <!-- Options. -->
     <!-- Thinktime value overrides those set in the session nodes if override=true. -->

      <!-- HTTP parameters -->
        Mozilla/5.0 (Windows; U; Windows NT 5.2; fr-FR; rv:1.7.8) Gecko/20050511
    <!-- ********************************************** --> <!-- start your recorded session --> <!-- ********************************************** -->

In this case, we’re using clients on the local machine only, with a new user arriving every 5 seconds for 10 minutes, and with each user pausing for 3 seconds between requests to simulate thinking time. All clients claim to be Firefox, but that didn’t really matter to me — you may want to change it depending on how much browser-specific code you have.

When you have your tsung.xml file the way you like it, start Tsung with /opt/tsung/bin/tsung start. It will start and run until the last user session has completed (so a test defined to run for 10 minutes might take much longer than 10 minutes to complete if the server was heavily loaded). When the test is completed, Tsung will exit and write out a log file in ~/.tsung/log with the timestamp included in the filename.

Tsung has a tool included to generate HTML formatted reports that summarize the test, but they have to be generated by running /opt/tsung/lib/tsung/bin/lib_stats.pl in that test run’s directory. It will generate several html files and charts in png and ps format. Open up report.html or graphs.html in a browser to view an array of data from the test run. The Tsung home page has some samples of what the report will look like.

One last thing — by default, the charts are pretty low-res PNG files, but if you want to, you can create higher-resolution images from the PS (PostScript) files that each chart links to using a tool like GSView.

As resources, both the Tsung documentation and this post by 21 croissants were very useful to me as I was getting started.

Read Full Post »

So, if you were going to do some load-testing on your Rails app, you might think, hey, Siege is pretty cool. It supports load-testing multiple URLs at once (either sequential or in random order), with a delay in-between, with lots of options for setting the duration and characteristics of the load. Plus, there’s a tool that works with Siege, Sproxy, that lets you generate that list of URLs automatically by using Sproxy as a web proxy, recording your actions on the site.

Siege doesn’t support sessions (of course, that’s not clear from the docs, but I assure you it’s the case), which at first might be a deal-breaker if your app handles security by authenticating the session… but then you remember that you can pass in the _session_id parameter, and think all will be well.

Complication #1: In recent Rails versions, the session id can only be pulled from the cookie. However, we remember that you can set

ActionController::CgiRequest::DEFAULT_SESSION_OPTIONS[:cookie_only] = false

And allow the session id to be taken from the parameter as well. However, if you’re running Rails 2, this won’t work. See Complication #2.

Complication #2: Thanks to a fix intended to prevent session fixation attacks, but which had some unintended consequences, the ability to override the cookie_only setting is broken. There is a monkey patch attached as a comment to that bug, but that’s not something I want to do when load-testing code. I do feel bad for the people out there (mainly mobile browsers and other non-traditional clients) that depend on parameter-passed session IDs.

So, time to look for a new tool, initial candidates are httperf/autobench, Flood, ab, and Tsung.

Thanks to JL2003 for the image.

Read Full Post »