Home arrow BrainDump arrow Benchmarking Tomcat Performance

Benchmarking Tomcat Performance

In this second part of a five-part article series on Tomcat performance tuning, you will learn how to do benchmarks so you can see what effects your changes produced. It is excerpted from chapter four of Tomcat: The Definitive Guide, Second Edition, written by Jason Brittain and Ian F. Darwin (O'Reilly; ISBN: 0596101066). Copyright © 2008 O'Reilly Media, Inc. All rights reserved. Used with permission from the publisher. Available from booksellers or direct from O'Reilly Media.

  1. Benchmarking Tomcat Performance
  2. Web Server Performance Comparison
  3. Benchmarked hardware and software configurations
  4. Building the APR Connector
By: O'Reilly Media
Rating: starstarstarstarstar / 3
February 19, 2009

print this article



Apache Jakarta JMeter

JMeter can be run in either graphical mode or in text-only mode. You may run JMeter test plans in either mode, but you must create the test plans in graphical mode. The test plans are stored as XML configuration documents. If you need to change only a single numeric or string value in the configuration of a test plan, you can probably change it with a text editor, but itís a good idea to edit them inside the graphical JMeter application for validityís sake.

Before trying to run JMeter to run a benchmark test against Tomcat, make sure that you start JMeterís JVM with enough heap memory so that it doesnít slow down while it does its own garbage collection in the middle of trying to benchmark. This is especially important if you are doing benchmark testing in graphical mode. In the bin/jmeter startup script, there is a configuration setting for the heap memory size that looks like this:

  # This is the base heap size -- you may increase or decrease it to fit your
  # system's memory availablity:
  HEAP="-Xms256m -Xmx256m"

It will make use of as much heap memory as you can give it; the more it has, the less often it may need to perform garbage collection. If you have enough memory in the machine on which youíre running JMeter, you should change both of the256numbers to something higher, such as512. It is important to do this first because this settingís default could skew your benchmark test results.

To create a test plan for the benchmark, first run JMeter in graphical mode, like this:

  $ bin/jmeter

JMeterís screen is laid out as a tree view on the left and a selection details panel on the right. Select something in the tree view and you can see the details of that item in the details panel on the right. To run any tests, you must assemble and configure the proper objects in the tree, and then JMeter can run the test and report the results.

To set up a benchmark test like the one we did above with both ab and siege, do this:

  1. In the tree view, right click on theTest Plantree node and selectAdd -> Thread Group
  2. In theThread Groupdetails panel, change theNumber of Threads (users)to149, change theRamp-Up Period (in seconds)to0, and theLoop Countto671.
  3. Right click on theThread Grouptree node and selectAdd -> Sampler -> HTTP Request.
  4. In the HTTP request details panel, change theWeb Serversettings to point to your Tomcat server and its port number, and change thePathunder theHTTP Requestsettings to the URI in your Tomcat installation that you would like to benchmark. For instance/.
  5. Right click on theThread Grouptree node again and selectAdd -> Post Processors -> Generate Summary Results.
  6. In the top pull-down menu, selectFile -> Save Test Plan as and type in the name of the test plan you wish to save. JMeterís test plan file extension is .jmx, which has an unfortunate similarity to the unrelated Java Management eXtension (JMX).

Figure 4-1 shows the JMeter GUI with the test plan, assembled and ready to run. The tree view is on the left, and the detail panel is on the right.

Once you are done building and saving your test plan, you are ready to run the benchmark. ChooseFile -> Exitfrom the top pull-down menu to exit from the graphical JMeter application. Then, run JMeter in text-only mode on the command line to perform the benchmark, like this:

  $ bin/jmeter -n -t tc-home-page-benchmark.jmx
  Created the tree successfully
  Starting the test
  Generate Summary Results = 99979 in 71.0s = 1408.8/s Avg:   38 Min:    0 Max:
  25445 Err:     0 (0.00%)
  Tidying up ...
  ... end of run

Notice that the requests per second reported by JMeter (an average of1408.8requests per second) is significantly lower than that reported by both ab and siege, for the same hardware, the same version of Tomcat, and the same benchmark. This demonstrates that JMeterís HTTP client is slower than that of ab and siege. You can use JMeter to find out if a change to your webapp, your Tomcat installation, or your JVM, accelerates or slows the response times of web pages; however, you cannot use

Figure 4-1.  Apache JMeter GUI showing the fully assembled test plan

JMeter to determine the serverís maximum number of requests per second that it can successfully serve because JMeterís HTTP client appears to be slower than Tomcatís server code.

You may also graph the test results in JMeter. To do this, run JMeter in graphical mode again, then:

  1. Open the test plan you created earlier.
  2. In the tree view, select theGenerate Summary Resultstree node and delete it (one easy way to do this is to hit the delete key on your keyboard once).
  3. Select theThread Grouptree node, then right click on it and selectAdd -> Listener -> Graph Results.

  4. Save your test plan under a new name; this time for graphical viewing of test results.

  5. Select theGraph Resultstree node.

Now, youíre ready to rerun your test and watch as JMeter graphs the results in real time.

Again, make sure that you give the JMeter JVM enough heap memory so that it does not run its own garbage collector often during the test. Also, keep in mind that the Java VM must spend time graphing while the test is running, which will decrease the accuracy of the test results. How much the accuracy will decrease depends on how fast the computer youíre running JMeter on is (the faster the better). But, if youíre just graphing to watch results in real time as a test is being run, this is a great way to observe.

When youíre ready to run the test, you can either selectRun -> Start from the top pull-down menu, or you can hit Ctrl-R. The benchmark test will start again, but you will see the results graph being drawn as the responses are collected by JMeter. Figure 4-2 shows the JMeter GUI graphing the test results.

Figure 4-2.  Apache JMeter graphing test results

You can either let the test run to completion or you can stop the test by hitting Ctrl-. (hold down the Control key and hit the period key). If you stop the test early, it will likely take JMeter some seconds to stop and reap all of the threads in the requestThread Group. To erase the graph before restarting the test, hit Ctrl-E. You can also erase the graph in the middle of a running test, and the test will continue on, plotting the graph from that sample onward.

Using JMeter to graph the results gives you a window into the running test so you can watch it and fix any problems with the test and tailor it to your needs before running it on the command line. Once you think you have the test set up just right, save a test plan that does notGraph Results, but has a
Generate Summary Resultstree node so that you can run it on the command line, and then save the test plan again under a new name that conveys the kind of test it is and that it is configured to be run from the command line. Use the results you obtain on the command line as the authoritative results. Again, the ab benchmark tool gives you more accurate benchmark results but does not offer as many features as JMeter.

JMeter also has many more features that may help you test your webapps in numerous ways. See the online documentation for more information about this great test tool at http://jakarta.apache.org/jmeter.

>>> More BrainDump Articles          >>> More By O'Reilly Media

blog comments powered by Disqus
escort Bursa Bursa escort Antalya eskort


- Apple Founder Steve Jobs Dies
- Steve Jobs` Era at Apple Ends
- Google's Chrome Developer Tool Updated
- Google's Chrome 6 Browser Brings Speed to th...
- New Open Source Update Fedora 13 is Released...
- Install Linux with Knoppix
- iPad Developers Flock To SDK 3.2
- Managing a Linux Wireless Access Point
- Maintaining a Linux Wireless Access Point
- Securing a Linux Wireless Access Point
- Configuring a Linux Wireless Access Point
- Building a Linux Wireless Access Point
- Migrating Oracle to PostgreSQL with Enterpri...
- Demystifying SELinux on Kernel 2.6
- Yahoo and Microsoft Create Ad Partnership

Developer Shed Affiliates


Dev Shed Tutorial Topics: