Apache Jakarta JMeter

JMeter can be run in either graphical mode or in text-only mode. You may run JMeter test plans in either mode, but you must create the test plans in graphical mode. The test plans are stored as XML configuration documents. If you need to change only a single numeric or string value in the configuration of a test plan, you can probably change it with a text editor, but it’s a good idea to edit them inside the graphical JMeter application for validity’s sake.

Before trying to run JMeter to run a benchmark test against Tomcat, make sure that you start JMeter’s JVM with enough heap memory so that it doesn’t slow down while it does its own garbage collection in the middle of trying to benchmark. This is especially important if you are doing benchmark testing in graphical mode. In the bin/jmeter  startup script, there is a configuration setting for the heap memory size that looks like this:

  # This is the base heap size -- you may increase or decrease it to fit you r
  # system's memory availablity:
  HEAP="-Xms256m -Xmx256m"

It will make use of as much heap memory as you can give it; the more it has, the less often it may need to perform garbage collection. If you have enough memory in the machine on which you’re running JMeter, you should change both of the 256 num bers to something higher, such as 512 . It is important to do this first because this setting’s default could skew your benchmark test results.

To create a test plan for the benchmark, first run JMeter in graphical mode, like this:

  $ bin/jmeter

JMeter’s screen is laid out as a tree view on the left and a selection details panel on the right. Select something in the tree view and you can see the details of that item in the details panel on the right. To run any tests, you must assemble and configure the proper objects in the tree, and then JMeter can run the test and report the results.

To set up a benchmark test like the one we did above with both ab  and siege , do this:

1. In the tree view, right click on the Test Plan tree node and select Add -> Thread Group . 
   
2. In the Thread Group details panel, change the Number of Threads (users) to 149 , change the Ramp-Up Period (in seconds) to 0 , and the Loop Count to 671 .
   
3. Right click on the Thread Group tree node and select Add -> Sampler -> HTTP Request .
   
4. In the HTTP request details panel, change the Web Server settings to point to your Tomcat server and its port number, and change the Path under the HTTP Request settings to the URI in your Tomcat installation that you would like to benchmark. For instance / .
   
5. Right click on the Thread Group tree node again and select Add -> Post Processors -> Generate Summary Results .
   
6. In the top pull-down menu, select File -> Save Test Plan as and type in the name of the test plan you wish to save. JMeter’s test plan file extension is .jmx, which has an unfortunate similarity to the unrelated Java Management eXtension (JMX).

Figure 4-1 shows the JMeter GUI with the test plan, assembled and ready to run. The tree view is on the left, and the detail panel is on the right.

Once you are done building and saving your test plan, you are ready to run the benchmark. Choose File -> Exit from the top pull-down menu to exit from the graphical JMeter application. Then, run JMeter in text-only mode on the command line to perform the benchmark, like this:

  $ bin/jmeter -n -t tc-home-page-benchmark.jmx
  Created the tree successfully
  Starting the test
  Generate Summary Results = 99979 in 71.0s = 1408.8/s Avg:   38 Min:    0 Max:
  25445 Err:     0 (0.00%)
  Tidying up ...
  ... end of run

Notice that the requests per second reported by JMeter (an average of 1408.8 requests per second) is significantly lower than that reported by both ab  and siege , for the same hardware, the same version of Tomcat, and the same benchmark. This demonstrates that JMeter’s HTTP client is slower than that of ab and siege . You can use JMeter to find out if a change to your webapp, your Tomcat installation, or your JVM, accelerates or slows the response times of web pages; however, you cannot use

Figure 4-1.  Apache JMeter GUI showing the fully assembled test plan

JMeter to determine the server’s maximum number of requests per second that it can successfully serve because JMeter’s HTTP client appears to be slower than Tomcat’s server code.

You may also graph the test results in JMeter. To do this, run JMeter in graphical mode again, then:

1. Open the test plan you created earlier.
   
2. In the tree view, select the Generate Summary Results tree node and delete it (one easy way to do this is to hit the delete key on your keyboard once).
   

3. Select the Thread Group tree node, then right click on it and select Add -> Listener -> Graph Results .

4. Save your test plan under a new name; this time for graphical viewing of test results.

5. Select the Graph Results tree node.

Now, you’re ready to rerun your test and watch as JMeter graphs the results in real time.

Again, make sure that you give the JMeter JVM enough heap memory so that it does not run its own garbage collector often during the test. Also, keep in mind that the Java VM must spend time graphing while the test is running, which will decrease the accuracy of the test results. How much the accuracy will decrease depends on how fast the computer you’re running JMeter on is (the faster the better). But, if you’re just graphing to watch results in real time as a test is being run, this is a great way to observe.

When you’re ready to run the test, you can either select Run -> Start from the top pull-down menu, or you can hit Ctrl-R. The benchmark test will start again, but you will see the results graph being drawn as the responses are collected by JMeter. Figure 4-2 shows the JMeter GUI graphing the test results.

Figure 4-2.  Apache JMeter graphing test results

You can either let the test run to completion or you can stop the test by hitting Ctrl-. (hold down the Control key and hit the period key). If you stop the test early, it will likely take JMeter some seconds to stop and reap all of the threads in the request Thread Group . To erase the graph before restarting the test, hit Ctrl-E. You can also erase the graph in the middle of a running test, and the test will continue on, plotting the graph from that sample onward.

Using JMeter to graph the results gives you a window into the running test so you can watch it and fix any problems with the test and tailor it to your needs before running it on the command line. Once you think you have the test set up just right, save a test plan that does not Graph Results , but has a
Generate Summary Results tree node so that you can run it on the command line, and then save the test plan again under a new name that conveys the kind of test it is and that it is configured to be run from the command line. Use the results you obtain on the command line as the authoritative results. Again, the ab  benchmark tool gives you more accurate benchmark results but does not offer as many features as JMeter.

JMeter also has many more features that may help you test your webapps in numerous ways. See the online documentation for more information about this great test tool at http://jakarta.apache.org/jmeter .