Tomcat Benchmark Procedure - Benchmark results and summary continued
( Page 3 of 4 )

 

Does the dual EM64T again serve roughly double the number of requests per second as the single processor AMD64 when serving the image files? No. For some reason, it’s more like four times the number of requests per second. How could it be possible that by adding one additional processor, the computer can do four times the work? It probably can’t. The only explanation we can think of is that something is slowing down the AMD64 laptop’s ability to serve the image files to the processor’s full potential. This isn’t necessarily a hardware problem; it could be that a device driver in this version of the kernel is performing inefficiently and slowing down the benchmark. This hints that the benchmark results for the 9k image benchmark on the AMD64 computer may not be accurate due to a slow driver. However, this is the observed performance on that computer. Until and unless a different kernel makes it perform better, this is how it will perform. Knowing that, it is unclear whether Tomcat or Apache httpd  is faster serving the 9k image files, although we would guess that the EM64T benchmark results are more accurate.

Does the dual EM64T again serve roughly double the number of requests per second as the single processor AMD64 when serving the image files? No. For some reason, it’s more like four times the number of requests per second. How could it be possible that by adding one additional processor, the computer can do four times the work? It probably can’t. The only explanation we can think of is that something is slowing down the AMD64 laptop’s ability to serve the image files to the processor’s full potential. This isn’t necessarily a hardware problem; it could be that a device driver in this version of the kernel is performing inefficiently and slowing down the benchmark. This hints that the benchmark results for the 9k image benchmark on the AMD64 computer may not be accurate due to a slow driver. However, this is the observed performance on that computer. Until and unless a different kernel makes it perform better, this is how it will perform. Knowing that, it is unclear whether Tomcat or Apache   is faster serving the 9k image files, although we would guess that the EM64T benchmark results are more accurate.

Here is a summary of the benchmark results, including some important stats:

1. Tomcat standalone was faster than Apache httpd compiled for worker MPM in all of our benchmark tests except the 9k image benchmark test on Intel 64-bit Xeon, and even in that benchmark, httpd  was only 1 percent faster than Tomcat. We observed that Tomcat standalone JIO was almost always the fastest way to serve static resources. Tomcat served them between 3 percent and 136 percent faster than Apache httpd  in our benchmarks—Tomcat standalone JIO was a minimum of 3 percent faster than Apache httpd  (worker MPM) for 9k image files, except for the Intel 64-bit Xeon benchmark, where httpd  appeared to perform 1 percent faster than Tomcat. But in the small files benchmark, Tomcat was a minimum of 99 percent faster than Apache httpd  and a maximum of 136 percent faster than Apache httpd .
   
2. Apache httpd  built to use worker MPM was the fastest configuration of Apache httpd  we tested; Apache httpd  built to use prefork MPM was slower than worker MPM in all of our standalone tests. We observed worker MPM serving a minimum of 0.4 percent faster than prefork MPM and a maximum of 26 percent faster than prefork MPM. There was almost no difference in performance between the two in our small text files benchmarks, but in the 9k image files benchmark, the difference was at least 22 percent.
   
3. Tomcat standalone (configured to use any HTTP connector implementation) was always faster than Apache httpd  built and configured for prefork MPM; Tomcat standalone was a minimum of 21 percent faster than Apache httpd  and a maximum of 30 percent faster than Apache httpd  for 9k image files, and for small files Tomcat was a minimum of 103 percent faster than Apache httpd and a maximum of 136 percent faster than Apache httpd  prefork MPM.
   
4. Apache httpd was quite a bit slower at serving small files. Tomcat standalone’s JIO, APR, and NIO connectors were each faster than Apache httpd —Tomcat’s JIO connector performed as much as 136 percent faster than Apache
   httpd ’s fastest configuration, Tomcat’s APR connector performed 89 percent faster than Apache httpd, and Tomcat 6.0’s NIO connector performed 25 percent faster than Apache httpd . In this common use case benchmark, Apache httpd  dropped to fourth place behind all of Tomcat standalone’s three HTTP connectors.
   
5. Serving Tomcat’s resources through Apache httpd  was very slow compared to serving them directly from Tomcat. When we compared the benchmark results between Tomcat standalone and Tomcat serving through Apache httpd via mod_ proxy , Tomcat standalone consistently served at least 51 percent faster when using only Tomcat’s JIO connector without Apache httpd . (including all three Apache httpd connector modules: mod_jk , mod_proxy_ajp , and mod_proxy_ http ). In the small text files benchmark, Tomcat standalone was a minimum of 168 percent faster than the Apache httpd  to Tomcat configurations and a maximum of 578 percent faster! That’s not a misprint—it’s really 578 percent faster. For the 9k image files benchmark, Tomcat standalone was at least 51 percent faster and at most 274 percent faster.
   
6. AJP outperformed HTTP when using mod_proxy . The benchmark results show that mod_proxy_ajp  was consistently faster than mod_proxy_http . The margin between the two protocols was as low as 1 percent and as high as 30 percent when using the same Tomcat connector design, but it was usually smaller, with mod_proxy_ajp  averaging about 13 percent faster than mod_proxy_http .
   
7. Serving Tomcat’s static resources through an Apache httpd  connector module was never faster than serving the same static resources through just Apache httpd by itself. The benchmark results of serving the resources through an httpd connector module (from Tomcat) were always somewhat slower than just serving the static resources straight from Apache httpd . This means that benchmarking Apache httpd  standalone will tell you a number slightly higher than the theoretical maximum that you could get by serving the same resource(s) through an httpd  connector module. This also means that no matter how performant Tomcat is, serving its files through Apache httpd throttles Tomcat down so that Tomcat is slower than Apache httpd .
   
8. mod_jk  was not faster than mod_proxy, except in the 9k image benchmark and then only on AMD64. In our tests, serving Tomcat’s resources through Apache httpd via mod_jk  was only faster than using mod_proxy  on the AMD64 laptop and only in the 9k image benchmark. In all the other benchmarks, mod_jk  was slower than mod_proxy_ajp .

How is it possible for pure-Java Tomcat to serve static resource faster than Apache httpd ? The main reason we can think of: because Tomcat is written in Java and because Java bytecode can be natively compiled and highly optimized at runtime, well-written Java code can run very fast when it runs on a mature Java VM that implements many runtime optimizations, such as the Sun Hotspot JVM. After it runs and serves many requests, the JVM knows how to optimize it for that particular use on that particular hardware. On the other hand, Apache httpd  is written in C, which is completely compiled ahead of runtime. Even though you can tell the compiler to heavily optimize the binaries, no runtime optimizations can take place. So, there is no opportunity with Apache httpd  to take advantage of the many runtime optimizations that Tomcat enjoys.

Another potential reason Tomcat serves the web faster than Apache httpd  is that every release of Sun’s JVM seems to run Java code faster, which has gone on for many release cycles of their JVM. That means that even if you’re not actively changing your Java program to perform better, it will likely keep improving every time you run it on a newer, faster JVM if the same progress on JVM performance continues. This does, however, make the assumption that newer JVMs will be compatible enough to run your Java program’s bytecode without any modifications.

>>> More BrainDump Articles          >>> More By O'Reilly Media