Siege - BrainDump

To use siege to perform exactly the same benchmark, the command line is similar, only you must give it the number of requests you want it to make per thread. If you’re trying to benchmark 100,000 HTTP requests, with 149 concurrent clients, you must tell siege that each of the 149 clients needs to make 671 requests (as 671 requests times 149 clients approximately equals 100,000 total requests). Give siege  the-bswitch, telling siege that you’re running a benchmark test. This makes siege’s client threads not wait between requests, just like ab. By default, siege does wait a configurable amount of time between requests, but in the benchmark mode, it does not wait. Example 4-3 shows the siege command line and the results from the benchmark test.

Example 4-3. Benchmarking with siege with keep-alive connections disabled

$ siege -b -r 671 -c 149 tomcathost:8080
** siege 2.65
** Preparing 149 concurrent users for battle.
The server is now under siege..      done. Transactions:                  99979 hits Availability:                 100.00 % Elapsed time:                  46.61 secs Data transferred:             775.37 MB Response time:                  0.05 secs Transaction rate:            2145.01 trans/sec
Throughput:                    16.64 MB/sec Concurrency:                  100.62 Successful transactions:       99979
Failed transactions:               0
Longest transaction:           23.02 Shortest transaction:           0.00

Some interesting things to note about siege’s results are the following:

• The number of transactions per second that were completed by siege is significantly lower than that of ab. (This is with keep-alive connections turned off in both benchmark clients,* and all of the other settings the same.) The only explanation for this is that siege  isn’t as efficient of a client as ab is. And that points out that siege’s benchmark results are not as accurate as those of ab.
  
• The throughput reported by siege is significantly lower than that reported by ab, probably due to siege not being able to execute as many requests per second as ab.
  
• The reported total data transferred with siegeis approximately equal to the total data transferred with ab.
  
• ab completed the benchmark in slightly more than half the time that siegecompleted it in; however, we do not know how much of that time siege spent between requests in each thread. It might just be that siege’s request loop is not as optimally written to move on to the next request.

For obtaining the best benchmarking results, we recommend you use ab instead of siege. However, for other kinds of testing when you must closely simulate web traffic from human users, ab is not suitable because it offers no feature to configure an amount of time to wait between requests. Siege does offer this feature in the form of waiting a random amount of time between requests. In addition to that, siege can request random URLs from a prechosen list of your choice. Because of this, siege can be used to simulate human user load whereas ab cannot. See the siege manual page (by running “man siege”) for more information about siege’s features.

Please check back next week for the continuation of this article.