Benchmarking PHP Frameworks - PHP

Benchmarking Methodology

First Test: Hardware and software used:
Mac OS X/10.6.7
Processor 2.5 GHz Intel Core Duo
Memory 2 GB 667 MHz DDR2 SDRAM
Macintosh HD
Apache/2.0.64
PHP/5.3.5

Second Test: Hardware and software used:
Intel Pentium4 2.4 Ghz Processor
2GB RAM
Linux/Ubuntu 10.04 Operating system
Apache server version (XAMPP): Apache/2.2.14 (Unix) DAV/2 mod_ssl/2.2.14 OpenSSL/0.9.8l PHP/5.3.1 mod_apreq2-20090110/2.7.1 mod_perl/2.0.4 Perl/v5.10.1
Local test folders in encrypted sector.

Load Test Factors:

In all of the tests, we will be using the Apache Bench utility (ab). To simulate a practical scenario the following load testing factors will be employed:

Number of Connections: 500
Number of Concurrent users: 50

Note: No caching implemented.

Methods employed for First Test:

Step 1.) Download these benchmarking files to be used in this article: http://bit.ly/inWQQa or this URL: http://cid-c3bc6a3c5463e218.office.live.com/self.aspx/.Public/php%5E_framework%5E_test.zip

Step 2.) Right click and extract them.
Step 3.) Copy all of those files to your test server. Make sure the path is right; for example benchmark.php must be accessible as http://localhost/benchmark.php in your browser. Below are the localhost path of each of these frameworks to be tested:

Cakephp-1.3.7: http://localhost/cakephp-1.3.7/index.php/hello
CodeIgniter_2.0.0: http://localhost/CodeIgniter_2.0.0/index.php/
Symfony 2: http://localhost/symfony-2.0/web/app.php/hello 
yii-1.1.6.r2877: http://localhost/yii-1.1.6.r2877/demos/helloworld/
RainFramework 2.3: http://localhost/rainframework%202.3/ajax.php/hello
Doophp 1.4.1: http://localhost/doophp-1.4.1/app/
Kohana-3.1: http://localhost/kohana-3.1/index.php/welcome

Step 4.) Before running the benchmark, try executing all of the above paths provided above in your browser to make sure they output “Hello World”.

Step 5.) If the output is OK, you are now ready to start the benchmark - clear all the browser cache and history. Close the browser, then restart again. Close all other running applications in your computer. Make sure only this test is running.

Step 6.) Copy and paste this URL to the browser: http://localhost/benchmark.php

It will take some time, because there are around 7 frameworks to be tested. Wait until the script provides some output.

Methods employed for the Second Test:

Step 1.) Follow Step 1 to 6 of the First step procedure.
Step 2.) Do 3 trials for each framework, then reboot the server.
Step 3.) Average the results of the 3 trials.