Suppose a test is done comparing the speed between two possible approaches in coding PHP that yields the following average results: Test 1 average execution time = 630 Microseconds You cannot simply conclude and recommend that the Test 2 approach is faster and better simply because it has a lower average or mean (which means faster). Numerically, 602 microseconds < 630 microseconds and the difference between the results is 630 – 602 = 28 microseconds. The main question, however, that is of so much importance to PHP developers/coders/programmers is this: “Does the difference of 28 microseconds really matter in terms of performance?” or stated another way: “Does the difference of 28 microseconds really significantly improve the script in terms of speed?”. What you are facing every time you code are a lot of suggestions on how to optimize speed, but these may not be really significant at all. Instead of making your program run faster and more efficient, it actually contributes too little to be considered worthwhile. In some cases it can even make your program more complex and harder to maintain. According to Donald Knuth: http://en.wikipedia.org/wiki/Donald_Knuth “Programmers waste enormous amounts of time thinking about, or worrying about, the speed of noncritical parts of their programs, and these attempts at efficiency actually have a strong negative impact when debugging and maintenance are considered. We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil. Yet we should not pass up our opportunities in that critical 3%” The Scientific Method: Comparing Performance The most reliable methods of comparing performance are those that are used by scientists and engineers in their studies. For small samples (let's say 10 observations), you can use a Student's ttest method to objectively compare performance. This is a form of statistical testing: References: The following are steps on how to conduct the test: Step 1: Set up the test scripts in a localhost environment. Since most web hosts use PHP 5 nowadays (as of July 2011), PHP 5 is used throughout this test. Specifically, below is the system used: Operating system: Ubuntu 10.04 To make these tests relate to other previously done tests, we will be using similar benchmarking scripts such as thoughs used by http://www.phpbench.com. Step 2: Clear cache before running any test in the browser. Echo the execution time in the browser and gather the data. Step 3: Repeat the test 10 times to get 10 observations. Step 4: Analyze the data using ttest. To speed up the analysis, an online tool will be used, such as the one found here: http://www.physics.csbsju.edu/stats/ttest_bulk_form.html Establish the following hypothesis':
Rules for rejecting Null hypothesis: Reject null hypothesis if the probability of the result is less than 0.05 (using a 95% confidence level). This level of confidence has only around a 5% chance of error. Illustration #1: PHP Echo vs PHP Print One of the most commonly asked questions pertaining to PHP speed practices is: “Which is faster: PHP echo or PHP print? Is the speed differences significant?" Sample Test code:
For print, simply replace:
With:
Group B Execution time (in seconds) for print (10 observations):
Ttest Analysis Results: t= 2.54 Degrees of freedom = 18. The probability of this result, assuming the null hypothesis, is 0.020 Box plot of mean (average): Conclusion and Recommendation Based on the analysis results, you can conclude that php echo is indeed significantly faster than print at a 95% confidence level. Based on the box plot, the execution time of echo is significantly lower and more stable than print (which has wider spread). As a recommendation, you should use PHP echo when outputting HTML to the browser to get speed benefits. Illustration #2: Single vs Double Quotes When Outputting Suppose you want to find out which one is faster: Group A: Single quote and concatenated variable
Group B: Or simply this (double quotes and enclosing the variable):
Sample Test Code:
Group A Execution time in seconds:
Group B Execution time in seconds:
Ttest Analysis Results: t= 7.94 The mean of Group A is: 579.214 microsecond Degrees of freedom = 18. The probability of this result, assuming the null hypothesis, is less than .0001. The probability of this result, assuming the null hypothesis, is less than .0001 Box plot of mean (average): Conclusion and Recommendation Using single quotes is faster than double quotes; indeed, when you look at their box plot, the mean and variation of execution time using single quotes is small compared to double quotes. As a recommendation to output something to the browser, it is faster to use single quotes and the concatenation method at a 95% confidence level. Illustration #3: For Loop vs While Loop in PHP Both for and while do loops are commonly used looping methods in PHP programming. But which one is really faster? Test code used:
Simply replace:
With:
When evaluating For loops. Group A (While loop) data:
Group B (For Loop) data:
Ttest analysis result: t=0.371 The mean of Group A is: 0.136 Second Degrees of freedom = 18. The probability of this result, assuming the null hypothesis, is 0.71 Box plot of mean (average): Conclusion and Recommendation This is a classic example of why you cannot judge just by looking at the averages or mean. Using the “while loop” is 0.136 second while the “For loop” is 0.141 second. The “While loop” is only faster by around 0.005 seconds. It is evident by their box plots being so similar. According to ttest analysis results, the probability is 0.71  which is greater than 0.05. Thus, you cannot reject the null hypothesis and can conclude that there no significant difference in terms of speed. As a recommendation, you can use either “while” and “for loops” in your code without worrying about its efficiency in terms of speed performance. If a specific section in your code requires a “while loop”, then use it, otherwise use a “for loop”. It won't slow anything down. Summary There are still a lot of remaining issues in PHP regarding speed benchmarks. By following the methods outlined in this article, a PHP developer or researcher can reliably determine whether a certain change in code can significantly contribute to speed or not.
blog comments powered by Disqus 






