Google compares performance of C++, Java, Scala, and Go - Java

Google's Robert Hundt just released a research paper titled “Loop Recognition in C++/Java/Go/Scala,” where he compared the performance of the four programming languages.  The release took place at a recent Silicon Valley conference, and it had some rather interesting results.

Out of the four languages, C++ is the winner when it comes to speed.  While speed is impressive, the paper noted some limitations regarding C++.  Most notably, it requires extensive tuning that surpasses the expertise of the average programmer.

On the flip side of C++ is Google's relatively new programming language, Go.  Google released the language on an experimental basis over a year ago with the hopes of increasing programmer productivity through clean and concise code.  Perhaps Go's young nature is its downfall, as it came in last on most of the performance tests.  Hundt discussed the relative youth of Go, as well as possible reasons for what could be deemed as lackluster performance in his paper.  “Go offers interesting language features, which also allow for a concise and standardized notation.  [But] the compilers for this language are still immature, which reflects in both performance and binary sizes."  One area where Go did not come in last was in compile time.  In fact, Go was the fastest of the four languages in compiling.  The benchmark did not measure concurrency, however, which both Go and Scala offer.  

Those interested in Scala's results would be pleased to know that it finished ahead of Java in terms of runtime.  It also offered more concise code and a smaller memory footprint.  With that being said, Scala still suffers some of Java's deficiencies.  Hundt discussed the two languages further: “"Scala['s] concise notation and powerful language features allowed for the best optimization of code complexity.  The Java version was probably the simplest to implement, but the hardest to analyze for performance. Specifically the effects around garbage collection were complicated and very hard to tune. Since Scala runs on the JVM, it has the same issues."

Google did its best to run a fair comparison through its benchmarks.  It first encoded a benchmark that corresponded to each language's container classes, looping constructs, and memory/object allocation schemes.  Google did this sans the use of any performance maximizing tools.  The reasoning behind this format was explained in the paper: “This approach allows an almost fair comparison of language features, code complexity, compilers and compile time, binary sizes, run-times, and memory footprint.” 

Once the results were obtained and published, Google engineers then optimized the benchmark for each language and completed another round of testing.  Hundt said, “While this effort is an anecdotal comparison only, the benchmark, and the subsequent tuning efforts, are indicative of typical performance pain points in the respective languages.”  C++ offered the most impressive runtime in the optimization stage, but optimizing its code was a complex task.

Martin Odersky, Scala's creator, offered his approval of the Google testing format: “I particularly like this idea to build a single algorithm from a non-expert point of view, compare the data, and ... then challenge people to optimize.  In a sense, it's fair."

For more on this topic, visit http://www.theregister.co.uk/2011/06/03/google_paper_on_cplusplus_java_scala_go/