A short while ago I got the opportunity to compare the performance of a number of operating systems on the same piece of hardware.
The server in question was a HP DL580G5, but with four 6-core Intel Dunnington CPUs in, each running at 2.66ghz.Not wanting to spend my entire life running every form of benchmark known to man, I kept the exercise fairly simple. Run Primate Labs’ Geekbench with different core configurations to determine processor scaling of each operating system.
The exercise was ran on RedHat Enterprise Linux 4, RedHat Enterprise Linux 5, Solaris 10 x86 and Windows Server 2003r2. The 32bit and 64bit Geekbench tests were ran, with 2 cores enabled, then 4, 8, 12, 16, 20 and finally all 24 cores enabled.
I had my preconceptions about how this would turn out. I firmly expected Solaris to be bottom of the pile, with RHEL5 at the top and Windows and RHEL4 fighting it out for the middle ground. What actually popped out was quite a surprise…
Click on the graph for a larger version.
There was an odd scaling point there for Solaris, with little difference between 16 and 20 cores. I re-ran the tests just to make sure I hadn’t messed up the data, but it turned out the same again. I didn’t put any more effort into establishing why there was this anomaly, but I’m sure there must be some valid reason. This pattern just didn’t see to add up.
Look at those numbers again – Microsoft have been doing some good work with Windows to make it competitive with Linux, but the real gain is to be had with Solaris x86.
In every core count, besides the oddballs at 16 and 20, Solaris produces better numbers. It’s quicker at number crunching operations.
Benchmarks being benchmarks this, of course, doesn’t tell the full story. An Operating System should be benchmarked against the task you want to do – be it running a web server, database, or a piece of ISV software.
However, the one thing you shouldn’t do – as I would have in the past I’ll admit – is discount Solaris 10 x86. Not only does it have the best filesystem and debugging tool in the business these days, it clearly also has the groundings of the best performance. Ignore it at your own peril.
I’m curious why you would discount Solaris when looking at scalability? It’s the only OS in the list that has been running on huge multi-processor systems for decades so Sun have had quite a while to get it right ;o)
I work for an Enterprise where we do run a bit of everything (right OS for the job and all that), but with the core-count on x86 systems rapidly growing with Nehalem-EX/Magny-Cours, I’m expecting Solaris 10/Next to grow significantly on this platform (especially with the RAS features of EX introducing more SPARC-like capabilities).
Hi Tim,
Thanks for your comments. Is your opening question aimed at myself, or the populous in general? If the former – then I wouldn’t, and I didn’t mean to imply that I would. Quite the opposite. But for many years Solaris has been known as Slowaris for good reason (and I’m playing devils advocate here, my personal preference for server OS these days is always Solaris 10 x86), therefore to find it was the quickest in terms of real numbers was actually a very pleasant surprise.
I am pretty curious about your results. I did a similar test with a Sun x4600 recently and my results were not at all similar. We use an in house, bench with FFTW, SSL, SHA, SMTP, and large file break and recombine. I am going to reconfigure soon, I am interested in your configuration options. I installed Solaris with SUNWCore network configured and net services minimal. I installed Rhel 5.2 minimal, it takes two cd’s don’t know why, and configured network. I installed windows 2k8 core x64 and configured networking. I made sure to configure each machine as a replica of what we would run in production. I got amazing scaling on Windows up to 8 cores and diminished scaling up to 16 and then very little improvement after. On Linux the results were the best from 1 core to 16 cores, after 16 it didn’t scale as well, after 32 there were diminishing returns. On Solaris we say close performance up to 16 cores. At 16 cores Solaris was under by about 12%, Solaris eclipsed Linux at 22 cores and reigned supreme to 48. Next we installed Oracle on all platforms and the results were similar except the performance lagged on Solaris. Solaris didn’t win the database challenge until 27 or 28 cores. I didn’t get any really wierd result except when I installed Rhel4 with everything the performance was about half of the previously observed performance. It was power management and CPU scaling. I am interested in Geekbench, I just downloaded it and will need to see how close it replicates our results.
Hi Jerry,
All my OS installs were pretty stock – no tuning was carried out. They had minimal package sets, and not too many daemons running.
Geekbench is pretty tailored towards math tests too, so it’s not exactly ‘real world’ – unless you’re building a heavy compute platform.
Still, since shifting numbers around is fundamental to an OS I’d say it’s still a pertinent test.
I’d love to have the time to do much more detailed tests for sure.
main concern with geekbench is that executables are compiled with different compilers – e.g. old version of gcc on linux and version of suns studio compiler on solaris … how much of benchmark is showing which compiler/optimisations are best…
Very good point. It’s long been suggested Studio builds quick binaries on Solaris. If I’d had the time I would’ve done much more scientific benchmarking, this was a rather rough and ready exercise. The odd results for 16 and 20 cores on Solaris are suspect too – I actually went back and redid the tests again because of that anomaly and still got the same result.