Understanding the impact of using virtualized servers instead of real ones is perhaps one of the most complex issues when migrating from a traditional configuration to a cloud-based setup. Especially because virtualized servers are created equal … but only on paper.
A Rackspace-funded “report” tries to find out the performance differences between Rackspace Cloud Servers and Amazon EC2. I guess the only conclusion we can get from their so-called report is that Cloud Server disk throughput is better than EC2’s. As the “CPU test” is a kernel compile which also stresses the disk, I don’t think we can reliably get any conclusion from these.
An intrepid commenter ran a CPU-only test (Geekbench) and found out that EC2 performs slightly better than Rackspace in terms of raw processor performance. The same commenter, affiliated with Cloud Harmony, mentions that a simple hdparm test shows that Rackspace hdd has more than twice the throughput of EC2 hdd, at least in terms of buffered reads. Last but not least, don’t forget that for better disk performance Amazon recommends EBS instead of the VM disk.
We cannot reliably make an informed cloud vendor choice just using VM benchmarks. Ideally, you should benchmark your own app on each cloud infrastructure and choose the one which gives you the best user-facing performance, because at the end of the day this is what matters most. Sadly, today this means experimenting with sometimes wildly different APIs and provisioning models.
You are wrong, kernel compilation is not I/O intensive. You might want to be qualified in computing before spreading inaccurate assumptions and revealing your ignorance in the matter.
In many cases EBS will get worse performance than local instance storage for disk I/O.
Not to mention, the GeekBench folks were not using industry standard utilities, but their own in-house benchmarking utility. The validity of a software benchmark called GeekBench that is supposedly cross-platform should be questioned. Scheduling and device drivers work differently across operating systems.
The amount of inaccuracy in your assessment probably means that you shouldn’t make them, at least until you get a little more experience with the field. You are right about one thing though, benchmarking the cloud is not simple.
Matthew
31 Jan 10 at 3:33 pm
Hi Matthew, thank you for visiting my blog and for commenting. Here’s my point.
While not I/O intensive, the kernel compilation still needs to fetch a reasonably large amount of sources from the disk and write back different compilation artifacts in the process. Therefore, a slow disk [especially a very slow one] can skew the results -> kernel compilation is not 100% relevant as a CPU benchmark. Couldn’t find the exact point where I said it’s “I/O intensive” in my article, can you please point the erroneous phrase so I can fix it?
Please also note I’m just aggregating data from different sources, so you may want to consider these statements as pertaining to their respective authors, not to *me*.
Please feel free to come up with more relevant information and point to the place where we could see the “more accurate” data you are referring to.
I would appreciate in the future if you could avoid ad hominem attacks and stick to the facts! Thanks.
Adrian
1 Feb 10 at 10:06 am