• Correct something wrong on “the internets”
• Produce a little bit of graphviz candy (what a fantastic tool)

Imagine my total happiness discovering that I can exercise both of them simultaneously. I can only thank Digg’s John Quinn for creating the opportunity in his otherwise very interesting presentation, by including this directed graph on one of the slides:

You would get the impression that besides Cassandra the other “NoSQL” solutions have a single major website using them. I cannot pronounce for all the products with their logos embedded in the graph nodes, but for HBase this is somewhat incorrect. Starting from the Powered By page for HBase I thought it’d be a good idea to fill in the blanks and also add a few high-traffic sites (I’ve only selected websites which are in the Alexa top 1000). Here’s the new (and supposedly, more accurate) graph:

While it looks like Cassandra got a few higher-profile clients lately (even if for relatively mundane tasks such as persistent caching) this does not mean HBase is only used by StumbleUpon. I think we’ll see more and more success stories for both of these solutions and perhaps for some un-announced future products as well. The domain is still in its infancy, let’s not imply “consolidation” just yet.

My original contribution here is a comparison between the original memcached server from Danga and the tugela fork from the MediaWiki programmers. I’ve also tried memcachedb but the pre 1.0 version (from Google Code) in November 2007 was quite unstable and unpredictible.

In a nutshell, these memcache versions are using BerkeleyDB instead of memory slab allocator. There are 2 direct consequences:

• when the memory is large enough for the whole cache, database-backed servers will be slower (my tests shown 10-15% which might be tolerable – or not – for your app)
• when you’ve got lots of data to cache and your server’s memory is low, relying on bdb is significantly better than letting the swap mechanism to do its job (from my benchmarks, the difference can go up to 10 times faster especially under very high concurrency conditions)

Tugela will prove especially useful when running it on virtualized servers with very low memory.

My tests were performed with the “Tummy” Python client and Stackless for the multithreaded version. In one of the following weeks I’ll update the benchmarks for memcachedb 1.0.x – and I promise never ever to wait 4 months for a presentation, again …