Netuality

Yes, language propaganda again. Ain’t it fun ?

Here comes a nice quote from the latest Steve Yegge post (read it entirely if you have the time, it’s both fun and educational – at least for me). So, there:

I made the famously, horribly, career-shatteringly bad mistake of trying to use Ruby at Google, for this project. And I became, very quickly, I mean almost overnight, the Most Hated Person At Google. And, uh, and I’d have arguments with people about it, and they’d be like Nooooooo, WHAT IF… And ultimately, you know, ultimately they actually convinced me that they were right, in the sense that there actually were a few things. There were some taxes that I was imposing on the systems people, where they were gonna have to have some maintenance issues that they wouldn’t have. [...] But, you know, Google’s all about getting stuff done.

[...]

Is it allowed at Google to use Lisp and other languages?

No. No, it’s not OK. At Google you can use C++, Java, Python, JavaScript… I actually found a legal loophole and used server-side JavaScript for a project.

Mmmmm … key ?

According to O’Reilly Radar, sales of Java books have declined in the last 4 years by almost 50%. C# is selling more books from year to year and will probably level up with Java in 2008. Javascript is on the rise (due to AJAX, for sure) and PHP is on a surprising decrease path (although the job statistics indicate quite the contrary).

In 2007, the number of sold Ruby books was larger than the number of Python books. In their article they qualify Ruby as being a “mid-major programming language” and Python as “mid-minor programming language”. However, after the announcement of Google App Engine the number of Python downloads from ActiveState has tripled in May. This should become visible in the book sales statistics, pretty soon.

This presentation was planned for an older Wurbe event, but as this never quite happened in the last 4 months I am publishing it now, before it becomes totally obsolete.

My original contribution here is a comparison between the original memcached server from Danga and the tugela fork from the MediaWiki programmers. I’ve also tried memcachedb but the pre 1.0 version (from Google Code) in November 2007 was quite unstable and unpredictible.

In a nutshell, these memcache versions are using BerkeleyDB instead of memory slab allocator. There are 2 direct consequences:

• when the memory is large enough for the whole cache, database-backed servers will be slower (my tests shown 10-15% which might be tolerable – or not – for your app)
• when you’ve got lots of data to cache and your server’s memory is low, relying on bdb is significantly better than letting the swap mechanism to do its job (from my benchmarks, the difference can go up to 10 times faster especially under very high concurrency conditions)

Tugela will prove especially useful when running it on virtualized servers with very low memory.

My tests were performed with the “Tummy” Python client and Stackless for the multithreaded version. In one of the following weeks I’ll update the benchmarks for memcachedb 1.0.x – and I promise never ever to wait 4 months for a presentation, again …

A newcomer in the world of “code snippets” sites in programmingishard.com. Although the site is a few months old, only recently it started to gain some steam. Unlike its competition Krugle and Koders, this is not a code search engine but a snippet repository entirely tag-based, user-built. The author has a blog at tentimesbetter.com.

As for watering your mouth, this is a Python code fragment that I found on the site, for the classic inline conditional which does not exist “such as” in Python:

n = ['no', 'yes'][thing == 1]

Obviously it has the big disadvantage of having to compute both values no matter what the condition thing is, but is very short and elegant. Simple but nice code sugar.

Memcached is a clusterable cache server from Danga. Or, as they call, it a distributed memory object caching system. Well, whatever. Just note that memcached clients exist for lots of languages (Java, PHP, Python, Ruby, Perl) – mainstream languages in the web world. A lighter version of server was rewritten in Java by Mr. Jehiah Czebotar. Major websites such as Facebook, Slashdot, Livejournal and Dealnews use memcached in order to scale for the huge load they’re serving. Recently, we needed to monitor the memcache servers on a high-performance web cluster serving the Planigo websites. By googling and reading the related newsgroups, I was able to find two solutions:

• from faemalia.net, a script which is integrated with the MySQL server templates for Cacti. Uses the Perl client.
• from dealnews.com, a dedicated memcached template for Cacti and some scripts based on the Python client. The installation is thoroughly described here.

These two solutions have the same approach – provide a specialized Cacti template. The charts drawn by these templates are based on data extracted by the execution of memcached client scripts. Maybe very elegant, but could become a pain in the dorsal area. Futzing with Cacti templates was never my favorite pasttime. Just try to import a template exported from a different version of Cacti and you’ll know what I mean. In my opinion, there is a simple way, which consists in installing a memcached client on all the memcached servers, then extracting the statistical values using a script. We’ll use the technique described in one of my previous posts, to expose script results as SNMP OID values. Then, track these values in Cacti via the generic existing mechanism. My approach has the disadvantage of installing a memcached client on all the servers. However, it is very simple to build your own charts and data source templates, as for any generic SNMP data. All you need now a simple script which will print the memcached statistics, one per line. I will provide one-liners for Python, which will obviously work only on machines having Python and the “tummy” client installed. This is the recipe (default location of Python executable on Debian is /usr/bin/python but YMMV):

1. first use this one liner as snmpd exec :

/usr/bin/python -c “import memcache; print (‘%s’%[memcache.Client(['127.0.0.1:11211'], debug=0).get_stats()[0][1],]).replace(\”‘\”,”).replace(‘,’,'\n’).replace(‘[','')
.replace(']‘,”).replace(‘{‘,”).replace(‘}’,”)”

This will display the name of the memcached statistic along with its value and will allow you to hand pick the OIDs that you want to track. Yes, I know it could be done simpler with translate instead of multiple replace. Left as an exercise for the Python-aware reader.

2. after having a complete list of OIDs use this one-liner:

/usr/bin/python -c “import memcache; print ‘##’.join(memcache.Client(['127.0.0.1:11211'], debug=0).get_stats()[0][1].values()).replace(‘##’,'\n’)”

The memcached statistics will be displayed in the same order, but only their values not their names.

And this is the mandatory eye candy: