Netuality

Memcached is a clusterable cache server from Danga. Or, as they call, it a distributed memory object caching system. Well, whatever. Just note that memcached clients exist for lots of languages (Java, PHP, Python, Ruby, Perl) – mainstream languages in the web world. A lighter version of server was rewritten in Java by Mr. Jehiah Czebotar. Major websites such as Facebook, Slashdot, Livejournal and Dealnews use memcached in order to scale for the huge load they’re serving. Recently, we needed to monitor the memcache servers on a high-performance web cluster serving the Planigo websites. By googling and reading the related newsgroups, I was able to find two solutions:

• from faemalia.net, a script which is integrated with the MySQL server templates for Cacti. Uses the Perl client.
• from dealnews.com, a dedicated memcached template for Cacti and some scripts based on the Python client. The installation is thoroughly described here.

These two solutions have the same approach – provide a specialized Cacti template. The charts drawn by these templates are based on data extracted by the execution of memcached client scripts. Maybe very elegant, but could become a pain in the dorsal area. Futzing with Cacti templates was never my favorite pasttime. Just try to import a template exported from a different version of Cacti and you’ll know what I mean. In my opinion, there is a simple way, which consists in installing a memcached client on all the memcached servers, then extracting the statistical values using a script. We’ll use the technique described in one of my previous posts, to expose script results as SNMP OID values. Then, track these values in Cacti via the generic existing mechanism. My approach has the disadvantage of installing a memcached client on all the servers. However, it is very simple to build your own charts and data source templates, as for any generic SNMP data. All you need now a simple script which will print the memcached statistics, one per line. I will provide one-liners for Python, which will obviously work only on machines having Python and the “tummy” client installed. This is the recipe (default location of Python executable on Debian is /usr/bin/python but YMMV):

1. first use this one liner as snmpd exec :

/usr/bin/python -c “import memcache; print (‘%s’%[memcache.Client(['127.0.0.1:11211'], debug=0).get_stats()[0][1],]).replace(\”‘\”,”).replace(‘,’,'\n’).replace(‘[','')
.replace(']‘,”).replace(‘{‘,”).replace(‘}’,”)”

This will display the name of the memcached statistic along with its value and will allow you to hand pick the OIDs that you want to track. Yes, I know it could be done simpler with translate instead of multiple replace. Left as an exercise for the Python-aware reader.

2. after having a complete list of OIDs use this one-liner:

/usr/bin/python -c “import memcache; print ‘##’.join(memcache.Client(['127.0.0.1:11211'], debug=0).get_stats()[0][1].values()).replace(‘##’,'\n’)”

The memcached statistics will be displayed in the same order, but only their values not their names.

And this is the mandatory eye candy:

My previous article was focused on Linux monitoring. Often, you’ll have in your datacenter at least a few Windows machines. SQL Server is one of the best excuses these days to get a Microsoft machine in your server room – and you know what, it’s a decent database – well, at least for medium-sized companies like the one I’m working for right now.

It is less known, but yes you can have SNMP support out of the box with Windows 2000 and XP, and it doesn’t need to be the Server flavor [obiously it works the same in 2003 Server]:

1. Invoke the Control Panel.
2. Double click the Add/Remove Programs icon.
3. Select Add/Remove Windows Components. The Windows Component Wizard is displayed.
4. Check the Management and Monitoring Tools box.
5. Click the Details button.
6. Check the Simple Network Management Protocol box and click OK, then Next. You may have to reboot the machine.

After the server is installed, the SNMP service has to be configured. Here’s how:

1. Invoke the Control Panel.
2. Double click the Administrative Tools icon.
3. Double click the Services icon.
4. Select SNMP Service.
5. Choose the Security tab.
6. Add whatever community name is used in your network. Chances are in a local internal LAN the default public works out of the box.
7. For a sensitive server, you may want to fiddle a little bit with the IP restriction settings, for instance allowing SNMP communication only with the monitoring machine.
8. Click OK then restart the service.

Next step is Cacti integration. Unfortunately, there is no Windows-specific profile for devices in Cacti. Therefore if you have lots of Windows machines, you’ll have to define your own. Or, take a Generic SNMP-enabled host and use it as a scaffold for each device configuration.

Out of the graphs and datasources already defined in Cacti [I am using 0.8.6c] only two work with Windows SNMP agents: processes and interface traffic values.

It’s a good start, but if you are serious about monitoring, you need to dig a little bit deeper. Once again, the MIB Browser comes to save the day. It’s very simple, just search on the Windows machine for any .mib files you are able to find, copy on your workstation, load them into the MIB browser and make some recursive walks (Get subtree on the root of the MIB).This way, I was able to find some interesting OID for the Windows machine. For instance, .1.3.6.1.2.1.25.3.3.1.2.1 -> .1.3.6.1.2.1.25.3.3.1.2.4 the OID for CPU load on each of the 4 virtual CPUs [it's a dual Xeon with HT].

Memory-related OIDs for my configuration are :

• .1.3.6.1.2.1.25.2.3.1.5.6 – Total physical memory
• .1.3.6.1.2.1.25.2.3.1.6.6 – Used physical memory
• .1.3.6.1.2.1.25.2.3.1.6.6 – Total virtual memory ["virtual"="swap" in Windows lingo]
• .1.3.6.1.2.1.25.2.3.1.6.6 – Used virtual memory

Here’s a neat memory chart for a windows machine. Notice that the values are in “blocks” which in my case is 64kb. The total physical memory is 4GB.

Most hardware manufacturers do offer SNMP agents for their hardware, as well as the corresponding .mib file . In my case, I was able to install an agent to monitor an LSI Megaraid controller. Here is a chart for the number of disk operations/second:

In one of my next articles, we’ll take a look together at the way you can export “non-standard” data over SNMP from Windows, in the same manner we did on Linux, using custom scripts. Till then, have an excellent week.