Tag Archives: python

Http response time monitoring with Munin

There’s nothing better than graphs, and for a web site there are few better things to graph than the response times for your web pages. While there are plenty of external services out there that will probe your web site and graph the results, it’s a good idea to do this on your own too.

Munin is a monitoring tool that can provide graphs in plentiful for your servers. Out of the box, or at least out of its Ubuntu box, it monitors a variety of system metrics and applications, but there is no bundled support for response-time monitoring.

Luckily, it’s really easy to extend Munin with new plugins, so I decided to write my own plugin for monitoring response times, which you can download from my bitbucket repository. It will produce graphs like the one below:

http_response_time_ example graph

Here the plugin has been configured to monitor three URLs in the same graph. Unlike normal Munin probes these URLs are external to the actual server running the plugin, but you could just as well monitor localhost URLs too.

To get the plugin up and running you first need to install Munin, if you haven’t already. For a one-server setup under Ubuntu, with master and client both running on the same machine, you do:

sudo apt-get install munin munin-node

This will start the munin-node service in the background, and also add the master-node cron jobs to /etc/cron.d/munin. To install the plugin itself you add the Python script to a folder of your choice, make it executable, and then symlink it from the Munin plugin folder, like so:

sudo cp ./http_response_time_ /usr/local/bin/
sudo chmod 755 /usr/local/bin/http_response_time_
sudo ln -s /usr/local/bin/http_response_time_ /etc/munin/plugins/http_response_time_example

If you want to monitor more than one set of URLs, and thus have more than one graph, you can accomplish that by creating one symlink for each graph that you need. The names of the symlinks are used as section titles when configuring the plugin in /etc/munin/plugin-conf.d/munin-node. For the graph shown above, the configuration would look something like this:

[http_response_time_example]
env.url1_url http://www.djangoproject.com/
env.url1_name django
env.url1_label djangoproject.com
env.url2_url http://rubyonrails.org/
env.url2_name rails
env.url2_label rubyonrails.org
env.url3_url http://php.net/
env.url3_name php
env.url3_label php.net

The plugin requires that the following environment variables are specified for each URL to be monitored:

  • urlX_url — the URL that should be monitored
  • urlX_name — Munin field name for the probe
  • urlX_label — legend description of the URL

The ‘X’ in the variable names above should be replaced with an incremental index for each URL, e.g. url1 and url2. In addition, the following environment variables are also supported:

  • graph_title — the title of the graph (default is “Response time”)
  • graph_category — the category Munin should show the graph in
  • request_timeout — the socket request timeout (same for all URLs)
  • urlX_warning — warning level for the probe (for Nagios)
  • urlX_critical — critical level for the probe (for Nagios)

Note that Munin uses its own timeout when fetching plugin data. The default value is 10 seconds, which also is the default value for the URL request timeout. Because of this it might be appropriate to increase the Munin fetch timeout so that it equals the number of URLs being monitored times the request timeout, to make sure all probes have time to run.

Once you have configured the plugin to your satisfaction you need to restart the Munin node to make it discover the new plugin:

sudo /etc/init.d/munin-node restart

Happy graphing!

Advertisements

Django and memcached library confusion

I’ve been playing around a bit with Python and Django lately, and I’m pretty impressed with them both so far. Django brings some nice features out of the box to the developer, one of them being the cache framework and the built-in support for memcached. I’ve never actually used memcached before, so I thought this would be a good time to try it out.

Installing memcached itself went fairly smoothly, but I got a bit confused when it was time for the Python client library. According to the documentation, Django supports cmemcache and python-memcached, of which cmemcache is the recommended option since it’s built on top of the C library libmemcache and almost twice as fast as the pure Python library.

On the other hand, the library used by cmemcache, i.e. libmemcache, is according to the memcached documentation wiki and this thread no longer under active development, which the early-2006 timestamps of the file listings agree with. It’s a bit strange that a seemingly abandoned library is recommended by the Django documentation, especially since memcached surely must have evolved a bit during the last 3 years.

To the detriment of both libraries, neither seem to support the consistent hashing introduced in memcached by libketama. Consistent hashing makes it possible to add or remove memcached nodes to a live environment without having all keys become remapped to different servers, effectively nuking the cache. This will not be a problem for all environments of course, but it makes dynamic scaling less attractive.

Consistent hashing is also supported by libmemcached (not to be confused with libmemcache), which is still actively maintained and used by python-libmemcached (not to be confused with python-memcached). Despite the low version number, it looks like a more appealing choice for Python clients, given that it’s actively developed, presumably fast, and supports consistent hashing. A pity that Django doesn’t support it.

So far this is just theory and speculation, of course, and only testing may tell what’s the best choice for a Django site. What’s the worst offense; the slower performance of python-memcached or the outdated underlying library of cmemcache? Does its age make any difference in practice? Thoughts, anyone?