Django and memcached library confusion

I’ve been playing around a bit with Python and Django lately, and I’m pretty impressed with them both so far. Django brings some nice features out of the box to the developer, one of them being the cache framework and the built-in support for memcached. I’ve never actually used memcached before, so I thought this would be a good time to try it out.

Installing memcached itself went fairly smoothly, but I got a bit confused when it was time for the Python client library. According to the documentation, Django supports cmemcache and python-memcached, of which cmemcache is the recommended option since it’s built on top of the C library libmemcache and almost twice as fast as the pure Python library.

On the other hand, the library used by cmemcache, i.e. libmemcache, is according to the memcached documentation wiki and this thread no longer under active development, which the early-2006 timestamps of the file listings agree with. It’s a bit strange that a seemingly abandoned library is recommended by the Django documentation, especially since memcached surely must have evolved a bit during the last 3 years.

To the detriment of both libraries, neither seem to support the consistent hashing introduced in memcached by libketama. Consistent hashing makes it possible to add or remove memcached nodes to a live environment without having all keys become remapped to different servers, effectively nuking the cache. This will not be a problem for all environments of course, but it makes dynamic scaling less attractive.

Consistent hashing is also supported by libmemcached (not to be confused with libmemcache), which is still actively maintained and used by python-libmemcached (not to be confused with python-memcached). Despite the low version number, it looks like a more appealing choice for Python clients, given that it’s actively developed, presumably fast, and supports consistent hashing. A pity that Django doesn’t support it.

So far this is just theory and speculation, of course, and only testing may tell what’s the best choice for a Django site. What’s the worst offense; the slower performance of python-memcached or the outdated underlying library of cmemcache? Does its age make any difference in practice? Thoughts, anyone?


Comments are closed.