Tuesday, September 18, 2012

Implementing a ReDiS COUNT command in Lua

I have recently been implementing a master/slave redis cache for our enterprise application and at certain points during the development/test phases, have found it useful to be able to query redis for the # of keys that have been inserted that match a certain regex.
redis has the 'keys' command that returns all keys that match a certain pattern, so for example, if I wanted to see all the keys that match the pattern '*:D' I would simply issue the command :

keys *:D

and redis would return a list of all keys that meet that pattern. This is all very well when the number of keys is small, but when you are handling millions of keys, it quickly becomes unmanageable. I've seen a lot of requests on the redis message boards requesting that the development team implement the COUNT command and at first thought it does seem intuitive, however its implementation would violate the O(1) performance requirements of basic commands that the developers have (rightly, in my NSHO) imposed upon themselves.

Thankfully, the redis lua integration comes quickly to your rescue. I'm not suggesting that the lua implementation of this command achieves O(1), far from it, in fact, but I'm pointing out that the functionality of a COUNT command is already easily achievable with the following simple command issued in the client :

eval "return #redis.pcall('keys', '*')" 0

Very smartly, the developers have implemented lua bindings so that if you need a command that would violate their very stringent requirements for performance, you can create as much hanging rope as you need.

Smart.