Toru Maesaka

Last week I was in the valley for the fifth memcached Hackathon at Sun Microsystems and visiting some friends at Six Apart HQ. The hackathon was so fun, we ended up leaving at 2am on a weeknight! Thanks to Matt Ingenthron and Sun Microsystems for organizing the event and providing food and space for this hackathon

In the previous hackathon, we mostly exchanged ideas on the binary protocol and the storage engine interface. This time it was more code oriented and we reviewed and tested the progress everyone had made in the latest binary protocol tree. Unfortunately I couldn’t cover the whole hackathon but here is a summary of discussions from the agenda that I was involved in:

Binary Protocol - Add an engine specific OPCODE

No disagreements here. An opcode is represented by a 1 byte unsigned integer so the consensus was that we should dedicate anything over 127 (0×7F) for special operations.

Storage Interface

We didn’t get around to discussing the interface in depth since getting the binary protocol released has greater priority at the moment. Trond however showed me some of the interesting work that he has been doing which will hopefully be out in the open soon.

Test Framework

The issue here is that tests aren’t actively been written. Opinions voiced on this issue was that some people aren’t comfortable with Perl, and thus difficult to understand the current Perl based test system.

Switching to a different test framework in a different language is easy but the problem is that this is a never-ending story. People can easily start demanding other languages that they feel comfortable in (python, java, ruby, lua, …). We briefly discussed that the ideal model is to be able to add tests written in any language but we didn’t go into depth on how we would actually achieve this.

Personally, I have nothing against the current test framework (mind you I like Perl) but I think if we were to switch, a solely C based framework is a good move. I am saying this because those that would think about opening up the memcached package and editing it can most likely write C (is this an assertive assumption? heh).

Client Libraries

Unfortunately I couldn’t get around to participating in the client talk but client-side replication work was being done for libmemcached and I heard from Brian Aker that there was good progress.

Jonathan (hachi) reviewed my binary protocol patch for Cache::Memcached and found that some protocol negotiation assumptions I made in the code can be improved. He is also looking at optimizing the code by subclassing the patch (reduces the number of conditional selections, perl method calls and hash lookups).

Scaling on Highly Threaded Servers

We didn’t really discuss this in depth since we were busy reviewing and testing the server code but as far as I know, we talked about how locking can be improved in memcached. Looking into and preparing for this is a good idea since we are entering a massively concurrent age. To the contrary, guys from Facebook mentioned that they were getting sufficient throughput with the current locking scheme which was awesome to hear.

The engine plugin rearchitecture should fit well with this project since we can interchange different versions of the slabber engine with different locking strategies and make them compete to be the next default memcached engine.

Conclusion

The hackathon was fun and we got a lot done in terms of finding things to improve on. It was great to catch up with guys that I communicate a lot with online and talk tech in person. It was awesome that Brad turned up as well. As for code improvements, Dustin’s test code found an issue in the stats subsystem always returning a zero for an opaque value. A little bit of coding looked necessary to get around this problem since an opaque value is held by the connection structure, which the engine does not have access to (it shouldn’t) but I was bored on my flight back to Tokyo so this problem is now fixed and pushed to my tree