This is a read-only archive!

Tokyo Cabinet, Engage

I posted recently about being dissatisfied with how my blog was playing with MySQL. I was going to try for some kind of file-based storage, but in the end I decided to go with Tokyo Cabinet, which is a very lightweight key/value store.

This simplifies a lot of code, because a blog is pretty much one list of objects (posts) with a bunch of sub-objects (tags, categories, comments). An SQL DB is made for independent objects that are related to each other via keys. So storing a post means deconstructing it into its parts and stuffing all the parts into their own tables, and fetching a post means fetching all the parts and putting it back together. A bit of a bother.

With a key/value store, a post is a hashmap, and it has sub-lists of comments and tags etc., and you serialize the whole thing and stuff it into the DB; when fetching you un-serialize it and you're good. Clojure being a Lisp, there's already a nice serialization format (s-expressions), so there's hardly any work to be done.

I'll give it a few days and if everything actually works, I'll push the code to github. The code is still pretty nasty and ad-hoc; I did this rewrite in the matter of a couple of hours on a weekend. But parts of it may be useful to someone.

I also plan to do some kind of simplified "How to make a blog in Clojure" tutorial in the near future. The code for this blog is bloated with a lot of functionality that is pretty specific to my site and that most people don't need. A more to-the-point tutorial would probably be more helpful.

This post is related to Who needs a DB?
July 05, 2009 @ 1:50 PM PDT
Cateogory: Programming


Quoth Nathab on July 05, 2009 @ 2:32 PM PDT

That should be interesting. I've always found blogs to be really good samples of what an application on a particular language/framework will be like.

Brandon Gray
Quoth Brandon Gray on July 06, 2009 @ 9:49 PM PDT

Very nice! I look forward to digging through the code.

DC Recruiters
Quoth DC Recruiters on July 09, 2009 @ 1:16 PM PDT

I am curious does using the Tokyo Cabinet key/value store have any effect on page load times.

Seems like if there were less computational steps involved, the load times might be faster.

Quoth Brian on July 11, 2009 @ 1:25 PM PDT

Fetching one record from Tokyo Cabinet is as instant as I could hope for. Fetching all records (which is necessary in many places in this blog) is not fast enough, so I cache everything in a ref.