This is a read-only archive!

Tokyo Cabinet, Engage

I posted recently about being dissatisfied with how my blog was playing with MySQL. I was going to try for some kind of file-based storage, but in the end I decided to go with Tokyo Cabinet, which is a very lightweight key/value store.

This simplifies a lot of code, because a blog is pretty much one list of objects (posts) with a bunch of sub-objects (tags, categories, comments). An SQL DB is made for independent objects that are related to each other via keys. So storing a post means deconstructing it into its parts and stuffing all the parts into their own tables, and fetching a post means fetching all the parts and putting it back together. A bit of a bother.

With a key/value store, a post is a hashmap, and it has sub-lists of comments and tags etc., and you serialize the whole thing and stuff it into the DB; when fetching you un-serialize it and you're good. Clojure being a Lisp, there's already a nice serialization format (s-expressions), so there's hardly any work to be done.

I'll give it a few days and if everything actually works, I'll push the code to github. The code is still pretty nasty and ad-hoc; I did this rewrite in the matter of a couple of hours on a weekend. But parts of it may be useful to someone.

I also plan to do some kind of simplified "How to make a blog in Clojure" tutorial in the near future. The code for this blog is bloated with a lot of functionality that is pretty specific to my site and that most people don't need. A more to-the-point tutorial would probably be more helpful.

This post is related to Who needs a DB?
July 05, 2009 @ 1:50 PM PDT
Cateogory: Programming

4 Comments

Nathab
Quoth Nathab on July 05, 2009 @ 2:32 PM PDT

That should be interesting. I've always found blogs to be really good samples of what an application on a particular language/framework will be like.

Brandon Gray
Quoth Brandon Gray on July 06, 2009 @ 9:49 PM PDT

Very nice! I look forward to digging through the code.

DC Recruiters
Quoth DC Recruiters on July 09, 2009 @ 1:16 PM PDT

I am curious does using the Tokyo Cabinet key/value store have any effect on page load times.

Seems like if there were less computational steps involved, the load times might be faster.

Brian
Quoth Brian on July 11, 2009 @ 1:25 PM PDT

Fetching one record from Tokyo Cabinet is as instant as I could hope for. Fetching all records (which is necessary in many places in this blog) is not fast enough, so I cache everything in a ref.