Suppose I have this:

user> (def foo [{:id 1 :foo 123} {:id 2 :foo 456}])
#'user/foo
user> (def bar [{:foo_id 1 :bar 111} {:foo_id 1 :bar 222}])
#'user/bar

What I want is to "join" foo and bar so that each item in foo ends up with a sub-list of bars based on matching key fields.

In real life, these lists-of-hash-maps are coming out of a database via clojure.contrib.sql, so this is something I actually want to do pretty often. This is also vaguely similar to what you get out of a Rails-like ORM, where you end up with an object that has lists of sub-objects anywhere you have a one-to-many relationship.

Here's how I end up doing this in Clojure:

(defn one-to-many
  ([xs name ys f]
      (for [x xs :let [ys (filter (partial f x) ys)]]
        (assoc x name ys))))

Now I can do this:

user> (pprint (one-to-many foo :bars bar #(= (:id %1) (:foo_id %2))))
({:bars ({:foo_id 1, :bar 111} {:foo_id 1, :bar 222}), :id 1, :foo 123}
 {:bars (), :id 2, :foo 456})

And if I define a helper function:

(defn key=
  ([xkey ykey]
     #(= (xkey %1) (ykey %2))))

Then I can write it more concisely:

user> (pprint (one-to-many foo :bars bar (key= :id :foo_id)))
;; same as above

And if I have another "table" of data like this:

user> (def baz [{:foo_id 1 :baz 555} {:foo_id 2 :baz 999}])
#'user/baz

Then I can join them all like this:

user> (pprint (-> foo
                  (one-to-many :bars bar (key= :id :foo_id))
                  (one-to-many :bazzes baz (key= :id :foo_id))))
({:bazzes ({:foo_id 1, :baz 555}),
  :bars ({:foo_id 1, :bar 111} {:foo_id 1, :bar 222}),
  :id 1,
  :foo 123}
 {:bazzes ({:foo_id 2, :baz 999}), :bars (), :id 2, :foo 456})

This is pretty concise. It may be possible to do it in an even more concise way, (if so, do share). If I was willing to adhere to some Rails-y naming convention for my table names and for the id fields in my tables, I could make this shorter by not having to specify the names of the id fields, but I don't want to go there. It's trivial to write similar functions for a one-to-one relationship, or to use a join-table to "join" two tables with a many-to-many relationship.

I am happily surprised sometimes by how simple it is to roll my own version of things that previously seemed like dark magic. I used Rails for a long time and it seemed like a crapload of code must have gone into making the ORM work. But four lines of code gets me 75% of what I ever needed Rails' ORM for.

This may be more thanks to me opening my eyes a bit than to Clojure being awesome, but either way, I'll take it.

I started a new RoR project recently, and for the first time tried using migrations. They're mildly acceptable yet I still find myself not caring much for them.

The benefits of migrations as I understand it (and why those benefits don't matter to me) are:

• You get to "version control" your database schema.

But if I want version control, I'll use a real version control system. A bunch of files in a directory sorted by a numeric prefix is rudimentary compared with Bazaar or SVN or anything else.

Migrations also require you to write all the code to "rollback" your changes yourself by hand, which is tedious. Every time I write an add_column I have to write a corresponding remove_column too. Many times I've gotten the add part right, and typoed the remove part, so the migration was broken, and I didn't realize until much later. Many times I wrote a remove part that I never ever had a need to use.

Because migrations are a single big long linear path, rolling way back to some point in history will obliterate a lot of changes you may want to keep that were made after that point; the older a migration is, the less likely you'll ever run it again.

Maybe I should be unit testing my migrations, you may say. I say what's the point? I'm adding a column to a DB. Do I really need to test that? Unit testing is fine, but there's a point beyond which it becomes silly. Especially for a web app, especially for something so trivial. * You can write DB stuff in Ruby rather than writing SQL manually.

This saves you a little bit of effort. But if you know SQL, it's not THAT hard to write an ALTER TABLE query. In fact given the whole script/generate migration SomeLongName step and then opening and editing and saving and testing and uploading and running the migration file, it's actually far faster for me to drop into a mysql prompt and do it that way.

There are also some things you can't do with Rails migration methods, and you need raw SQL. In which case you end up writing SQL anyways. Depending what you're doing, this may never happen or it may happen frequently enough to be annoying. * You can deploy your DB changes by uploading the migrations and running rake.

This is a good thing, but you can also save some SQL into a text file and run it by piping it into mysql from the command line, or importing it via phpmyadmin, or various other methods. * Migrations are database-agnostic, whereas SQL is usually tied to a single database.

My first reaction is, so what? What are the chances that I'm going to change my DB engine? Not very likely in the kinds of things I work on. My second reaction is that the kind of queries you write to create and edit tables are so generic that they're likely to work in most or all DB engines that Ruby supports anyways, or will work after minor edits.

• Migrations do some magic things like automatically setting up an id field for your models etc.

This is the only real benefit to migrations that I can see: a small amount of time savings and convenience and integration with the rest of the tangled mess of RoR.

What I'm trying right now is writing all my table creation code in SQL, putting it all into .sql files, and putting those files under version control in Bazaar. This is better version control than migrations offer. And my table definitions are completely intact in a single file each, which is easily browsable and editable. (I think Rails lets you export schemas too, but it's not as nice as having them in files to look at and edit directly in my opinion.) If I want to see differences in my schema over time, I can bzr diff the files that contain them. If I want to take a single table and reproduce it in some DB somewhere, I can run that single .sql file. If I want to deploy, I rsync the files to my server and pipe them into a mysql prompt.

The good thing about RoR is that I can completely ignore migrations if I don't like them. And for my large legacy RoR app, I am ignoring them. However I plan to stick with them on this new app as far as I can; maybe more benefits will reveal themselves.

Today at 5:00 I bought Agile Web Development with Rails. As of now 10:15 I have finished reading it. Pretty good book. I think for Rails you really need either a good web tutorial or a good book or about a month to read through the API. Way too much stuff is done by convention to ever learn it all otherwise. And the jargon thrown around is impressively extensive. Routes, actions, views, partials, layouts... these things don't mean much without a really good knowledge of the context of Rails.

I decided I needed to ingest a book sooner rather than later. I have too much tendency to dive face-first into coding without doing enough preparation or planning, and then having to backtrack and redo a lot of things differently. I guess that's always part of programming, but I rather want to minimize it as much as possible.

So this weekend begins the grand Rails experiment. I'm going to port all of my other website into Ruby. Or get as much done this weekend as I can, at least. We'll see how this goes.