briancarper.net

Clojure, SLIME, ODBC, SQL Server

Fri, 26 Jun 2009 11:27:22 -0700

I had a lot of trouble connecting to an MS SQL Server at work via Clojure. Java 6 comes with a JDBC-ODBC bridge which worked fine from a Clojure REPL at a command prompt, or from inferior-lisp in Emacs, but in SLIME it would hang every time I tried to connect and I'd have to kill Java. Couldn't for the life of me figure out why.

I got it to work eventually by using Microsoft's own JDBC driver, which you can download here.

Once you put the downloaded .jar file on your CLASSPATH (in my case, sqljdbc4.jar) you can connect like this:

user> (def db {:classname "com.microsoft.sqlserver.jdbc.SQLServerDriver"
               :subprotocol "sqlserver"
               :subname "//server_hostname;database=SomeDatabase;user=SomeUser;password=SomePassword"})
#'user/db
user> (use 'clojure.contrib.sql)
nil
user> (with-connection db 
        (with-query-results rs ["SELECT * FROM whatever"] (prn rs)))
... results ...

Posted for the sake of Googlebot and for my own future sanity.

Five Things that Mildly Annoy Me in Clojure

Tue, 16 Jun 2009 21:56:21 -0700

This infamous blog post suggests that someone familiar with a language should be able to name five things they hate about it. "Hate" is a strong word, but I decided to think of five things I find mildyly annoying about Clojure, my favorite language of the moment.

Hashing integers

Clojure automatically converts integers between Integer, Long and BigInteger as needed to prevent overflow. This is good. Integers of the various classes test as equal too. This is also good.

user> (= 123 (int 123) (long 123) (bigint 123))
true

So would you expect this?

user> (hash-map (int 123) :foo (long 123) :bar (bigint 123) :baz)
{123 :foo, 123 :bar, 123 :baz}

Yes, each of the integer classes, though equal via =, do not have the same hash value when put into a hash-map. This is because:

user> (.equals (int 123) (long 123))
false

This is a wart inherited from the JVM. See here for discussion and explanation.

What's more, if you print this map and then try to read it back in, the integers will be read as int, long or bigint arbitrarily depending how big they are. This means you may not get the same class of object back that you output originally.

user> (def x {(bigint 123) :foo})
#'user/x
user> (= x x)
true
user> (def y (read-string (pr-str x)))
#'user/y
user> (= x y)
false
user> (class (first (keys y)))
java.lang.Integer
user> (class (first (keys x)))
java.math.BigInteger

This means that if you ever use integers as hash keys, you must be very careful to cast them all to the same integer type manually.

Metadata doesn't work on everything

Clojure lets you stick arbitrary metadata on various objects. This is higly useful; you can decorate objects with information that doesn't affect the value of the object. However metadata doesn't work everywhere.

user> (with-meta "foo" {:bar :baz})
java.lang.ClassCastException: java.lang.String cannot be cast to clojure.lang.IObj (NO_SOURCE_FILE:0)

You can only stick metadata on certain Clojure objects like Symbols, Vars, Refs, Agents, all of the Clojure collections and so on. You can't stick metadata on, say, a String or an Integer, because those are closed Java classes and can't be touched. It would be nice if you could.

use vs. require vs. import vs. load vs. ...

There are a startling number of ways to import a library into your code in Clojure. You have to choose from load, import, require, use, refer, and so on. Some work on Java classes, some work on Clojure libs. Some of them import symbols into your namespace, some of them don't. Some of them take strings as arguments, some take symbols, some take quoted lists of symbols, some take quoted lists of symbols with sub-lists of arguments. And all of these can be and usually are weirdly inlined into a namespace declaration, with a completely different list-quoting style.

So in Ruby you can do this:

require 'util'
require 'config'
require 'whatever'

Whether it's a gem, or a Ruby source file sitting locally, it all works the same as long as the load path is set up right.

But in Clojure, you do this (actual code from an IMAP library I wrote):

(ns qt4-mailtray.mail
  (:import (java.util Properties)
           (javax.mail Session Store Folder Message Flags Flags$Flag FetchProfile FetchProfile$Item)
           (javax.mail.internet InternetAddress))
  (:use clojure.contrib.str-utils))

This can quickly become unwieldy, especially if you start using the :as or :only or :rename arguments. It's made worse by Java's insane API's full of a billion classes that you need to import to do simple things. (And those things with dollar signs are mangled Java inner class names.) Clojure also lacks the ability to import a whole package worth of classes at once using java.io.* syntax, so you must name all of the classes explicitly.

every? vs some.

This is such a trite pet-peeve that it's barely worth mentioning, but it seems to be brought up repeatedly and endlessly on the Clojure mailing list so at least I'm not the only one bugged by it.

Clojure has a function (every? pred coll) which tests whether every item in a collection tests true via some predicate. To test whether every item in a collection tests false, we have not-any?. And we have a not-every? which tests whether any item tests false.

user> (every? even? [2 4 6])
true
user> (not-every? even? [2 4 6])
false
user> (not-any? even? [2 4 6])
false

Now what would you expect a function to be called which tests whether any item in a collection tests true via some predicate? If you said any? you are wrong! It's some.

Note that some isn't a predicate (hence no question mark in the name); it doesn't return true or false, as above, but rather returns the result of running pred on an item in coll.

user> (some identity [nil 1 2 3])
1

any? is pretty easy to write so it doesn't matter that much. Probably many people have an identical function sitting in some utils.clj file on their systems.

(defn any? [pred coll]
  (when (seq coll)
    (if (pred (first coll))
      true
      (recur pred (next coll)))))

Stack trace madness

Give this function:

(defn foo []
  (throw (Exception. "BARFED")))

What does the stack trace look like in SLIME when you call foo? Like this:

java.lang.Exception: BARFED (NO_SOURCE_FILE:0)
  [Thrown class clojure.lang.Compiler$CompilerException]

Restarts:
 0: [ABORT] Return to SLIME's top level.
 1: [CAUSE] Throw cause of this exception

Backtrace:
  2: swank.commands.basic$eval_region__729.invoke(basic.clj:36)
  3: swank.commands.basic$listener_eval__738.invoke(basic.clj:50)
  4: clojure.lang.Var.invoke(Var.java:346)
  5: user$eval__1506.invoke(NO_SOURCE_FILE)
  6: clojure.lang.Compiler.eval(Compiler.java:4580)
  7: clojure.core$eval__4016.invoke(core.clj:1728)
  8: swank.core$eval_in_emacs_package__336.invoke(core.clj:55)
  9: swank.core$eval_for_emacs__413.invoke(core.clj:123)
 10: clojure.lang.Var.invoke(Var.java:354)
 11: clojure.lang.AFn.applyToHelper(AFn.java:179)
 12: clojure.lang.Var.applyTo(Var.java:463)
 13: clojure.core$apply__3269.doInvoke(core.clj:390)
 14: clojure.lang.RestFn.invoke(RestFn.java:428)
 15: swank.core$eval_from_control__339.invoke(core.clj:62)
 16: swank.core$eval_loop__342.invoke(core.clj:67)
 17: swank.core$spawn_repl_thread__474$fn__505$fn__507.invoke(core.clj:173)
 18: clojure.lang.AFn.applyToHelper(AFn.java:171)
 19: clojure.lang.AFn.applyTo(AFn.java:164)
 20: clojure.core$apply__3269.doInvoke(core.clj:390)
 21: clojure.lang.RestFn.invoke(RestFn.java:428)
 22: swank.core$spawn_repl_thread__474$fn__505.doInvoke(core.clj:170)
 23: clojure.lang.RestFn.invoke(RestFn.java:402)
 24: clojure.lang.AFn.run(AFn.java:37)
 25: java.lang.Thread.run(Thread.java:619)

Yeouch. Now imagine that the above error is coming not from a simple function, but from some random line among hundreds of lines of source code.

Stack traces in Clojure will often tell you little to nothing about what is causing the error, or more importantly, where it's coming from in your code. Clojure functions are translated into Java classes when they're run through the JVM. Often can't even see the name of the function that's throwing the error; names are mangled into things like user$eval__1473.invoke, which is really really confusing when you use anonymous functions.

Per Jason Wolfe and Randall Schulz sometimes you can get a better stack trace if you dig a bit deeper:

user> (.printStackTrace (.getCause *e))

java.lang.Exception: BARFED
    at user$foo__1503.invoke(NO_SOURCE_FILE:1)
    at user$eval__1509.invoke(NO_SOURCE_FILE:1)
    at clojure.lang.Compiler.eval(Compiler.java:4580)
    at clojure.core$eval__4016.invoke(core.clj:1728)
    at swank.commands.basic$eval_region__729.invoke(basic.clj:36)
    at swank.commands.basic$listener_eval__738.invoke(basic.clj:50)
    at clojure.lang.Var.invoke(Var.java:346)
    at user$eval__1506.invoke(NO_SOURCE_FILE)
    at clojure.lang.Compiler.eval(Compiler.java:4580)
    at clojure.core$eval__4016.invoke(core.clj:1728)
    at swank.core$eval_in_emacs_package__336.invoke(core.clj:55)
    at swank.core$eval_for_emacs__413.invoke(core.clj:123)
    at clojure.lang.Var.invoke(Var.java:354)
    at clojure.lang.AFn.applyToHelper(AFn.java:179)
    at clojure.lang.Var.applyTo(Var.java:463)
    at clojure.core$apply__3269.doInvoke(core.clj:390)
    at clojure.lang.RestFn.invoke(RestFn.java:428)
    at swank.core$eval_from_control__339.invoke(core.clj:62)
    at swank.core$eval_loop__342.invoke(core.clj:67)
    at swank.core$spawn_repl_thread__474$fn__505$fn__507.invoke(core.clj:173)
    at clojure.lang.AFn.applyToHelper(AFn.java:171)
    at clojure.lang.AFn.applyTo(AFn.java:164)
    at clojure.core$apply__3269.doInvoke(core.clj:390)
    at clojure.lang.RestFn.invoke(RestFn.java:428)
    at swank.core$spawn_repl_thread__474$fn__505.doInvoke(core.clj:170)
    at clojure.lang.RestFn.invoke(RestFn.java:402)
    at clojure.lang.AFn.run(AFn.java:37)
    at java.lang.Thread.run(Thread.java:619)

This one at least mentions foo by name but you're still going to have a headache after a few hours of those stack traces.

Conclusion

So that's five things. You will notice a common theme. Most of these issues are inherited from the JVM. This is to be expected, I suppose. There's no way you can wrap one language in another without a few compromises.

But these things aren't show-stoppers. They are minor annoyances compared to the benefits you get from using the JVM, i.e. the good performance, tons of libraries, cross-platformness, and so on. Clojure is fun enough to work with and wart-less enough that it took me well over two weeks to write this post.

(If you were expecting me to mention loop/recur and the lack of native TCO in the JVM, you were PAINFULLY WRONG. No one who uses Clojure loses sleep over native TCO. It's largely a non-issue that's endlessly repeated by people looking for an excuse to pass up Clojure in favor of $their_pet_language. To each his own, but I have never found myself caring the slightest about loop/recur.)

ABLE

Thu, 04 Jun 2009 10:48:58 -0700

I learned recently of ABLE, which is a minimalistic GUI editor + REPL for Common Lisp. You download it and run it and there you go, Common Lisp with lots of included libraries, no Emacs trial-by-fire necessary.

This is a really great idea in my opinion. I wish I'd had this a year or two ago. Common Lisp has too many barriers for newbies to get started. Anything that removes a few of them can only benefit the Lisp world.

I paid for music

Sun, 24 May 2009 20:06:32 -0700

As a general rule, I don't pay for music. The main reason of course is that the music industry are a bunch of thugs. If you don't know that already, you've been living under a rock for the past few decades. I won't even buy music for other people as a gift if I can help it.

Recently however I did buy music, specifically Jonathan Coulton's latest DVD. JoCo releases his music under Creative Commons, which is awesome, and when you buy it (from What Are Records) you get MP3s that are not infested with DRM, which is also awesome. When you buy that particular DVD, you get a DVD of the concert, a music CD of the same concert, AND you can immediately download MP3s of said concert while you wait for the DVD in the mail. All for $20. Well worth it for such quality music.

I first heard most of JoCo's music via shaky concert recordings on Youtube and via MP3s acquired "elsewhere" (nearly all of which are free downloads on Joco's website though); otherwise I'd never even have known he existed. And yet I ended up giving him my money, happily and willingly, and probably will again. Amazing how things turn out.

The other music I bought recently is Stephen Lynch. Again I heard most of his music first on Youtube. Again I gleefully spent money on his latest CD because it's good music and because it's DRM-less and thug-less entertainment and a good portion of that money is going to the artists.

Most of the music I like comes from Japan or various corners of Europe. Amazon sells a few (very few) Japanese music CDs, for between $50 and $90 each (plus shipping). Do you know how much it costs to ship a stream of bytes from Japan to the US via the intertubes? Hint, it's not $90. How does a stream of bytes increase $90 in value when it's written onto a piece of plastic?

These are strange times. There's such disparity between what the average person believes is right and wrong on the internet and what the law says is lawful and unlawful. This kind of disparity can't last forever. My high school history teacher said that in America at least, a law that is opposed by the majority of citizens in the country never lasts long; I think that's true. And it's as it should be. In a few decades, we're going to look back at how things were in the 90's and 00's and laugh.

More Clojure Mandelbrot Goodness

Wed, 20 May 2009 23:39:23 -0700

After my brief stint in the world of fractal geometry and Clojure, I decided to make a real Mandelbrot set viewer. The resulting source code is here. Here's a simple output (click for bigger version):

It's a pretty naive implementation, barely 100 lines of code, but even with my brute-force approach, given a liberal sprinkling of type hints it runs fast enough. Programming Swing from Clojure couldn't be easier (though I doubt programming Swing from any language is ever really enjoyable, it's a painful bunch of libraries).

There's a discussion of different coloring algorithms on Wikipedia, but even after reading that, getting this thing to look good was difficult. I don't know enough math for it. I ended up cheating and I colored a couple of them in the GIMP, so I could use them as desktop wallpapers.

There are some more PNGS over here including one that's 16000x16000 (producing it almost melted my CPU last night).

Who needs a DB?

Wed, 20 May 2009 22:53:03 -0700

My blog is still working, in spite of my best efforts to crash it. So that's good. But lately I've been thinking that an SQL database is a lot of overkill just to run a little blog like this.

My blog only has around 450 posts total (over the course of many years), and about an equal number of user comments (thanks to all commenters!). Why do I need a full-blown database for that? All of my posts plus comments plus all meta-data is only 2 MB as a flat text file, 700k gzipped.

By far the most complicated part of my blog engine is the part that stuffs data into the database and gets it back out again in a sane manner (translating Clojure data to SQL values, and back again; splitting up my Clojure data structures into rows for different tables, and then re-combining values joined from multiple tables into one data structure). Eliminating that mess would be nice.

Inevitably I ended up with some logic in the database too: enforcing uniqueness of primary keys, marking some fields as NOT NULL, giving default values and so on. But a lot of other logic was in my Clojure code, e.g. higher-level semantic checking, and some things I wanted to set as default values were impossible to implement in SQL.

Wouldn't it be nice for all the logic to be in Clojure? And the data store on disk to be a simple dump of a Clojure data structure? I can (and did) write a few macros to give me SQL-like field declaration and data validation, for uniqueness of IDs and data types etc. For my limited needs it works OK.

The next question is what format to use for dumping to disk. Happily Clojure is Lisp, so dumping it as a huge s-exp via pr-str works fine, and reading it back in later via read-string is trivial.

Some Java data types can't be printed readably by default, for example java.util.Dates, which print like this:

#<Date Wed May 20 22:39:00 PDT 2009>

The #<> reader macro deliberately throws an error if you try to read that back in, because the reader isn't smart enough to craft Date objects from strings by default. But Clojure is extensible; you can specify a readable-print method for any data type like this:

(defmethod clojure.core/print-method java.util.Date [o w]
  (.write w (str "#=" `(java.util.Date. ~(.getTime o)))))

Now dates print as

#=(java.util.Date. 1242884415044)

and if you try to read that via read-string, it'll create a Date object like you'd expect.

user> (def x (read-string "#=(java.util.Date. 1242884415044)"))
#'user/x
user> (class x)
java.util.Date
user> (str x)
"Wed May 20 22:40:15 PDT 2009"

Storing data in a plain file has another benefit of letting me grep my data from a command line, or even edit the data in a text editor and re-load it into the blog (God help me if that's ever necessary).

Having multiple threads banging on a single file on disk is a horrible idea, but Clojure refs and agents and transactions handle that easily. But I do have to work out how not to lose all my data in case the server crashes in the middle of a file update. (I've lost data (in a recoverable way) due to a server crash in the middle of a MySQL update too, so this is a problem for everyone.) Perhaps I'll keep a running history of my data, each update being a new timestamped file, so old files can't possibly be corrupted. Or use the old write-to-tmp-file-and-rename-to-real-file routine. Or heck, I could keep my data in Git and use Git commands from Clojure. It'd be nice to have a history of edits.

If this idea works out I'll upload code for everything to github, as usual.

Clojure: ASCII Mandelbrot Set

Tue, 12 May 2009 18:46:01 -0700

Did you know there's this neat Lisp message board where from time to time someone posts a short problem similar in spirit to the infamous RubyQuiz?

Not a lot of people have participated so far, hopefully that changes. I participated this time; the problem is to render the Mandelbrot Set in ASCII. Here's my Clojure version (based loosely on this one).

(ns mandelbrot
  (:refer-clojure :exclude [+ * <])
  (:use (clojure.contrib complex-numbers)
        (clojure.contrib.generic [arithmetic :only [+ *]]
                                 [comparison :only [<]]
                                 [math-functions :only [abs]])))

(defn- mandelbrot-seq [x y]
  (let [z (complex x y)]
    (iterate #(+ z (* % %)) z)))

(defn- mandelbrot-char [x y]
  (loop [c 126
         m (mandelbrot-seq x y)]
    (if (and (< (abs (first m)) 2)
             (> c 32))
      (recur (dec c) (rest m))
      (char c))))

(defn- mandelbrot-line [xs y]
  (apply str (map #(mandelbrot-char % y) xs)))

(defn- m-range [min max num-steps]
  (range min
         max
         (/ (+ (abs min)
               (abs max))
            num-steps)))

(defn mandelbrot [rmin rmax imin imax]
  (let [rows 30
        cols 50
        xs (m-range rmin rmax cols)
        ys (m-range imin imax rows)]
    (dorun (map #(println (mandelbrot-line xs %)) ys))))

(comment
  ;Example run:
  (mandelbrot -2.0 1.0 -1.5 1.5)
"
~~~~~~~~~~~~}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}
~~~~~~~~~~~~}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}
~~~~~~~~~~}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}
~~~~~~~~~}}}}}}}}}}}}|||||||||}}}}}}}}}}}}}}}}}}}}
~~~~~~~}}}}}}}}|||||||||||||||||||||}}}}}}}}}}}}}}
~~~~~~}}}}}||||||||||||||{{{{zlxz{{{||||}}}}}}}}}}
~~~~~}}}}|||||||||||||{{{{{zzyxpvlz{{{||||}}}}}}}}
~~~~}}}|||||||||||||{{{{{{zzyxvnpwyzz{{{||||}}}}}}
~~~}}|||||||||||||{{{{{{zzyyws   .vyzzz{{|||||}}}}
~~~}||||||||||||{{{{{zzxwwwvus   muvxyywz{|||||}}}
~~}|||||||||||{{{zzzzyyu= p         oteqpz{|||||}}
~~||||||||||{zzzzzzyyyvtm              oxz{{|||||}
~}|||||{{{zyvwxxxxxxxwrG                vuz{|||||}
~||{{{{{zzzywsMsqRovvs                  pxz{{|||||
~|{{{{{zzzyxsq      pj                  `xz{{|||||
~{{{{yyyxwsrp                           wyz{{|||||
~?:3 3 #                              ovxzz{{|||||
~{{{{yyyxwsrp                           wyz{{|||||
~|{{{{{zzzyxsq      pj                  `xz{{|||||
~||{{{{{zzzywsMsqRovvs                  pxz{{|||||
~}|||||{{{zyvwxxxxxxxwrG                vuz{|||||}
~~||||||||||{zzzzzzyyyvtm              oxz{{|||||}
~~}|||||||||||{{{zzzzyyu= p         oteqpz{|||||}}
~~~}||||||||||||{{{{{zzxwwwvus   muvxyywz{|||||}}}
~~~}}|||||||||||||{{{{{{zzyyws   .vyzzz{{|||||}}}}
~~~~}}}|||||||||||||{{{{{{zzyxvnpwyzz{{{||||}}}}}}
~~~~~}}}}|||||||||||||{{{{{zzyxpvlz{{{||||}}}}}}}}
~~~~~~}}}}}||||||||||||||{{{{zlxz{{{||||}}}}}}}}}}
~~~~~~~}}}}}}}}|||||||||||||||||||||}}}}}}}}}}}}}}
~~~~~~~~~}}}}}}}}}}}}|||||||||}}}}}}}}}}}}}}}}}}}}
~~~~~~~~~~}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}
"
)

And here's some obligatory Jonathan Coulton.

Microsoft, you still surprise me

Mon, 11 May 2009 23:18:21 -0700

I use Windows XP at work (not by choice) and I've been continually saying "no" when it tried to install SP3. Why? No tangible reason other than that decades of experience with Windows has shown me that any time you touch any system files or settings in Windows, crap breaks. When it comes to Windows, you set things up and then like a teetering house of playing cards, you back away slowly and try not to breathe.

Which brings us to the other day. I first noticed something was up when a got a popup dialog on my work machine asking me every 15 minutes whether I wanted to Reboot Now or Reboot Later. Confused, I clicked "later" but again and again and again this prompt appeared. After hours of this interrupting my futile attempts at work I relented; I laboriously shut down my half-dozen command prompts and carefully-placed Vim sessions and various server daemons and all the tools I got to look forward to re-opening after Yet Another Unnecessary Reboot, and then I rebooted.

So then XP left me alone and all was well with the world. Ha, just kidding, it started doing the same thing again almost immediately. Reboot Now or Reboot Later? I hatefully tolerated this for as long as I could but it was a futile battle. Microsoft won in the end and I rebooted again.

A few other people at work reported the same thing on their systems, so I thought maybe it was a virus, but I checked a few things and noticed a shiny new SP3 installed on my system (so my initial guess was close). Somehow SP3 was forced onto my machine, not sure if it was the sysadmins pushing it out or Microsoft's doing, but either way: why was it possible to install a Service Pack on my machine without my even being aware it happened? I do not consider this a good thing.

In any case, after the second reboot, strange things happened. My taskbar settings were all reverted to defaults and I noticed my Address Bar was missing. The Address Bar is a little URL/file path bar in the taskbar where you can type a file path and open an Explorer window quickly. One of the very few semi-useful bits of the XP interface.

But it was gone. What happened? A short Google later and I learned that Microsoft removed the feature in SP3 permanently, by design. Why? Because of anti-monopoly regulatory concerns.

Wow. So it turns out I wasn't disappointed, and a few dozen cards toppled from the shaky tower as I watched, helpless. Not the end of the world, but what an annoyance.

The reason I bothered blogging this is because, hilariously enough, you can still add the Address Bar back in SP3. As I read somewhere or other, probably here, you simply 1) Drag a "My Computer" icon to the top of the screen to make a useless "My Computer" toolbar, 2) Right click that and add the Address Bar, which is still an option there, 3) Drag that Address Bar to your main taskbar, 4) Remove the useless toolbar from above. And then you have your Address Bar back. Oops!

So, in summary:

Two forced reboots via 20 repeated un-ignorable popup prompts.
Service Pack installed without my knowledge or consent.
Useful piece of functionality removed.
Item 3 caused by a history of monopolistic business practices and the resulting legal fallout.
Functionality in question removed so incompetently that it can be added back anyways in a matter of seconds.
Another hour of my life sucked into the black hole of the Microsoft Windows User Experience™, forever lost.

Git tutorial

Sat, 09 May 2009 13:55:59 -0700

Finally I found a good Git tutorial that starts from the absolute basics and goes steadily through more advanced things. I highly recommend it.

Crackberry Acquired

Thu, 07 May 2009 18:26:26 -0700

All I ever wanted out of life was to SSH to my computer from a cell phone. That dream has finally come true.

Up to this point I have not owned a cell phone. I bought one a few years back, then I returned it and got a refund because it was pointless. Communicating with other human beings via spoken voice? How trite. My current employer gave me a phone for free but I never used it.

But nowadays cell phones are pretty much mini computers that happen to be able to make phone calls as a side effect. I almost got an iPhone, but I am very wary about hype. Apple's business practices turn me off; the app store is a shystering waiting to happen, their crappy proprietariness makes me puke, their overblown marketing and "image" makes me puke even more. I don't want an MP3 player in my phone; my Cowon D2 is far superior to any silly iPod. And as I tried the touch screen keyboard, I quickly realized that the Blackberry's physical keys win in that category by a mile.

So I got a Blackberry Bold and I'm pretty happy with it so far. I have yet to make a single phone call, but I've put it to good use. I installed all kinds of silly stuff on there, including an SSH client so I can do system maintenance while driving. (Not really, don't worry.) I can look at Google maps when I get lost, which happens embarrassingly often in my car. I can look at Slashdot from the sushi restaurant. I can get the weather updated every 15 minutes, which saves me from rotating my head 25 degrees and looking out the window.

I still object to certain cell phone things on principle. Paying $3 for a 15-second song clip as a ring tone for example; the insanity of this is almost physically painful to me. The Blackberry let me set any old MP3 I wanted as the ring tone though, which is nice.

Paying for text messages is almost as painful. How can it cost a quarter to send 160 bytes of text to another phone, when the whole freaking internet costs orders of magnitude less? How do cell phone companies get away with this? It's such a racket. But I can put IM clients on my phone and use email and I have "unlimited" data transfer each month, so that's nice. (And I really grilled the salesperson about what "unlimited" means. She said some people go into the gigabytes of transfer each month without consequence, so it looks like I need to find a torrent client now!)

Maybe one of these days I'll call someone. What a novel concept.

Vim and Emacs modelines

Tue, 05 May 2009 18:22:47 -0700

Both Vim and Emacs have "windows", resizeable panes inside the main app "frame". Windows are a very useful feature (aside from the ill-chosen name; should've been "panes" or something). I find windows superior to tabs in my opinion; you can easily view more than one file at a time, view two files side-by-side for comparison, and such. Both Vim and Emacs use the border under the window to show helpful information like the filename and editing mode such; this is called the "status line" or "modeline". Again, very handy.

How do you set a nice custom modeline in Vim? Here's my config from my .vimrc:

set statusline=%f\ %2*%m\ %1*%h%r%=[%{&encoding}\ %{&fileformat}\ %{strlen(&ft)?&ft:'none'}\ %{getfperm(@%)}]\ 0x%B\ %12.(%c:%l/%L%)

Yikes! What a mess. It's just a big string with a bunch of special codes and literal characters all mixed up. If you do :h 'statusline' you can read about all the options. That string is ugly though.

Or so I thought, until I witnessed the trainwreck that is Emacs. The documentation for Emacs' modeline comprises a small encylcopedia. The configuration itself is spread across a couple dozen configuration variables. This is the default value for just one of them:

(#("%[" 0 2
   (help-echo "Recursive edit, type C-M-c to get out"))
 #("(" 0 1
   (help-echo "mouse-1: Select (drag to resize)\nmouse-2: Make current window occupy the whole frame\nmouse-3: Remove current window from display"))
 (:propertize
  ("" mode-name)
  help-echo "Major mode\nmouse-1: Display major mode menu\nmouse-2: Show help for major mode\nmouse-3: Toggle minor modes" mouse-face mode-line-highlight local-map
  (keymap
   (mode-line keymap
          (down-mouse-3 keymap
                (abbrev-mode menu-item "Abbrev (Abbrev)" abbrev-mode :help "Automatically expand abbreviations" :button
                     (:toggle . abbrev-mode))
                (auto-fill-mode menu-item "Auto fill (Fill)" auto-fill-mode :help "Automatically insert new lines" :button
                        (:toggle . auto-fill-function))
                (auto-revert-mode menu-item "Auto revert (ARev)" auto-revert-mode :help "Revert the buffer when the file on disk changes" :button
                          (:toggle bound-and-true-p auto-revert-mode))
                (auto-revert-tail-mode menu-item "Auto revert tail (Tail)" auto-revert-tail-mode :help "Revert the tail of the buffer when buffer grows" :enable
                           (buffer-file-name)
                           :button
                           (:toggle bound-and-true-p auto-revert-tail-mode))
                (flyspell-mode menu-item "Flyspell (Fly)" flyspell-mode :help "Spell checking on the fly" :button
                       (:toggle bound-and-true-p flyspell-mode))
                (font-lock-mode menu-item "Font Lock" font-lock-mode :help "Syntax coloring" :button
                        (:toggle . font-lock-mode))
                (glasses-mode menu-item "Glasses (o^o)" glasses-mode :help "Insert virtual separators to make long identifiers easy to read" :button
                      (:toggle bound-and-true-p glasses-mode))
                (hide-ifdef-mode menu-item "Hide ifdef (Ifdef)" hide-ifdef-mode :help "Show/Hide code within #ifdef constructs" :button
                         (:toggle bound-and-true-p hide-ifdef-mode))
                (highlight-changes-mode menu-item "Highlight changes (Chg)" highlight-changes-mode :help "Show changes in the buffer in a distinctive color" :button
                            (:toggle bound-and-true-p highlight-changes-mode))
                (outline-minor-mode menu-item "Outline (Outl)" outline-minor-mode :help "" :button
                        (:toggle bound-and-true-p outline-minor-mode))
                (overwrite-mode menu-item "Overwrite (Ovwrt)" overwrite-mode :help "Overwrite mode: typed characters replace existing text" :button
                        (:toggle . overwrite-mode))
                "Minor Modes")
          (mouse-2 . describe-mode)
          (down-mouse-1 menu-item "Menu Bar" ignore :filter
                (lambda
                  (_)
                  (mouse-menu-major-mode-map))))))
 ("" mode-line-process)
 (:propertize
  ("" minor-mode-alist)
  mouse-face mode-line-highlight help-echo "Minor mode\nmouse-1: Display minor mode menu\nmouse-2: Show help for minor mode\nmouse-3: Toggle minor modes" local-map
  (keymap
   (header-line keymap
        (down-mouse-3 keymap
                  (abbrev-mode menu-item "Abbrev (Abbrev)" abbrev-mode :help "Automatically expand abbreviations" :button
                       (:toggle . abbrev-mode))
                  (auto-fill-mode menu-item "Auto fill (Fill)" auto-fill-mode :help "Automatically insert new lines" :button
                          (:toggle . auto-fill-function))
                  (auto-revert-mode menu-item "Auto revert (ARev)" auto-revert-mode :help "Revert the buffer when the file on disk changes" :button
                        (:toggle bound-and-true-p auto-revert-mode))
                  (auto-revert-tail-mode menu-item "Auto revert tail (Tail)" auto-revert-tail-mode :help "Revert the tail of the buffer when buffer grows" :enable
                             (buffer-file-name)
                             :button
                             (:toggle bound-and-true-p auto-revert-tail-mode))
                  (flyspell-mode menu-item "Flyspell (Fly)" flyspell-mode :help "Spell checking on the fly" :button
                         (:toggle bound-and-true-p flyspell-mode))
                  (font-lock-mode menu-item "Font Lock" font-lock-mode :help "Syntax coloring" :button
                          (:toggle . font-lock-mode))
                  (glasses-mode menu-item "Glasses (o^o)" glasses-mode :help "Insert virtual separators to make long identifiers easy to read" :button
                        (:toggle bound-and-true-p glasses-mode))
                  (hide-ifdef-mode menu-item "Hide ifdef (Ifdef)" hide-ifdef-mode :help "Show/Hide code within #ifdef constructs" :button
                           (:toggle bound-and-true-p hide-ifdef-mode))
                  (highlight-changes-mode menu-item "Highlight changes (Chg)" highlight-changes-mode :help "Show changes in the buffer in a distinctive color" :button
                              (:toggle bound-and-true-p highlight-changes-mode))
                  (outline-minor-mode menu-item "Outline (Outl)" outline-minor-mode :help "" :button
                          (:toggle bound-and-true-p outline-minor-mode))
                  (overwrite-mode menu-item "Overwrite (Ovwrt)" overwrite-mode :help "Overwrite mode: typed characters replace existing text" :button
                          (:toggle . overwrite-mode))
                  "Minor Modes"))
   (mode-line keymap
          (down-mouse-3 keymap
                (abbrev-mode menu-item "Abbrev (Abbrev)" abbrev-mode :help "Automatically expand abbreviations" :button
                     (:toggle . abbrev-mode))
                (auto-fill-mode menu-item "Auto fill (Fill)" auto-fill-mode :help "Automatically insert new lines" :button
                        (:toggle . auto-fill-function))
                (auto-revert-mode menu-item "Auto revert (ARev)" auto-revert-mode :help "Revert the buffer when the file on disk changes" :button
                          (:toggle bound-and-true-p auto-revert-mode))
                (auto-revert-tail-mode menu-item "Auto revert tail (Tail)" auto-revert-tail-mode :help "Revert the tail of the buffer when buffer grows" :enable
                           (buffer-file-name)
                           :button
                           (:toggle bound-and-true-p auto-revert-tail-mode))
                (flyspell-mode menu-item "Flyspell (Fly)" flyspell-mode :help "Spell checking on the fly" :button
                       (:toggle bound-and-true-p flyspell-mode))
                (font-lock-mode menu-item "Font Lock" font-lock-mode :help "Syntax coloring" :button
                        (:toggle . font-lock-mode))
                (glasses-mode menu-item "Glasses (o^o)" glasses-mode :help "Insert virtual separators to make long identifiers easy to read" :button
                      (:toggle bound-and-true-p glasses-mode))
                (hide-ifdef-mode menu-item "Hide ifdef (Ifdef)" hide-ifdef-mode :help "Show/Hide code within #ifdef constructs" :button
                         (:toggle bound-and-true-p hide-ifdef-mode))
                (highlight-changes-mode menu-item "Highlight changes (Chg)" highlight-changes-mode :help "Show changes in the buffer in a distinctive color" :button
                            (:toggle bound-and-true-p highlight-changes-mode))
                (outline-minor-mode menu-item "Outline (Outl)" outline-minor-mode :help "" :button
                        (:toggle bound-and-true-p outline-minor-mode))
                (overwrite-mode menu-item "Overwrite (Ovwrt)" overwrite-mode :help "Overwrite mode: typed characters replace existing text" :button
                        (:toggle . overwrite-mode))
                "Minor Modes")
          (mouse-2 . mode-line-minor-mode-help)
          (down-mouse-1 . mouse-minor-mode-menu))))
 #("%n" 0 2
   (local-map
    (keymap
     (mode-line keymap
        (mouse-2 . mode-line-widen)))
    mouse-face mode-line-highlight help-echo "mouse-2: Remove narrowing from the current buffer"))
 #(")" 0 1
   (help-echo "mouse-1: Select (drag to resize)\nmouse-2: Make current window occupy the whole frame\nmouse-3: Remove current window from display"))
 #("%]" 0 2
   (help-echo "Recursive edit, type C-M-c to get out"))
 #("--" 0 2
   (help-echo "mouse-1: Select (drag to resize)\nmouse-2: Make current window occupy the whole frame\nmouse-3: Remove current window from display")))

What the hell?

Instead of a string, it's an s-expression, which is better, right? Well no, it's still just a big construct with arbitrary meanings assigned to its contents. Lists mean one thing, strings mean another thing (and those strings, like Vim's, can contain special escape sequences). Symbols mean something else, symbols that are keywords mean something else, numbers mean something else, and so on.

To decipher this I had to learn this mini-language. And also learn about "text properties" and a bunch of elisp stuff. It also required knowledge about a bunch of minor modes and how they tie into the modeline, all of which is essentially a bowl of spaghetti code. And keymaps, and maps to control mouse click events and such. Eventually I figured out that most of that crap is controlling tooltip text.

If you do M-x customize-apropos in Emacs and search for "mode-line", you'll get a helpful list of all of the configuration values and their values. (The default values contain literal tab characters, which you can't even type into the customize text fields without C-qing, because tab jumps you between fields. Ughhhhhhh.)

I gave up even trying to get Emacs to have all the helpful information my Vim modeline has. Even deleting the default crap to pare this down to something readable took some effort. This what I ended up with:

(mode-line-format (quote ("%e--[" mode-line-buffer-identification "]" (vc-mode vc-mode) "  " mode-line-modes global-mode-string " %-")))
(mode-line-in-non-selected-windows t)
(mode-line-modes (quote ("%[" "(" (:propertize ("" mode-name)) ("" mode-line-process) (:propertize ("" minor-mode-alist)) "%n" ")" "%]")))
(mode-line ((((class color) (min-colors 88)) (:background "#333333" :foreground "#bcbcbc" :box (:line-width -1 :color "#333333")))))
(mode-line-highlight ((((class color) (min-colors 88)) nil)))
(mode-line-inactive ((default (:inherit mode-line)) (((class color) (min-colors 88) (background dark)) (:foreground "#8b8b8b" :weight light))))

What does that mean? Don't ask me, I can no longer read it. If customize hadn't produced a lot of that for me, I probably wouldn't have managed. My favorite part is the four-deep nested list of lists of lists of lists for the colors.

Verdict?

Vim's modeline is less powerful than Emacs. But who cares if you can't even read it to edit it? Most of Emacs' modeline features are annoying. (My motivation for editing this to begin with was to turn off all the mouse buttons and mini-menus and crap.)

Vim's status line is exactly configurable enough. I don't want to build a small GUI app in my modeline. I want it to show certain bits of information about the buffer, that's it. Vim does this.
Vim script is less elegant than elisp. Or is it? Vim's modeline is a custom DSL for formatting modelines. It's hard to think of anything more concise. Concision is a very good thing. Emacs' version is more general, at the expense of horrid verbosity and unreadability.
Vim's modeline is a string, which means you either write it literally, or you construct it by concatenating lots of other strings. This is a faux pas, right? It's like using eval in Ruby or Perl. It's fragile and error-prone. Emacs uses a Lisp, with its macros and quoted lists and code is data and so on.

But who cares? In this case, a simple string is powerful enough. I don't need a whole Turing-complete programming language to configure a modeline. It's massive overkill and you pay a price for it. The minute a human being is supposed to be keeping track that the second element in the 5-deep nested list means "x", something has gone horribly wrong.

Vim wins this round.

Clojure 1.0

Mon, 04 May 2009 21:04:37 -0700

Clojure 1.0 was released today. For those who care about version numbers, this should be good news. Clojure has been good and "ready" for a good while in my opinion but it's kind of nice to see it made somewhat official.

The pace of development and the level of enthusiasm in the Clojure community are something to behold. Here's hoping it holds up. The community is knowledgeable and helpful and overall positive and people are churning out an awful lot of useful code. Kudos to the Clojure devs and contributors, thanks for the wonderfully fun tool and toy to work and play with.

Clojure Reader Macros

Sat, 18 Apr 2009 17:14:50 -0700

Unlike Common Lisp, Clojure doesn't support user-defined reader macros. You can read some of the rationale for why in this chat log, among other places. I think that's probably a good decision; I don't see a lot of need for mangling the reader. Regular macros get you pretty far already and Clojure has built-in reader support for all the good stuff.

But how hard would it be to have custom reader macros in Clojure if you wanted them? Turns out not too hard if you're willing to ruthlessly break encapsulation and rely on implementation details. Here's one way you could define a dispatch reader macro (i.e. one starting with # and some specified second character):

(defn dispatch-reader-macro [ch fun]
  (let [dm (.get (.getDeclaredField clojure.lang.LispReader "dispatchMacros") nil)]
    (aset dm (int ch) fun)))

Pass in a character and an fn and you get a reader macro. For a silly example let's make reader syntax to uppercase a literal string.

(defn uppercase-string [rdr letter-u]
  (let [c (.read rdr)]
    (if (= c (int \"))
      (.toUpperCase (.invoke
                     (clojure.lang.LispReader$StringReader.)
                     rdr
                     c))
      (throw (Exception. (str "Reader barfed on " (char c)))))))

The function is passed a reader and the dispatch character (which you can usually ignore). I cheat and use Clojure's StringReader to do the real work.

Now I can do this:

user> (dispatch-reader-macro \U uppercase-string)
#<user$uppercase_string__1295 user$uppercase_string__1295@9b59a2>

user> #U"Foo bar BAZ"
"FOO BAR BAZ"

user> (println #U"foo\nbar")
FOO
BAR
nil

user> #U(blarg)
java.lang.Exception: Reader barfed on (

(= "FOO" "foo")
false

(= "FOO" #U"foo")
true

Oh sweet Jesus don't use this in real code, because:

The community will rightly hunt you down with torches and pitchforks.
Reader macro characters are reserved and may conflict with later changes to the core language.
These are set globally, not per-namespace.
And so on. Just don't.

But I think it's a nice demonstration. I've read opinions that Clojure isn't a Real Lisp™ because a lot of Clojure is written in Java and isn't extensible in Clojure itself, but that's generally not true. The reader code for Clojure was all written in Java, but above I modify it from Clojure. There is no line separating Java-land and Clojure-land. It's all one big happy family.

Vim regexes are awesome

Sat, 18 Apr 2009 14:47:54 -0700

Two years ago I wrote about how Vim's regexes were no fun compared to :perldo and :rubydo. Turns out I was wrong, it was just a matter of not being used to them.

Vim's regexes are very good. They have all of the good features of Perl/Ruby regexes, plus some extra features that don't make sense outside of a text editor, but are nonetheless very helpful in Vim.

Here are a few of the neat things you can do.

Very magic

Vim regexes are inconsistent when it comes to what needs to be backslash-escaped and what doesn't, which is the one bad thing. But Vim lets you put \v to make everything suddenly consistent: everything except letters, numbers and underscores becomes "special" unless backslash-escaped.

Without \v:

:%s/^\%(foo\)\{1,3}\(.\+\)bar$/\1/

With \v:

:%s/\v^%(foo){1,3}(.+)bar$/\1/

Far easier to read. Along with \c to turn on and off case sensitivity, these are good options to make a habit of prepending to regexes when needed. It eventually becomes second-nature. See also :h /\v

Spanning newlines

One thing that :perldo and :rubydo can't do is span newlines; you can't combine two lines and you can't break one line into two.

But Vim's regexes can span newlines if you use \_. instead of .. I find this to be a lot more aesthetically pleasing than Perl's horrible s and m modifiers tacked onto the end of a regex. e.g. this strips <body> tags from a text document.

:%s@<body>\v(\_.+)\V</body>@\1@

(Note: in real life, never use a regex to parse HTML or XML. Down that path lies madness. The above is OK because I'd expect only one <body> tag to appear in any document.)

(Note^2: being able to turn on and off magic in the middle of a regex is awfully helpful.)

(Note^4: You can use arbitrary delimiters like @ for the regex, which is useful if your pattern includes literal /'s.)

`\zs`

Vim lets you demand that some text match, but ignore that text when it comes to the substitution part. This is handy for certain specific kinds of regexes. Normally if you want to match some text and then leave it alone in the substitution, you have to capture it and then put it back manually; \zs lets you avoid this.

Say you want to chop some text off the end of a line, but leave the rest of the line alone. Normally you'd have to do this:

:%s/\v^(foobar)(baz)/\1/

to put the foobar back. Of course you can also use a zero-width lookbehind assertion:

:%s/\v(^foobar)@<=baz//

But that's even more line-noise. This is the easiest way:

:%s/^foobar\zsbaz//

See :h /\zs. (And :h /\@<= if you're so inclined.)

Expressions

Using \=, you can put arbitrary expressions on the right side of a regex substitution. For example say you have this text:

~/foo ~/bar

If you do this:

:%s/\v(\S+)/\=expand(submatch(1))/g

You end up with:

/home/user/foo /home/user/bar

Because you can also call your own user-defined functions in the expression part, this can end up being pretty powerful. For example it can be used to insert incrementing numbers into arbitrary places in your text. See :h sub-replace-\=.

And so on

Read :h regexp if you haven't already. Tons of other features in there that can make your life easy if you manage to internalize them. It is difficult to get used to Vim's funky syntax if you're very familiar with Perl/Ruby-style regexes, but I think it's worth it. Only took me two years! (OK, more like a couple days of concerted effort after a year-and-a-half delay.)

Unicomp Customizer keyboard review

Thu, 16 Apr 2009 19:52:39 -0700

I got my Unicomp Customizer 105 in the mail today. This is a keyboard using the same technology as the infamous IBM keyboards of yore.

Why?

The Customizer is an enormous blocky hunk of hard black and grey matte plastic. It is the very antithesis of modern, soft, rounded, Apple-esque fashion. It has no "multimedia" keys, it doesn't glow in the dark, it doesn't have a built-in USB hub, it looks distinctly 80's-ish, and it costs $70. Why on earth would anyone want this thing?

A couple of reasons... one is that it's a status symbol of grizzled old hackers. This keyboard has gotten a lot of good reviews, e.g. last year on Slashdot, but I've heard the sentiment repeated elsewhere. There are stories of people rescuing old IBM keyboards out of dumpsters and selling them on ebay.

If it was simply a status symbol I would look away without a second glance. (Which is why I own a Cowon D2 and not an iPod. I like to research my purchases to the point of paranoia.)

But the popularity seems to be backed up by real functionality and build quality. These keyboards have a reputation for being great to type on due to the unique feel of their buckling spring "clicky" keys, and for being indestructible, with some keyboards still in use after two decades. So I decided why not see for myself?

A keyboard is the main tool of my livelihood and one of the main tools of most of my hobbies. It makes sense to try to get the best tool for the job. The three most important parts of a computer in my opinion are the keyboard, mouse, and monitor. CPU? RAM? Hard disk space? I'll take whatever you give me. But the things I interact with on a constant basis, I want those things to be comfortable.

Clicka clicka clicka

Yeah, this thing is clicky. Even after all the reviews, I was unprepared for just how clicky it is. You can feel the click of each keypress in your fingers and hear the clicking from 3 miles away.

I tried pushing a key down slowly to make it click without activating a keypress, and I found it very difficult if not impossible. You can always tell when you've successfully pressed a key on this keyboard: if it clicked, you did; if it didn't click, you didn't.

One bad thing about the clicking is annoying everyone in the room with you. I'm a bit worried I'm slowly going to drive my wife insane.

Finger workout

The keys have a lot of weight to them compared to the mushy feel of modern keyboards (which usually use some rubber or plastic dome under the keys). The Customizer's keys have little springs in them, and you can feel the keys pushing back on your fingers as you type. It feels much different than any other keyboard I've used.

Is it a good or bad feel? I'm undecided. It does feel pretty good, there's a lot of response to the keyboard and you can more easily tell when you miss a key or flub a keypress and hit two keys at once. I think this probably aids accuracy. I don't type more accurately but I more easily notice my mistakes.

I'm afraid the weight might lead to fatigue though; the keys are harder to press than other keyboards and my hands feel like they're getting a workout in comparison. However I've had a few long nights of typing on this keyboard and haven't noticed any more fatigue than usual, so the worry may be unfounded. On the other hand, I do often notice how annoying it is to type on a laptop which has no resistance and no distance to the keys at all. The resistance in this keyboard is a nice change of pace.

Built well?

I think "indestructible" is probably an apt word. I've only had mine for a couple days, but just hefting the thing, you can tell it's built like a tank. Very thick hard plastic all around. It weighs a ton. If I had to choose a keyboard to use as a weapon in a pinch, I'd grab this one immediately.

The keys come off easily; every key is just a cap over a smaller plastic key beneath, and that cap is a simple piece atop a tube with a spring in it. There isn't a lot of room for mechanical failure here unless you lose the springs. Everything comes off and goes back on very easily, which is nice for when I need to clean out the gunk in a year.

I have heard that if you spill a cup of milk into one of these keyboards, you may find it hard to drain. So don't do that.

Lack of features is a feature

Multimedia keys suck. I've never used them. They waste space and the only time I remember they exist is when I push them accidentally.

The Customizer is very "traditional". There are no multimedia keys, no volume controls, no programmable (i.e. useless) macro keys, no email or internet shortcuts. Just the standard 105 keys. This is a plus in my book.

Caps Lock is slightly shortened with a gap between itself and the A key, which is nice to avoid hitting it accidentally. The version of the keyboard I got has a modern Super ("windows") modifier key, but you can get a version without even that, if you like. Otherwise there are no frills.

Speed typing

I took a couple of silly online typing tests, and I got between 75 and 95 WPM with 98% accuracy, which is as good as I've ever gotten. My six-fingered typing style is a bit odd but this keyboard suits me well.

WPM is a terrible measure of programming speed, because programming has a much higher punctuation-to-letter ratio than English prose. So I also tried an Emacs session and a bunch of Vimming, and I experienced no problems. I forgot I was using this keyboard almost immediately, which is a good thing. It means it wasn't annoying me.

Very important to me, as a Vimmer, is the position and size of the Escape key. I have one other keyboard that has Escape offset to the right a half inch, which is horrendous and messes up my Vimming all the time. My other other keyboard has a tiny little Escape key, half as big as a normal key, which is equally bad.

On the Customizer, Escape is positioned off by itself in the corner as it should be, with a ton of space between itself and the number row, and the Escape key itself is freaking enormous. This is a huge plus in my book. You can't miss Escape on this keyboard.

Similarly, all the other keys are the right sizes and in the right places.

Verdict

So how is the Unicomp Customizer?

It's solid, standard, unique, and has a nice retro, minimalist style that I personally enjoy.

It's also huge, loud, and expensive. Is it worth buying? If you have the money to spend, I think it is. I don't regret the buy after a few days. When I come home from work and start typing on this guy, I'm always pleasantly surprised.

A Sad, Dark Day

Mon, 13 Apr 2009 01:27:12 -0700

Today was a terrible day. I found myself subconsciously trying to use Emacs keystrokes in Vim. I feel dirty. I took a bath but it won't come clean. : (

It just goes to show that you can get used to anything if you do it often enough. Emacs still drives me up the wall but maybe I've achieved a critical mass of enough custom keybindings to let me tolerate it.

Aside from paredit, which has no equal even in Vim, Emacs does have some vaguely non-sucky features. hi-lock is pretty nice (Vim has an equivalent of course). Once I learned a few of the shortcuts for git-emacs I actually found myself using Git much more effectively. Having to drop into a shell to type Git commands is just enough of a disruption to prevent me from doing it often enough. I never got the hang of any version control library in Vim.

I'm almost even getting used to the Emacs buffer model. I find myself C-x bing and flipping back and forth between buffers by name, rather than my Vim practice of opening buffers in certain carefully-placed windows and leaving them there.

On the subject of typing, I broke down finally and ordered a Unicomp Customizer 104 keyboard. I've heard too many hackers say that the old IBM clicky keyboards are good for typing. It should arrive Tuesday, and I'm a lot more excited than anyone should be over a keyboard.

Expect a keyboard review. Try to contain your excitement until then. I know it'll be hard.

KDE4 Konsole Kolor Skheme Kdownload

Sun, 12 Apr 2009 20:31:19 -0700

I put a color scheme for KDE4's Konsole up for download. From a cursory glance I think KDE3 and KDE4 color schemes are the same format, but I haven't tried it.

Also I know I'm not the first to say it, but all of the K's in KDE program names are a bit annoying after a while, aren't they?

Blog and CRUD

Sun, 12 Apr 2009 20:01:35 -0700

I updated my blog source code on github. I also split my CRUD library out into its own clj-crud repo. It is cruddy, so the name is apt.

This code still isn't polished enough for someone to drop it on a server and fire it up, but maybe it'll give someone some ideas. I think the new code is cleaner and it'll be easier for me to add features now.

Beware bugs, I'm positive I introduced some.

EDIT: A word about the CRUD library... persisting data to disk is hard when the data may be mutated by many threads at once and the destination for your data is an SQL database that may or may not even be running. I have more respect for people who've written libraries that actually do this kind of thing and work right. Granted I only spent 3 days on mine but still, it's tricky.

I gave up for a while and tried clj-record, but it was prohibitively slow. It has the old N+1 queries problem when trying to select an object which has N sub-objects. In real life you'd write SQL joins to avoid such things. Ruby on Rails on the other hand gets around this via some nasty find syntax.

I get around it by having all my data in a Clojure ref in RAM already so it doesn't matter. And by using hooks so each object keeps a list of its sub-objects and the list is always up-to-date (updates of sub-objects propagate to their parents). But the crap I have to do to get this to just barely work is pretty painful.

Lisp Syntax Doesn't Suck

Wed, 08 Apr 2009 04:03:12 -0700

I spend a lot of time talking about what I don't like about various languages, but I never talk about what I do like. And I do like a lot, or I wouldn't spend so much time programming and talking about programming.

So here goes. I like the syntax of Lisp. I like the prefix notation and the parentheses.

Common Complaints

A common criticism of Lisp from non-Lispers is that the syntax is ugly and weird. The parentheses are impossible to keep balanced. It ends up looking like "oatmeal with fingernail clippings mixed in".

Also, prefix notation is horrible. 1 + 2 is far superior to (+ 1 2). Infix notation is how everyone learns things and how all the other languages do it. There are countless numbers of people (example) who have proposed to "fix" this, to give Lisp some kind of infix notation. The topic inevitably comes up on Lisp mailing lists and forums.

Partly this is subjective opinion and can't be argued with. I can't say that Lispy parens shouldn't be ugly for people, any more than I can say that someone is wrong to think that peanut butter is gross even though I like the taste of it. But in another sense, does it matter that it's painful? Does it need to be changed? Should the weird syntax stop you from learning Lisp?

Prefix Notation: Not Intuitive?

There is no "intuitive" when it comes to programming. There's only what we're used to and what we aren't.

What does = mean in a programming language? Most people from a C-ish background will immediately say assignment. x = 1 means "give the variable/memory location called X the value 1".

For non-programmers, = is actually an equality test or a statement of truth. 2 + 2 = 4; this is either a true or false statement. There is no "assignment". The notion of assignment statements is an odd bit of programming-specific jargon. In most programming languages we've learned instead that == is an equality test. Of course some have := for assignment and = for equality tests. But = and == seems to be more common. Some languages even have ===. Or x.equals(y). Even less like what we're used to. (Don't get started on such magic as +=.)

Most of us have no problem with these, after a while. But few of us were programmers before we learned basic math. How many of us remember the point in time when we had to re-adjust our thinking that = means something other than what we've always learned it to mean? I actually do remember learning this, over a decade ago. This kind of un-learning is painful and confusing, there's no question.

But it's also necessary, because these kinds of conventions are arbitrary and vary between fields of study (and between programming languages). And there are only so many symbols and words available to use, so we re-use them. None of the meanings for = is "right" or more "intuitive" than the other. = has no inherent meaning. It means whatever we want it to mean. Programming is chock-full of things like this that makes no sense until you memorize the meaning of them.

Consider a recent article that got a lot of discussion, about why all programmers should program in English. How much less intuitive can you get, for a native speaker of another language to program using words in English? Yet they manage. (Have you ever learned to read sheet music? Most of the terms are in Italian. I don't speak a word of Italian, yet I managed.)

The point is that it's very painful to un-learn things that seem intuitive, and to re-adjust your thinking, but it's also very possible. We've all done it before to get to where we are. We can all do it again if we need to.

Prefix notation is unfamiliar and painful for many people. When I first started learning Lisp, the prefix notation was awfully hard to read without effort, even harder to write. I would constantly trip up. This is a real distraction when you're trying to write code and need to concentrate. But it only took me maybe a week of constant use to ingrain prefix notation to the point where it didn't look completely alien any longer.

At this point prefix notation reads to me as easily as infix notation. I breeze right through Lisp code without a pause. In Clojure, you can write calls to Java methods in Java order like (. object method arg arg arg) or you can use a Lispy order like (.method object arg arg arg); I find myself invariably using the Lispy way, as does most of the community, even though the more traditional notation is available.

You can get used to it if you put in a minimal amount of effort. It's not that hard.

Benefits of Prefix Notation

Why bother using prefix notation if infix and prefix are equally good (or bad)? For one thing, prefix notation lets you have variable-length parameter lists for things that are binary operations in other languages. In an infix language you must say 1 + 2 + 3 + 4 + 5. In a prefix language you can get away with (+ 1 2 3 4 5). This is a good thing; it's more concise and it makes sense.

Most languages stop at offering binary operators because that's as good as you get when you have infix operators. There's a ternary operator x?y:z but it's an exception. In Lisp it's rare to find a function artificially limited to two arguments. Functions tend to take as many arguments as you want to throw at them (if it makes sense for that function).

Prefix notation is consistent. It's always (function arg arg arg). The function comes first, everything else is an argument. Other languages are not consistent. Which is it, foo(bar, baz), or bar.foo(baz)? There are even oddities in some languages where to overload a + operator, you write the function definition prefix, operator+(obj1, obj2), but to call that same function you do it infix, obj1 + obj2.

The consistency of Lisp's prefix notation opens up new possibilities for Lispy languages (at least, Lisp-1 languages). If the language knows the first thing in a list is a function, you can put any odd thing you want in there and the compiler will know to call it as a function. A lambda expression (anonymous function)? Sure. A variable whose value is a function? Why not? And if you put a variable whose value is a function in some place other than at the start of a list, the language knows you mean to pass that function as an argument, not call it. Other languages are far more rigid, and must resort to special cases (like Ruby's rather ugly block-passing syntax, or explicit .call or .send).

Consistency is good. It's one less thing you have to think about, it's one less thing the compiler has to deal with. Consistent things can be understood and abstracted away more easily than special cases. The syntax of most languages largely consists of special cases.

Parens: Use Your Editor

The second major supposed problem with Lisp syntax is the parens. How do you keep those things balanced? How do you read that mess?

Programming languages are partly for human beings and partly for computers. Programming in binary machine code would be impossible to read for a human. Programming in English prose would be impossible to parse and turn into a program for a computer. So we meet the computer halfway. The only question is where to draw the line.

The line is usually closer to the computer than to the human, for any sufficiently powerful language. There are very few programing languages where we don't have to manually line things up or match delimiters or carefully keep track of punctuation (or syntactic whitespace, or equivalent).

For example, any language with strings already makes you pay careful attention to quotation marks. And if you embed a quotation mark in a quote-delimited string, you have to worry about escaping. And yet we manage. In fact I think that shell-escaping strings is a much hairier problem than balancing a lot of parens, but we still manage.

This is sadly a problem we must deal with as programmers trying to talk to computers. And we deal with it partly by having tools to help us. Modern text editors do parenthesis matching for you. If you put the cursor on a paren, it highlights the match. In Vim you can bounce on the % key to jump the cursor between matching parens. Many editors go one step further and insert the closing paren whenever you insert an opening one. Emacs of course goes one step further still and gives you ParEdit. Some editors will even color your parens like a rainbow, if that floats your boat. Keeping parens matched isn't so hard when you have a good editor.

And Lisp isn't all about the parens. There are also generally-accepted rules about indentation. No one writes this:

(defn foo [x y] (if (= (+ x 5) y) (f1 (+ 3 x)) (f2 y)))

That is hard to read, sure. Instead we write this:

(defn foo [x y]
  (if (= (+ x 5) y)
    (f1 (+ 3 x))
    (f2 y)))

This is no more difficult to scan visually than any other language, once you're used to seeing it. And all good text editors will indent your code strangely if you forget to close a paren. It will be immediately obvious.

A common sentiment in various Lisp communities is that Lispers don't even see the parens; they only see the indentation. I wouldn't go that far, but I would say that the indentation makes Lisp code easily bearable. As bearable as a bunch of gibberish words and punctuation characters can ever be for a human mind.

When I was first learning Lisp I did have some pain with the parens. For about a week. After learning the features of Vim and Emacs that help with paren-matching, that pain went away. Today I find it easier to work with and manipulate paren-laden code than I do to work with other languages.

Benefits of the Parens

Why bother with all the parens if there's no benefit? One benefit is lack of precedence rules. Lisp syntax has no "order of operations". Quick, what does 1 + 2 * 3 / 4 - 5 mean? Not so hard, but it takes you a second or two of thinking. In Lisp there is no question: (- (+ 1 (/ (* 2 3) 4)) 5). It's always explicit. (It'd look better properly indented.)

This is one less little thing you need to keep in short-term memory. One less source of subtle errors. One less thing to memorize and pay attention to. In languages with precedence rules, you usually end up liberally splattering parens all over your code anyways, to disambiguate it. Lisp just makes you do it consistently.

As I hinted, code with lots of parens is easy for an editor to understand. This make it easier to manipulate, which makes it faster to write and edit. Editors can take advantage, and give you powerful commands to play with your paren-delimited code.

In Vim you can do a ya( to copy an s-exp. Vim will properly match the parens of course, skipping nested ones. Similarly in Emacs you can do C-M-k to kill an s-exp. How do you copy one "expression" in Ruby? An expression may be one line, or five lines, or fifty lines, or half a line if you separate two statements with a semi-colon. How do you select a code block? It might be delimited by do/end, or curly braces, or def/end, or who knows. There are plugins like matchit and huge syntax-parsing scripts to help editors understand Ruby code and do these things, but it's not as clean as Lisp code. Not as easy to implement and not as fool-proof that it'll work in all corner cases.

ParEdit in Emacs gives you other commands, to split s-exps, to join them together, to move the cursor between them easily, to wrap and unwrap expressions in new parens. This is all you need to manipulate any part of Lisp code. It opens up possibilities that are difficult or impossible to do correctly in a language with less regular syntax.

Of course this consistency is also partly why Lisps can have such nice macro systems to make programmatic code-generation so easy. It's far easier to construct Lisp code as a bunch of nested lists, than to concatenate together strings in a proper way for your non-Lisp language of choice to parse.

Conclusion

Yeah Lisp syntax isn't intuitive. But nothing really is. You can get used to it. It's that not hard. It has benefits.

Sometimes it's worth learning things that aren't intuitive. You limit yourself and miss out on some good things if you stick with what you already know, or what feels safe and sound.

Disabling Ctrl-Alt-Backspace

Sat, 04 Apr 2009 23:44:41 -0700

After being reminded the hard way yet again that C-S-Backspace in Emacs invokes the very handy kill-whole-line function, but that C-M-Backspace, while uncomfortably similar to that key-chord, does something very different, I have now officially added to my /etc/X11/xorg.conf:

Section "ServerFlags"
    Option "DontZap" "True"
EndSection

to prevent me from accidentally murdering my X server at the worst possible times.

Vim cterm colors

Sat, 04 Apr 2009 12:48:06 -0700

Note to self. Vim color schemes that only set cterm colors don't work unless you export TERM=xterm-256color in your terminal emulator. Konsole in KDE4 seems to default to plain xterm. Took me a half hour to figure out why my color scheme wasn't working in Konsole.

Real Confusing Haskell

Thu, 02 Apr 2009 23:44:54 -0700

I can pinpoint the exact page in Real World Haskell where I became lost. I was reading along surprisingly well until page 156, upon introduction of newtype.

At that my point my smug grin became a panicked grimace. The next dozen pages were an insane downward spiral into the dark labyrinth of Haskell's type system. I had just barely kept data and class and friends straight in my mind. type I managed to ignore completely. newtype was the straw that broke the camel's back.

As a general rule, Haskell syntax is incredibly impenetrable. => vs. -> vs. <-? I have yet to reach the chapter dealing with >>=. The index tells me I can look forward to such wonders as >>? and ==> and <|>. Who in their right mind thought up the operator named .&.? The language looks like Japanese emoticons run amuck. If and when I reach the \(^.^)/ operator I'm calling it a day.

Maybe Lisp has spoiled me, but the prospect of memorizing a list of punctuation is wearisome. And the way you can switch between prefix and infix notation using parens and backticks makes my eyes cross. Add in syntactic whitespace and I don't know what to tell you.

I could still grow to like Haskell, but learning a new language for me always goes through a few distinct stages:

Curiosity -> Excitement -> Reality Sets In -> Frustration -> Rage ...

At Rage I reach a fork in the road: I either proceed through Acceptance into Fumbling and finally to Productivity, or I go straight from Rage to Undying Hatred. Haskell could still go either way.

My Poor Headphones

Thu, 02 Apr 2009 11:45:01 -0700

My precious Grado SR-80's needed some emergency surgery a while back, resulting in this disaster. They still work today, in the sense that sound is still emitted from them, but in terms of aesthetics, the situation has rapidly deteriorated. I've got bare wire and sticky electrical tape hanging all over the place. Also I'm probably one good yank away from snapping the wires off again.

If anyone reading this has a good tutorial or information on re-wiring a set of headphones, it'd be appreciated. I've never soldered anything in my life. I don't know where to acquire the wires; I imagine any wire will do, but I'm clueless when it comes to such things. I think I might like to do something like this mod and run the wire up over the top, to prevent the inevitable twisting from destroying the wires in the future, but I'm uncertain I could pull it off without complete destruction.

(At least I know enough about these things to cringe when people start talking about the "performance" of their headphone wires. $400 for a hunk of wire? Wow.)

Trying Arch

Wed, 01 Apr 2009 00:52:41 -0700

Thanks to all who gave helpful suggestions about running VMs in Gentoo. The main reason I wanted a VM was to play around with some other distros and see what I liked.

But then I got to thinking, and I realized that I have over 250 GB of free hard drive space sitting around. So I made a new little partition and per Noah's suggestion, threw Arch Linux on there.

I'm fairly impressed so far. The install was easy. In contrast to the enormous Gentoo handbook, the whole Arch install guide fits on one page of the official Arch wiki. Why doesn't Gentoo have an official wiki? I know there are concerns over the quality of something anyone can edit, but in practice is it a big a deal? Is it worth the price of sending users elsewhere, to potentially even WORSE places, when the Gentoo docs don't cover everything we need? The quality of the unofficial Gentoo wiki is often very good but sometimes hit-or-miss, and it also sort of crashes and loses all data without backups every once in a while.

The Arch installer is a commandline app using ncurses for basic menus and such, which is more than sufficient and a good compromise between commandline-only and full-blown X-run Gnome bloat. The install itself went fine, other than my own mistakes. I'm sharing /boot and /home between Gentoo and Arch so I can switch between them easily. During the install Arch tried to create some GRUB files, but they already existed care of Gentoo, so the install bombed without much notification and I didn't notice until 3 steps later. No big deal to fix, but I'd have liked a louder error message right away when it happened. The base install took about 45 minutes.

Another nice thing is that the Arch install CD has vi on it. I didn't have to resort to freaking nano or remember to install vim first thing. A mild annoyance to be sure, but it bugged me every time I installed Gentoo.

After boot, installing apps via pacman is simple enough. KDE 4.2 installed in about 15 minutes, as you'd expect from a distro with binary packages. I found a mirror with 1.5 Mb/sec downloads, which is awfully nice. Syncing the package tree takes less than 2 seconds, which is also nice compared to Portage's 5-minute rsync and eix update times. Searching the tree via regex is also somehow instantaneous in Arch.

Oddly, KDE didn't seem to pull in Xorg as a dependency, but other dependencies worked fine so far. Time will tell how well this all holds up. Most package managers do fine on the normal cases but the real test is the funky little obscure apps. pacman -S gvim resulted in a Vim with working rubydo and perldo, which means Arch passed the Ubuntu stink test.

Another nice thing is that KDE4 actually works. My Gentoo install is years old and possibly crufted beyond repair, or something else was wrong, but I have yet to get KDE4 working in Gentoo without massive breakage. Possibly if I wiped Gentoo and tried KDE4 without legacy KDE3 stuff everywhere it'd also be smooth.

Regardless, it all works in Arch. NVidia drivers and Twinview settings were copy/pasted from Gentoo, and compositing all works fine. No performance problems in KDE with resizing or dragging windows, no Plasma crashes (yet), no missing icons or invisible notification area. QtCurve works in Qt3, Qt4 and GTK just fine. My sound card worked without any manual configuration at all. My mouse worked without tweaking, including the thumb buttons. Same with networking, the install prompted me for my IP and gateway etc. and then it worked, no effort.

I've mentioned before, but one nice thing about Linux is that if you have /home in its own partition, it's no big deal at all to share it between distros. With no effort at all I'm now using all my old files and settings in Arch, and I can switch back and forth between this and Gentoo without any troubles.

So we'll see how this goes. So far so good though. Arch seems very streamlined and its goal is minimalism, which is nice. Gentoo has not felt minimalistic to me in a while. Again, may be due to the age of my install, cruft and bit-rot.

Gentoo VMWare Fail

Tue, 31 Mar 2009 00:59:55 -0700

According to this bug, VMWare on Gentoo is in a sorry state, with one lone person trying to keep it going. I can't get vmware-modules to compile on my system no matter what I try, which is depressing. Kudos to all of our one-man army Gentoo devs who are keeping various parts of the distro going, but I wonder how many other areas of Gentoo are largely unmaintained nowadays.

KVM was braindead simple to get set up in comparison with VMWare, but I can't get networking to work. This is because I'm an idiot when it comes to TUN/TAP and iptables. I've read wiki articles that suggest setting up my system to NAT-forward traffic into the VM but I couldn't get that working and don't have a lot of time to screw with it.

On one of the Gentoo mailing lists I noticed that a dev has posted some KVM images of Gentoo suitable for testing. But I'm looking to start up an image from scratch and that doesn't help, and it's not going to help me get networking going any easier.

Why do I feel like this'd take 10 minutes to set up on Ubuntu? Look at this, or search for "ubuntu vmware" and see the hundreds of results. Given that it's a VM and it doesn't really matter what the host OS is anyways, I'll probably do that on my laptop, but it's still depressing.