briancarper.net (λ)

Vim: Find lines longer than current line

Thu, 21 Jun 2012 10:29:14 -0700

Vim isn't just for editing source code. It's also a pretty good data editor.

Sometimes I'm editing dirty fixed-width data files, and some lines are incorrectly a bit too long because one or two fields overflowed. It's easy to search for lines that are too long, once you know how long lines should be. Say your lines should be 1000 characters; search like this:

/^.\{1001}

Now all the longer lines are highlighted (assuming :set hls).

That's tedious to type though. Put your cursor on a non-overflowing line and call this function, and Vim will find all the lines that are too long, and move you to the first one:

function! FindLongerLines()
    let @/ = '^.\{' . col('$') . '}'
    silent! norm n$
endfunction

That mess about manually setting @/ is because of this somewhat unfortunate fact (from :help :function):

        The last used search pattern and the redo command "."
        will not be changed by the function.  This also
        implies that the effect of |:nohlsearch| is undone
        when the function returns.

That is, using something like norm / inside functions doesn't affect the last-remembered search once the function returns, but that's exactly what we want to happen.

Happy Day Against DRM

Fri, 04 May 2012 13:23:52 -0700

Books are 50% off at O'Reilly today, using code DRMFREE. (This includes my book, Clojure Programming, by the way...) I'm a bit late with this, given the offer expires in 9 hours, but there's still time.

Whether you want to buy books today or not, it's worth pointing out that today is International Day Against DRM!

Brand Loyalty. Step 1: Make good stuff.

My anti-DRM article is quickly going to turn into a pro-O'Reilly Media infomercial, so you've been warned.

I am not the kind of person to feel any kind of brand loyalty. I'm the kind of person who deliberately buys a different brand of peanut butter every time I go to the grocery store, to try to screw with the store's customer-tracking database.

O'Reilly is probably an exception. I like O'Reilly. Why is that?

First, O'Reilly books tend to be pretty good. At least, I have yet to buy one that wasn't pretty good.

Allow me to digress. My college's CS curriculum was based around C++. Now, I'm the kind of person who thinks that programming is vaguely enjoyable no matter what I'm doing. Computers are fun. But for a new programmer, coding in C++ is like an hours-long shouting match with the compiler where your goal is to try to get the compiler errors to shut up. Producing a working program is an occasional side-effect. C++ doesn't exactly promote explorative, imaginative programming.

The first class I had in college where I actually enjoyed programming was a class that taught Perl. My textbook was Learning Perl, aka the Llama Book¹. What a good book. I still have it. I remember feeling like I learned more reading that book that I had in two years of slogging through C++ data structures. And what fun Perl was.

I remember immediately spending a bunch of money I should've saved for food, and getting Programming Perl, aka the Camel Book¹. So good! Who knew a book could be witty and fun, and teach you things at the same time. You can tell when a book is written by someone who knows their stuff, and who enjoys talking about their craft.

Not sure if it was Perl itself, or the great Perl books, or probably some combination. But I've been cemented in dynamic, vaguely-Perly, powerful and fun languages since then. First Ruby, then Clojure.

I'm also likely to buy an O'Reilly book, given a choice between alternatives.

Step 2: Be Humans and give a crap.

A second thing that creates brand loyalty is when a company seems to be made of human beings that you can relate to.

When I heard O'Reilly was writing a Lisp book, and what's more, it was a Clojure book, and what's more, I could be involved in writing it... I was pretty excited.

Our book was written in ASCIIDOC, and lived in an SVN repo hosted at O'Reilly.² We could upload code with a certain string in the SVN commit log, and that'd trigger a rebuild of the ASCIIDOC on O'Reilly's server, which was compiled into PDF, and then we could download the PDF from SVN to see how the final product would look. Turnaround time was about 10 minutes. It was a nice, programmer-friendly setup, to be sure.

Whenever I dealt with people at O'Reilly, I generally got the feeling that I was working with programmers, or people who cared about programming. There aren't a lot of Clojure gurus there, but there were people who knew why wrapping long lines of could needed to be handled just right.

It's a great feeling to work with people whose goal is advancing the craft, as opposed to some kind of Death-Star-like entity whose goal is wringing extra pennies out of customers' bones.

DRM sucks

So does O'Reilly actually give a crap? Well, fiiiiiiiiinally getting to the point: O'Reilly's stance on DRM is pretty much spot-on. O'Reilly books are sold without DRM. DRM is not the way to make good stuff. DRM is a good sign that you don't give a crap. DRM doesn't advance the craft, but rather does the opposite.

I leant a guy my copy of K%R a while back. Now there's one more person in the world with a bit more knowledge of C. This is a really good thing. If my copy of K&R was a DRMed ebook that I couldn't lend out, the world would be a tangibly worse place.

I highly recommend this article by Mike Hendrickson at O'Reilly where he talks about piracy, DRM, and making books. Also this one by Tim O'Reilly where he talks about the same.

Now that my name is on a book, have my opinions about DRM changed? Not really. I'd obviously prefer that people pay for my book. I pay for books. It's only fair.

At the same time, I would be really disappointed if my book was sold with DRM all over it, and I'm glad it isn't.

Treating your customers like thieves a priori is not the way to build brand loyalty. Thinking that DRM is going to stop anyone from pirating a book is pretty much delusional. Using DRM to maintain some kind of iron-fisted control over stuff you're selling to other people is morally sketchy.

DRM is not the way to advance the craft. Advancing the craft is the important thing.

When you make smart decisions like not selling DRMed books, the result could be dorks like me spending an hour or two unprovoked, writing an article about how good your company is. And yeah, this is surely a bit self-serving because I want to sell my book, but I'd have written this same article two years ago too.

One way to tell a good book is if it's widely known by an affectionate nickname or acronym. K&R? TAOCP? SICP? The Camel Book? You probably know what I mean right away. ↩
Obviously I'd have preferred Git, but I'll take what I can get. ↩

cursorcolumn / cursorline slowdown

Mon, 30 Apr 2012 15:48:25 -0700

The cursorcol and cursorline options in Vim are great. Enabling them, and setting up your syntax highlighting correctly, will highlight the line and column that contains the cursor, drawing a sort of "crosshairs", to let you find the cursor easily.

This is especially useful when editing non-sourcecode files, like giant fixed-with data files. Or when you need to keep switching your attention back and forth from Vim to something else; the visual cue to draw your eyes back to the cursor can be useful to prevent a mental page fault.

Great. However, the help info for cursorcolumn says this, in part:

    Highlight the screen column of the cursor with CursorColumn
    |hl-CursorColumn|.  Useful to align text.  Will make screen redrawing
    slower.

"Will make screen redrawing slower" is an understatement, unfortunately. Over the past who-knows-how-long, I've noticed Vim slowing to a crawl when editing certain files, mostly big Ruby files. Moving the cursor around or scrolling the window became pretty painful. I could never quite figure out why, but today I got sick of it, and eventually found an old message on the Vim mailing list explaining the problem.

Apparently when you have cursorcolumn or cursorline enabled, the whole screen is redrawn every time you move the cursor. That explains a lot. When I disabled these options, editing complex Ruby files once again achieved notepad.exe-level speed.

I guess there's this:

function! CursorPing()
    set cursorline cursorcolumn
    redraw
    sleep 50m
    set nocursorline nocursorcolumn
endfunction

nmap <C-Space> :call CursorPing()<CR>

This will flash the cursor crosshairs for 50 milliseconds when I hit CTRL+Space in normal mode. Better than nothing.

Split page vertically in CSS (minus pixels)

Mon, 23 Apr 2012 21:43:06 -0700

I was designing an online database application recently. The layout I wanted was, I thought, fairly simple:

N pixel header at the top
The rest of the page split vertically into two panes
Each pane should scroll independently

Super easy to do in CSS, right? Of course not! You can't do this:

#header {  height: 50px; }

#panels {  height: 100% - 50px; }

#top, #bottom { overflow: auto; }

This is because (of course) you can't do simple arithmetic in CSS.

I can't think of a reason why it's not supported. My browser knows the height of the window at any given point in time. The browser can surely subtract two numbers. If someone knows of a solid reason why we can't do this in CSS, please clue me in.

I can think of many reasons why I would want to do it though. The above use case is just one of them.

I really dislike resorting to this (which does work, as seen here):

#header {  height: 50px; }

#panels {
    position: absolute;
    top: 50px;
    left: 0px;
    right: 0px;
    bottom: 0px;
}

#top, #bottom { overflow: auto; }

Whenever I start using absolute positioning, I know something went off the rails somewhere.

The worst part isn't that CSS doesn't support this, it's that even if CSS did suddenly support it, I couldn't use it until sometime in 2023 when all the major browsers implemented it and everyone using the old browsers switched or died of old age.

I don't have cancer

Wed, 04 Apr 2012 14:29:58 -0700

2011 was an interesting year. A year of firsts!

I worked on my first book, Clojure Programming (soon to be released, in fine bookstores near you).
I bought my first house.
The first of my maternal grandparents died.

Hmm, kind of took a turn for the worst there. Then, one fine sunny day in 2011, sitting at my favorite pub, enjoying my favorite beer, I started coughing up blood. Another first!

More firsts:

First Emergency Room visit.
First bronchoscopy.
First CT scan.

Coughing up blood¹ is caused by a huge number of things, from nose bleeds to lung cancer to food going down the wrong pipe to cocaine use. It turns out that 30-year-old non-smokers with no other symptoms tend not to have lung cancer. That didn't stop me from fearing the worst.

Canadian Health Care

This was my first chance to experience the Canadian health care system first-hand after immigrating here. "Free health care" is not entirely accurate, but is very close. I pay some small amount of money monthly ($30-40 I think) to be included in the government-provided Medical Services Plan (MSP). Many employers pay this fee for their employees, but mine doesn't. No big deal.

Once in this system, every "essential" form of health care is paid for completely by the government. Emergency room visit, bronchoscopy, blood test, x-ray, visit with my family doctor, visit with my pulmonologist, all of it was 100% paid for. Show them a government "Care Card" and you're set.

Prescriptions are not covered. Things like eyeglasses, non-emergency dentistry, and elective procedures are not covered. I can get private health insurance to pay for some of those things, but I never bothered, because the cost of that stuff is so low.

I'd hesitantly call this a step up from the US system of huge numbers of people being uninsured, and of insurance not actually covering all of your medical expenses even if you have it.

The one bad thing about Canadian health care is the wait times. It's often a month or longer to get an appointment to see my pulmonologist. I'm currently scheduled for another medical test... in June. This was scheduled about 4 weeks ago. Thank God I didn't have cancer, or I'd probably have been dead before I got to see a doctor.

I never went through a similar experience in the US, so I'm not sure what the wait times are like in comparison. I do remember my father waiting for over a month (in severe pain) to have a surgery performed because his insurance company dragged their feet in approving it, or something like that. So yeah. I probably can't complain much.

Now what?

After months of waiting and months of not knowing, and then having a few cameras shoved into my lungs, it turns out I probably don't have cancer. So that's pretty good news. I still don't know what's causing me to sporadically cough up blood, but as more and more "serious" things are ruled out by tests, I find myself in much better spirits.

2011 will go down in my biography² in as the Year of Lost Productivity. I didn't handle the stress very well, to put it very mildly. It's unfortunate that the act of worrying about dying and not having time to do things I want to do ended up hindering me from doing many things I wanted to do.

I sometimes hear about people who actually have terminal illness showing bravery in the face of their illness. By contrast, it didn't even take terminal illness to essentially blow me out of the water. Just the real threat of it. I feel a lot of shame and regret at how poorly I handled myself. I'm trying to use that regret as motivation. I have a lot of things I need to accomplish, and who knows, maybe not as much time to accomplish them as I'd like to imagine.

So I have a lot of plans for this year. Old projects need to be dusted off and brought up to speed. Step one is probably kicking some life back into this old blog.

Hemoptysis. From Greek hemo (blood) + ptýsis (spitting). A word I'm now intimately familiar with. ↩
I'm not actually writing a biography. ↩

Shuffle lines in Vim

Thu, 07 Jul 2011 14:07:41 -0700

In a pinch, I needed to randomize the order of a few thousand lines of plain text. In Linux you can just pipe the file through sort, even right inside Vim:

:%!sort -R

But I was stuck on Windows. And I don't know how to randomize a file in native Vim script. But doing it in Ruby is pretty easy, and luckily, Vim has awesome Ruby support. Tne minutes' work and a few peeks at :h ruby and we have a successful, working kludge:

function! ShuffleLines()
ruby << EOF
    buf = VIM::Buffer.current
    firstnum =  VIM::evaluate('a:firstline')
    lastnum = VIM::evaluate('a:lastline')
    lines = []
    firstnum.upto(lastnum) do |lnum|
      lines << buf[lnum]
    end
    lines.shuffle!
    firstnum.upto(lastnum) do |lnum|
      buf[lnum] = lines[lnum-firstnum]
    end
EOF
endfunction

2011-07-07 23:32 - Edited to remove a superfluous line.

2011-07-09 21:33 - Wrong parameter for sort, oops.

Keyword Arguments: Ruby, Clojure, Common Lisp

Fri, 24 Jun 2011 17:22:48 -0700

And suddenly I return to blogging, rising from the ashes like some kind of zombie phoenix. Turns out writing a book is a good absorber of time, like some sort of heavy-duty temporal paper towel. Now that I've gotten the terrible similes out of my system, let's talk about keyword arguments, one of my favorite features in any language that supports them.

Ruby, Clojure, and Common Lisp are all languages I enjoy to some degree, and they all have keyword arguments. Let's explore how keyword args differ in those languages.

Why keyword arguments?

Why are keyword arguments good?

You can omit arguments.
You can supply arguments in an arbitrary order.
Arguments are labeled, so you know what argument means what.

Positional arguments require mentally lining up the 7th argument in your function call with the 7th argument in the function signature, and so on. Keyword arguments become more and more attractive the more arguments you have in your function signature.

Keyword arguments trade a bit of verbosity for added explicitness, clarity and reduced mental burden. (Kind of like Lisps do overall. Fancy that.)

Ruby

Ruby doesn't have special support for keyword arguments. But Ruby likes its hashes, so you can just pass one in as an argument to a function. As some syntax sugar, if you pass a "flat" list of :key => val pairs, Ruby slurps them all together and stuffs them into a hash for you.

def foo(arg)
  p arg
end

foo({:x => 123})                  # => {:x=>123}
foo(:x => 123)                    # => {:x=>123}

With even more added sugar, you can leave off the parens in Ruby function calls. So this is pretty common in Ruby:

foo :x => 123                     # => {:x=>123}

How nice and punctuation-less. But then things get ugly. What about this?

foo {:x => 123}

That won't even compile. Ruby thinks {:x => 123} is a code block, and a bare key/value pair isn't valid syntax as the first thing in a code block. You need the parens. A bit unfortunate, but it gets worse...

def bar(arg1,arg2)
  puts "#{arg1} #{arg2}"
end

bar :x => 123, :y => 456          # Runtime error
bar {:x => 123}, {:y => 456}      # Won't compile

In the first example, all of the key/value pairs are slurped into one hash and end up in arg1. There's nothing left for arg2, so you get a "wrong number of arguments" exception. The second example won't even compile, of course, because again Ruby thinks {:x => 123} must be a code block with invalid syntax.

It gets worse if you change the argument list for bar slightly...

def bar2(arg1 = {}, arg2 = {})
  puts "#{arg1} #{arg2}"
end

This way, you don't even have to supply any arguments. This is nice, if all of your argument are optional.

bar2    # => {} {}

Now suppose you want arg1 to be {:x => 123}, and arg2 to be {:y => 456}. You might naively try this:

bar2 :x => 123, :y => 456         # => {:x=>123, :y=>456} {}

Oops, it all got dumped into arg1, and now instead of a missing argument error, arg2 silently ends up with a default, empty hash. You have to explicitly pass in an empty value for arg1 so that everything is slurped into arg2.

bar2 {}, :x => 123, :y => 456     # => WRONG!  Ruby thinks {} is a code block again.

bar2({}, :x => 123, :y => 456)    # => {} {:x=>123, :y=>456}

So much for syntax sugar. You might think you'd be unlikely to find this kind of thing in the wild, but in Ruby on Rails for example, there are quite a few functions whose argument lists look exactly like this. One signature for link_to is:

link_to(body, url_options = {}, html_options = {})

So...

link_to "foo", :controller => :x                          # OK
link_to "foo", :controller => :x, :class => "css_class"   # WRONG! 
link_to "foo", {:controller => :x}, :class => "css_class" # OK

What about support for default arguments? We might want to say that if you didn't pass in an :x argument, we want it to have some default value. You might think this would work:

def baz(x = {:x => 123})
   p x
end

But you would be sadly mistaken.

baz                               # => {:x=>123}
baz :y => 456                     # => {:y=>456} ... oops

Ruby doesn't merge your keyword arguments into the map in the parameter list. It uses that map if you don't supply any arguments, otherwise your map replaces the default entirely. So to get default arguments, you need something like

def baz2(args = {})
  args = {:x => 123}.merge(args)
  p args
end

baz2                              # => {:x=>123}
baz2 :x => 555                    # => {:x=>555}
baz2 :y => 456                    # => {:x=>123, :y=>456}

Kind of messy, but that's OK.

One last subtle ambiguity in Ruby is determining whether someone passed a nil argument for a keyword explicitly, or omitted a keyword entirely. It might make a difference in some circumstances.

def quux(args={})
  p args.include? :x
  args = {:x => nil}.merge(args)
  p args
end

quux                              # true, {:x=>nil}
quux :x => nil                    # false, {:x=>nil}

Pretty messy, but such is life.

Clojure

Part of the fun of Lisps is lack of ambiguity. Everything is spelled out in all its parenthesized glory. In Clojure, when you call a function like (foo :x 123), Clojure requires you to specify what you want to happen with those arguments.

It used to be that Clojure didn't have much support for keyword args at all. Clojure did have support for allowing variable numbers of arguments to functions though. So in the beginning, people used to slurp all of their arguments together into a list, and then turn it into a map inside the function.

user> (defn foo [& args]
        (let [args (apply hash-map args)]
          (prn args)))
#'user/foo
user> (foo)
{}
user> (foo :x 123)
{:x 123}
user> (foo :x 123 :y 456)
{:y 456, :x 123}

That worked. It still works today. But nowadays there's a better way. Why not slurp your arguments directly into a hash-map?

user> (defn foo [& {:as args}]
        (prn args))
#'user/foo
user> (foo)
nil
user> (foo :x 123)
{:x 123}
user> (foo :x 123 :y 456)
{:y 456, :x 123}

This is an example of destructuring. In this case, all of our arguments are slurped into a list (thanks to &), then this list is matched against our destructuring pattern, in this case {:as args}. :as says to take everything in the list, turn it into a map and give the resulting map the name args.

This isn't a special feature of defn. Destructuring works anywhere you're setting up bindings, for example in let, for, doseq etc. Like so:

user> (let [{:as args} (list :x 123 :y 456)] args)
{:y 456, :x 123}

It just so happens that & creates the list for you, out of the arguments you pass.

Destructuring does lots more than that though. We can immediately pull out the values for keywords we care about, so they'll be bound to names in our function body.

user> (defn foo [& {:keys [x y z]}]
        (prn x y z))
#'user/foo

user> (foo :z 123 :x 456)
456 nil 123

It's certainly a bit more verbose than Ruby in the function signature, but it lacks ambiguity and it saves you some repetition in the function body.

Clojure's approach also has the benefit of specifying directly in the function signature which keywords you're expecting. In a smart editor, like Emacs, you get an indicator of what kinds of keywords you should be passing in. See at the bottom?

This is also available at the Clojure REPL via the doc function.

user> (doc foo)
-------------------------
user/foo
([& {:keys [a b c]}])
  nil
nil

There's no better documentation than live, built-in documentation. There's nothing more distracting when programming than context shifts, and having to dig into a web browser to check a function signature is a huge mental page fault.

What about default values? Sure. You can use :or to specify defaults for some or all of your keywords. This works much more like I'd expect, compared to Ruby.

user> (defn foo [& {:keys [x y z] :or {x 1 y 2 z 3}}]
        (prn x y z))
#'user/foo

user> (foo :y 555)
1 555 3

What about determining whether the user passed nil for a keyword or whether they omitted the keyword entirely? For that, you have to resort to testing the argument map for the existence of the key, which isn't fun, but at least it's possible.

user> (defn foo [& {:keys [x y z] 
                    :or {x 1 y 2 z 3}
                    :as args}]
        (prn x y z)
        (prn args)
        (doseq [k [:x :y :z]]
          (println "Contains" k "=>" (contains? args k))))
#'user/foo

user> (foo :x 1 :y 2)
1 2 3
{:x 1, :y 2}
Contains :x => true
Contains :y => true
Contains :z => false

Why do I keep mentioning this? See Common Lisp below.

In any case, our parameter list is becoming huge and unweildy. The first time I saw sample Clojure code like this, I almost did a spit-take. But after a bit of getting used to, I'm finding that this syntax is pretty comfortable.

The simplest case may not be as concise as Ruby, but aside from lack of ambiguity, the benefit of Clojure's approach is being able to safely do powerful (and borderline insane) things, like nested destructuring. And this works everywhere you're setting up a binding.

(defn ow-my-eyes [a & {[b {:keys [x y]}
                        & {[g h] :r
                           :keys [p q]
                           :or {q 123}}]
                       :x}]
  (prn a b g h p q x y))



user> (ow-my-eyes 444 :x [1 {:x 555 :y 666} :p 3 :r [888 999]])
444 1 888 999 3 123 555 666

If you wrote code like that in real life, you'd likely be defenestrated, but at least you know you can do it. And destructuring in Clojure could likely be extended even further in the future, if someone came up with a use case for something that isn't supported.

Official documentation for all of this is here.

Common Lisp

When it comes to keyword arguments, Common Lisp supports mostly everything that Clojure does, and some things it doesn't. CL was likely a big inspiration for Clojure's destructuring. I highly recommend reading Practical Common Lisp to learn more about CL keyword arguments and list destructuring. (Read the rest of the book while you're at it.)

Keyword arguments in CL look like this:

> (defun foo (&key x (y 123) (z 456 z-supplied-p))
    (pprint (list x y z z-supplied-p)))
FOO

> (foo)
(NIL 123 456 NIL)

> (foo :z nil)             ; note, z-supplied-p tells us whether z was omitted or not
(NIL 123 NIL T)

> (foo :z 555 :x 666 :y 777)
(666 777 555 T)

> (foo :x 123)
(123 123 456 NIL)

> (foo :x 555)
(555 123 456 NIL)

Yeah, there's direct support for distinguishing supplied keys with nil values, and un-supplied keys. That's pretty nice.

Common Lisp also supports insane things like having keyword arguments' default values be a function of other arguments in the parameter list.

> (defun bar (&key (x 123) (y (+ x 1000)))
    (pprint (list x y)))
BAR

> (bar)
(123 1123)

> (bar :x 5)
(5 1005)

> (bar :x 5 :y 6)
(5 6)

Clojure destructuring can't do this by default, so you'd have to resort to a let in the function body.

(defn bar [& {:keys [x y] :or {x 123}}]
  (let [y (or y (+ x 1000))]
    (prn x y)

There's nothing stopping Clojure's destructuring from being patched to support this, of course. But this isn't a feature I've ever found myself wanting particularly badly.

As for how to destructure CL keyword arguments into a vector or hash-map, CL doesn't directly support doing it in function parameter lists like Clojure does. (Though there's nothing stopping someone from throwing together a reader macro to let CL do this, of course.) This isn't really surprising, because CL loves its cons cells, while Clojure embraces maps (and vectors and sets etc.), offering them default reader syntax and lots of other built-in support.

So there you have it. Keyword arguments. Use them. Love them.

Lessons:

Sugar can be bad for your health.
Ambiguous, bad. Explicit, good.

org-mode is awesome

Thu, 20 Jan 2011 16:14:44 -0800

I've seen org-mode for Emacs mentioned very frequently around the interwebs, so it went into my mental queue of topics to learn. It finally bubbled to the top this week, so I took a look.

Organizer? Nah.

As an organizer/calendar, well, I doubt I'll need it. Enforced use of MS Outlook is mandated by work. My Post-it-notes-all-over-my-desk method of organization will also continue to serve me well.

There are some nice agenda-related shortcuts that are probably worth using though, like typing C-c . to enter a datestamp, like <2011-01-20 Thu>. Then you can increment or decrement it one year/month/day at a time via S-up and S-down. I like this.

Plaintext editor? Yes!

As a plaintext outline and table editor... wow. org-mode rocks. Do you know how many hours of my life could have been saved by having a good ASCII table/bullet-list editor? org-mode lines everything up and keeps it all nice and neat for you.

You can also make plaintext check boxes and check/uncheck them. And you can insert hyperlinks and footnotes, and click them to open web pages or jump back and forth between footnote and reference.

There are ways to collapse and expand outlines, search for items and only display those items, and so on. The documentation for org-mode is very clear and took me less than an hour to read through. All-in-all a pleasant experience.

* Agenda
** Things to learn
1. [X] Clojure
2. [X] org-mode (see [fn:diagram1])
3. [ ] Haskell
4. [ ] Japanese
   1. [X] Hiragana
   2. [X] Katakana
   3. [ ] Kanji
5. [ ] The true meaning of friendship

* Footnotes
[fn:diagram1]

| Task                              | Annoyance (1-10) |
|-----------------------------------+------------------|
| Making ASCII tables by hand       |              9.5 |
| Making ASCII bullet lists by hand |              7.2 |
| Using org-mode                    |              0.4 |

It looks nice plastered into my blog, but you don't get a real idea of how many cool things you can do with it until you open it in Emacs and start shuffling items around, bumping them up/down a level in headlines, creating properly-numbered bullet items with one key, and seeing the columns in the table auto-resize as you type.

I also highly recommend putting (setq org-startup-indented t) into .emacs to make everything look pretty on-screen. It still saves as the simple plaintext above, but it looks like this in Emacs:

I can definitely see using org-mode for TODO files in some of my projects. (You can mark entries as TODO (just by typing TODO in front), and then toggle between TODO/DONE via C-c C-t.) I can also see using it as a general-purpose note-taker.

org-mode also has a mobile version for iPhone and Android, synced via WebDAV or Dropbox, so you can org-mode on your phone while you're driving to the grocery store¹. Again I don't really need this, but there it is.

The joy of plaintext

Plaintext is awesome.

It's the universal file format. It's readable and writeable by scripting langauges, terminals, text editors, IDEs, word processors, web browsers, even lowly humans.

Plaintext's one shortcoming is its lack of structure. It's just a bunch of letters. It doesn't have a color, it doesn't have a style, it doesn't line up into columns without a lot of effort. There's nothing stopping you from opening a parenthesized list and forgetting the closing paren.

Computers don't care about these problems, but humans are bad at producting plaintext by hand, and bad at editing it once it's produced. Our clumsy, stumpy fingers and inconsistent, chaotic brains can't handle the freedom.

Emacs (and Vim) are awesome because they let you do magical things to plaintext. They enforce structure. They provide shortcuts so you can get your plainext right the first time.

[ ] is just two braces and a space, but org-mode lets me hit C-c C-c and turn the space into an X. This may seem banal, hardly worth caring about, but add to this shortcut thousands upon thousands of others. Things like org-mode, or paredit, or all of Vim's built-in magic... it all adds up to something wonderful.

And best of all, you always still have the option of manually keyboarding over and typing that X between the braces yourself. It's still just plaintext underneath. So you end up with the best of both worlds.

I do not recommend using org-mode while driving, for public safety reasons. ↩

2010 in review

Wed, 05 Jan 2011 10:10:03 -0800

Another year down the drain. A good year, in the end.

2010 Geek Achievements

Wrote some code...
- cow-blog - The engine running this blog.
- oyako - Clojure ORM library.
- gaka - CSS compiler for Clojure
Finished a huge project for work, my first AJAX-y web app (in Rails). That was fun, albeit stressful.
Learned a lot of git.
Learned a lot of Clojure.
Learned a lot of Emacs.
Learned a lot of Javascript.
Learned a lot of PostgreSQL. It's good to be free of MySQL.
Switched to ZSH. This was a good switch.
Tried to learn a lot of Japanese, but kind of fizzled out at the end of the year.
Alllllllmost got a Clojure gold badge on Stack Overflow. I'll get it soon though. Not losing any sleep over it either way.
Read a lot of books. The best: probably Feynman's books of anecdotes.
Blogged a bit. Got an article in Hacker Monthly. Was flamed repeatedly. Learned a lot in the process.

2010 Non-Geek Achievements

Immigrated to Canada. A good move, without a doubt.
Lost 25ish lbs. :)
Learned how to cook better.
My most important achievement from 2010 is actually non-geek: I finally obtained a bit of an offline social life. This is not an easy task for one such as myself.
Continued to learn to appreciate good beer. Longwood Dunkelweizen, mmm.

2010 Failures

Did not blog enough.¹
Did not write enough code.¹
Missed the first Clojure Conj. Maybe next year.
Re-gained 10ish lbs. :( ²

Plans for 2011

Re-lose 25ish pounds. I'd like to reach the weight I had in college.
Finish my rewrite of oyako. I have ambitious plans for it, if I can just find the time.
Finish my rewrite of cow-blog to match oyako.
Keep working on the RPG my wife and I are creating (in Clojure).
Attend the next Clojure Conj, I hope.
Learn more Clojure.
Learn Haskell? Trying and failing to learn Haskell has become somewhat of a tradition, no sense stopping now.
Learn all 2000+ jouyou kanji by the end of the year.
Supar sekrit projekt. But I haven't signed the contract for it yet so I won't talk about it until I do.
Maintain social life at acceptable levels.
Buy a house.

I feel like I have solid plans for completing each of these things. Blogging more often and finishing oyako are high on my list of priorities. I expect 2011 to be my most productive year to date.

See also, non-geek achievement #4, "Obtained social life". ↩
See also non-geek achievement #3, "Learned how to cook better". ↩

Vim undo tree visualization

Mon, 18 Oct 2010 15:21:16 -0700

I wrote previously about an awsome plugin to give Emacs Vim-style undo trees.

Vim's undo trees are the best thing since sliced bread, but the interface for browsing through the tree is not pleasant. The Emacs undo-tree library has a way to visualize the tree and move through it with your keyboard, which solves this problem.

But now, thanks to Steve Losh, Vim has an undo-tree visualizer too. Delicious. Though it's still beta and promises to eat your babies, it seems to work pretty well. I think the diff view of the changes for the undo is a really good idea.

Thus continues the eternal Vim/Emacs arms race.

Productivity Booster

Fri, 08 Oct 2010 15:09:40 -0700

I came up with a great way to increase my productivity recently. You need a locally-running Apache server for this to work most effectively.

First you need to set up a redirect for 404 requests to localhost. On my system I determined that DocumentRoot is /srv/http, so I set this up in /etc/httpd/conf/httpd.conf:

<Directory "/srv/http">
    AllowOverride All
</Directory>

Next I edited /srv/http/.htaccess to redirect 404's to a main index page:

ErrorDocument 404 /index.html

Then I created /srv/http/index.html:

<html>
    <head>
        <title>GET BACK TO WORK, YOU HOBO</title>
        <style type="text/css">
            html {
                background: #f00;
            }
            h1 {
                color: #0f0;
                font-family: sans-serif;
                text-align: center;
                margin-top: 100px;
                font-size: 64pt;
            }
        </style>
    </head>
    <body>
        <h1>GET BACK TO WORK, YOU HOBO</h1>
    </body>
</html>

Almost done; the final step is to edit /etc/hosts:

127.0.0.1 reddit.com www.reddit.com slashdot.org news.ycombinator.com

Now I see this. Image hidden behind a link to spare the eyes of my readers.

If this doesn't work, you could try making the background color flash quickly between red and green, or add a background MIDI and some animated GIFs.

iPad? More like iAd. Vertisements.

Thu, 16 Sep 2010 18:43:35 -0700

Via Slashdot, it seems soon you may be able to subscribe to newspapers on the iPad in the near future.

Sure. Why pay $10 for a paper copy of something when you can pay the same $10 for a likely-DRM'ed copy that can only be read on a $500 portable computer?

In all honesty though, instant delivery, lack of clutter, "take it anywhere", being able to archive issues indefinitely, text search... those features might be worth the money, if it was a really good newspaper/magazine.

But wait, there's more.

The Cupertino company has agreed to provide an opt-in function for subscribers to allow Apple to share with publishers their information, which includes vital data that news organizations use to attract advertisers, industry sources say.

...

While the leap into the digital tablet market comes with short-term problems for newspapers, the iPad and future tablets will provide a new digital palette for publications to create sophisticated and lucrative ads, said Needham & Co. analyst Charles Wolf.

"I would say it's a risk, but I would argue it's a short-term risk," Wolf said. "If you can put animation and multimedia into ads, that will greatly enhance reader views. I am certain of that."

So as I understand it, first I buy a $500 gadget. Then I pay for a newspaper subscription. Then a bunch of companies want me to give them personal information about myself, so they can share it amongst themselves. And then I have to view ads.

Animated ads.

The only thing better would be if the iPad also woke you up at 4AM and tried to sell you life insurance. Maybe Apple would let me install a free ad-blocker script for my news reader though. It is my hardware, after all... pfffft, yeah, I could't keep going with a straight face.

And thus my desire to get an iPad, kind-of sort-of building over the past couple of months, once again flatlines.

Git info in your ZSH Prompt

Fri, 10 Sep 2010 13:31:37 -0700

Recently I discovered vcs_info recently. This nicely replaces the horrible hack I was using previously to show current Git status. vcs_info works with VCSes besides Git, and it handles some of the magic and keeps your .zshrc clean, so those are nice benefits.

I used some Unicode to display colored circles. Green if there are staged changes, yellow if there are unstaged changes, and red if there are new untracked-yet-unignored files. Below is a picture.

I like this because I'm constantly forgetting to git add newly-created files. Then I have to add them and amend my commit, and so on. I like a prompt that reminds me that new files showed up that need to be added or ignored.

Code:

autoload -Uz vcs_info

zstyle ':vcs_info:*' stagedstr '%F{28}●'
zstyle ':vcs_info:*' unstagedstr '%F{11}●'
zstyle ':vcs_info:*' check-for-changes true
zstyle ':vcs_info:(sv[nk]|bzr):*' branchformat '%b%F{1}:%F{11}%r'
zstyle ':vcs_info:*' enable git svn
precmd () {
    if [[ -z $(git ls-files --other --exclude-standard 2> /dev/null) ]] {
        zstyle ':vcs_info:*' formats ' [%F{green}%b%c%u%F{blue}]'
    } else {
        zstyle ':vcs_info:*' formats ' [%F{green}%b%c%u%F{red}●%F{blue}]'
    }

    vcs_info
}

setopt prompt_subst
PROMPT='%F{blue}%n@%m %c${vcs_info_msg_0_}%F{blue} %(?/%F{blue}/%F{red})%% %{$reset_color%}'

Picture:

Limitations

As you can see in the screenshot, when you have a brand new Git repo (no commits yet), vcs_info fails to show you that there are files staged. It works OK after you have at least one commit though.

vcs_info doesn't (yet?) handle showing untracked files. So I hacked a function to support it.

Finding a good Unicode symbol that displays nicely in monospace font was annoying. If I ever change fonts, I'll likely have to pick a new symbol. It also doesn't display too well in a real tty. Or over SSH when using Putty. So I may have to scrap the stoplights and use plus-signs or something. Sigh.

Vim :ruby and :rubydo scope

Tue, 31 Aug 2010 10:40:51 -0700

Note to self. In old Vim (tested in 7.2.320), I could do this:

:ruby x='foo'
:rubydo $_=x

Now every line in the file says foo. But in Vim 7.3 I get an error:

NameError: undefined local variable or method `x' for main:Object

The scoping rules for Ruby in Vim must have changed somewhere along the line. I was abusing this feature to do some handy things, so this is sad.

A workaround is to use global variables in Ruby instead. So this still works:

:ruby $x='foo'
:rubydo $_=$x

Phew.

Emacs undo trees

Tue, 17 Aug 2010 14:06:33 -0700

I've said it before: undo in Emacs is horrible. On the other hand, undo in Vim is awesome.

But this is true no longer. Now there are undo trees for Emacs! Yes, this news is so important I had to italicize and bold it. It's like Emacs has been punching me in the face for years, and today I got it to stop. I never thought I'd see the day.

And it works great too. You can even view the tree visually and navigate it with the cursor keys, which is a step up on what Vim offers out of the box.

In other news, Vim 7.3 is out and it now has persistent undo across reloads. It's like an arms race, and gleeful hackers reap the benefits.

Let's draw some pixels

Tue, 10 Aug 2010 11:19:42 -0700

I've been getting into pixel art a lot lately. It appeals to me on a lot of levels.

The coder in me likes it because it's so precise. Every pixel is placed just so. The color palette is limited to a dozen colors. Building a drawing out of such limited means reminds me of building programs out of primitives. There are design patterns in pixel art: dithering, manual anti-aliasing. There are abstractions that work and abstractions that don't. There's a lot of goofing around with RGB values and transparency settings; it's perhaps the most deeply computer-based art form you could come up with, and as a deeply computer-based human, I really like it.

The gamer in me is still partly stuck in the early 90's, so it's a huge injection of nostalgia to look at pixel art. NES- and SNES-era games had a charm that is unmatched by anything since. And I don't think that's entirely nostalgia talking; I still play old games and they're still so much fun. And the art in a lot of those games was just darned good. If you stop and look at it really carefully, and start to get an understanding of how it was made, you can't help but be impressed.

The "artist" in me (if there is such a thing in my brain somewhere) is blown away by some of the things good pixel artists can produce. Go look at foolstown.com and try not to slobber. Some of this stuff just looks amazing. Not "good for a pixel drawing", but good on a level anyone could appreciate.

Pixel doodles are also good practice for the RPG my wife and I are still ever-so-slowly creating. Creating art and music for a game are turning out to be much harder work than programming it.

In any case, I drew a cow standing beside a tree. And I made a new pixel art page to house my admittedly still-amateurish drawings.

Review: What Do You Care What Other People Think?

Fri, 06 Aug 2010 22:12:01 -0700

I recently reviewed Surely You're Joking, Mr. Feynman!. It was good enough that I had to get the sequel.

What Do You Care What Other People Think? is another collection of stories and anecdotes written by and/or about Richard Feynman. A bit in contrast to the first book, rather than a chronological series of anecdotes, this book focuses on a couple of main topics.

Feynman discusses his first wife in some detail. Of particular interest, he describes his and his wife's brutal devotion to honesty in their relationship, even in the face of highly unpleasant truths (terminal disease, in this case). It's the honesty of a scientist, carried into "everyday" life. This was bittersweet for me to read, because the story has a sad ending.

There is also a short series of letters from Feynman to others, where he discusses the silliness of pomp and circumstance, e.g. his foibles and breaches of protocol when meeting some king or other. As someone who hates ceremony, I got a huge kick out of these.

A large part of the book is devoted to discussing the Presidential Commission which investigated the cause of the Challenger shuttle disaster. Feynman's full report is included in the book as well.

As someone interested in astronomy and space flight (and who isn't interested in those?) I found this fascinating. There's a lot of behind-the-scenes stuff. Engineers are painted in a good light, managers and politicians not so much. (Software engineers come out looking especially good, which made me feel (unjustifiably) good about myself by proxy.) There are some diagrams and a lot of technical discussion of the shuttle. Not so much that it drowns the narrative, but enough that I'm probably going to spend the next week reading Wikipedia on the subject now.

Feynman explains his simple methods at getting to the truth in the investigation. Go talk to the guys who put things together. Get your hands on some O-ring rubber and test its resistance to temperature yourself in a glass of ice water. Cut to the heart of the matter. It's good stuff.

Ultimately, as you know if you've read the report, Feynman rips NASA apart, showing that they were fooling themselves into believing the shuttle was safer than it really was. The last sentence of the report says everything: "Nature cannot be fooled."

The last section of the book discusses the value of science. More specifically, Feynman discusses the value of doubt. I very much liked how the chapter ends:

It is our responsibility as scientists , knowing the great progress which comes from a satisfactory philosophy of ignorance, the great progress which is the fruit of freedom of thought, to proclaim the value of this freedom; to teach how doubt is not to be feared but welcomed and discussed; and to demand this freedom as our duty to all coming generations.

If there's one trait I had to pick to separate good people from bad, it would be the ability to admit being wrong. And if I had to separate the good from the excellent, it would be not just the ability to admit being wrong, but the eagerness to be proved wrong.

There's a certain kind of devotion to the truth that not many people achieve, and maybe not many people even want to achieve. There's comfort in thinking that you know things. It's very tempting. I think it's probably partly why most people are religious. I suspect it's a big reason why so many people are so stubbornly wrong about so many things in general. I suspect this comfort is an enormous source of suffering in the world.

But there's another kind of comfort that people miss out on. It's the comfort of knowing that although you're probably wrong about a lot of things, you're trying your hardest to be right. You pay the price of being aware of your own state of ignorance, but you can rest a bit easier knowing that you're maybe, hopefully, inching towards the truth. I never heard the word "freedom" used to describe this feeling before, as Feynman does above, but it fits.

That's why I like reading about Feynman and reading Feynman's words. He seemed to live this philosophy as well as anyone could hope to.

X automation with xte

Fri, 06 Aug 2010 13:03:01 -0700

I learned today (via a great blog post) about xte. This program lets you simulate X Windows mouse and keyboard events from the commandline. How much more awesome can you get?

Hans illustrates how to integrate xbindkeys and xte to make KDE4 effects activate. I wanted the KDE4 "Desktop Grid" to appear when I press a mouse button (because my new mouse has a lot of buttons to spare), so this is exactly what I was looking for.

xte is the kind of glue that makes Linux awesome. KDE lets you set global keyboard shortcuts for lots of things. xbindkeys lets you assign shell commands to mouse buttons. And xte ties the two together. Possibly none of the programmers on these three tools knew about the others, but they interact perfectly to let you do anything you want.

You may be thinking, "If you want to work with KDE from the commandline, why not use DBUS?" That's what I tried to do first. But I can't for the life of me figure it out. There's some indication that we might be able to do this somday, like so:

qdbus org.kde.kglobalaccel /component/kwin org.kde.kglobalaccel.Component.invokeShortcut ShowDesktopGrid

Or maybe it's already in the latest version of KDE and I haven't upgraded yet. Either way.

By the way: could DBUS possibly have a more verbose or cryptic interface? I was hunting through the available DBUS commands looking for something that would show the Desktop Grid, and I ended up having to scan through lists of crap like this:

~ % qdbus org.kde.kwin /KWin                         
method Q_NOREPLY void org.kde.KWin.cascadeDesktop()
method void org.kde.KWin.circulateDesktopApplications()
method bool org.kde.KWin.compositingActive()
signal void org.kde.KWin.compositingToggled(bool active)
method int org.kde.KWin.currentDesktop()
method QList<int> org.kde.KWin.decorationSupportedColors()
method void org.kde.KWin.doNotManage(QString name)
method Q_NOREPLY void org.kde.KWin.killWindow()
method QStringList org.kde.KWin.listOfEffects()
method void org.kde.KWin.loadEffect(QString name)
method QStringList org.kde.KWin.loadedEffects()
method void org.kde.KWin.nextDesktop()
method void org.kde.KWin.previousDesktop()
method Q_NOREPLY void org.kde.KWin.reconfigure()
method void org.kde.KWin.reconfigureEffect(QString name)
method void org.kde.KWin.refresh()
signal void org.kde.KWin.reinitCompositing()
signal void org.kde.KWin.reloadConfig()
method bool org.kde.KWin.setCurrentDesktop(int desktop)
method void org.kde.KWin.showWindowMenuAt(qlonglong winId, int x, int y)
method Q_NOREPLY void org.kde.KWin.toggleCompositing()
method void org.kde.KWin.toggleEffect(QString name)
method Q_NOREPLY void org.kde.KWin.unclutterDesktop()
method void org.kde.KWin.unloadEffect(QString name)
method bool org.kde.KWin.waitForCompositingSetup()
method QDBusVariant org.freedesktop.DBus.Properties.Get(QString interface_name, QString property_name)
method QVariantMap org.freedesktop.DBus.Properties.GetAll(QString interface_name)
method void org.freedesktop.DBus.Properties.Set(QString interface_name, QString property_name, QDBusVariant value)
method QString org.freedesktop.DBus.Introspectable.Introspect()

This is line noise to me.

gaka 0.2.0

Thu, 29 Jul 2010 13:59:22 -0700

Per many commenters' suggestions and thanks to code from Steve Purcell, you can now use maps for CSS attributes in gaka.

user> (println (gaka/css [:a {:color :red}]))
a {
  color: red;}

This looks more like vanilla CSS thanks to the curlies, which is nice. You just have to keep in mind that your key/value pairs could end up being printed in random order, and order is significant¹ in CSS.

It just so happens that maps are implemented in Clojure right now such that if they only have a few entries (16 key/value pairs), the order will be preserved, because you get a PersistentArrayMap instead of a PersistentHashMap. But it's highly dangerous to rely on such a thing. It could change at any time in the future.

In any case, you can also mix and match maps, lists and "flat" keyvals. They'll all be flattened That can help preserve attribute order in those cases where you need to.

user> (println (gaka/css [:a :color "red" {:padding 0} (list :margin 0)]))
a {
  color: red;
  padding: 0;
  margin: 0;}

I've also enhanced "mixins" a bit further. You can now mixin entire tags as well as attributes. Or a combination of both. Say you want a mixin that means "Make my element have no padding, and make links within the element be red":

user> (println (gaka/css [:div.foo mixin :margin 0]
                         [:div.bar mixin]))
div.foo {
  padding: 0;
  margin: 0;}

  div.foo a {
    color: red;}

div.bar {
  padding: 0;}

  div.bar a {
    color: red;}

You can get gaka from github or Clojars.

Order is only significant in cases where you're doing things like padding: 0; padding-left: 1px. This is arguably bad CSS style, but it's valid, and it's also possible you'll have this kind of thing if you're generating CSS procedurally. But most of the time, order is not significant. e.g. it doesn't matter if you set text color first and background color second, or vice versa. So maybe this isn't so much of a problem in practice. ↩

Footnotes

Tue, 27 Jul 2010 12:30:57 -0700

Did you ever notice how footnotes make your writing seem more important¹ somehow?

Maybe one reason is that "real" books use footnotes. At a glance, it looks like I have references² backing up everything I say. In reality, I don't, but the connotation carries through somehow³. Now my blog seems scholarly and authoritative.

And if you're like me, you can't resist clicking footnotes to see what they refer to⁴. According to my estimates, by utilizing footnotes, in one fell swoop I have decreased my readers' average reading efficiency by 73%.

In any case, I've added experimental, rudimentary support for footnotes to cow-blog.

I'm loosely copying the syntax from Markdown Extra for this. Markdown is great, except when it isn't. The standard doesn't have support for some useful extensions. I use Showdown for Markdown support, and I'm probably going to work on adding more features of Markdown Extra to Showdown in the near future.

I just dread actually doing it. Showdown (like Markdown itself) is implemented as a series of hackish regex transformations of blobs of text. It's not a proper grammar. Implementing more of Markdown Extra means more regex blobbing. It's brittle and fragile and even getting incomplete support for footnotes was less than enjoyable. But at the same time I find myself wanting to do things that Markdown can't so, so I may have to bite the bullet.

(If there's a Showdown Extra out there already, drop me a URL. It'd be most appreciated. But I couldn't find one.)

In reality nothing I say is important. ↩
Does my inner dialog count as a reference? ↩
Via telepathy. ↩
See? ↩

Review: Surely You're Joking, Mr. Feynman!

Mon, 26 Jul 2010 16:19:21 -0700

I'm probably the last person on earth to read "Surely You're Joking, Mr. Feynman!", a collection of stories and anecdotes from the life of Richard Feynman. But better late than never.

I'll keep this short. Feynman was the kind of nerd every nerd wishes he was, in one way or another. He was socially awkward. He was blunt and tactless. He felt out of place much of the time. And it was surprising to see how often Feynman expressed feelings of inadequacy. He wrote about being highly intimidated when talking in front of big names in physics, and jealous of the math abilities of some other people.

But none of that stopped him from having an exciting life. In fact he turned these traits to his advantage. He had romantic success, often via very highly unconventional means (e.g. walking up to girls and asking them for sex outright; they often said yes). His bluntness was seen as an admirable trait: where others might be intimidated by a famous physicist, Feynman would give his honest opinion, and that was often appreciated.

And Feynman went far outside his comfort zone. He did a stint in biology, even though he knew nothing of biology at the time. He went to Brazil and joined a samba band. He sold drawings and paintings for a while. He played drums for a ballet.

I picked up at least three lessons from this book.

Try new things, even if you suck at them. Life is boring if you stick to what you're good at.
Be intellectually honest. Brutally so. It's the only way to do good science, and arguably the only way to live a good life. I've always believed this, and Feynman hammers the point home well.
Even the best and brightest of us feel insecure at times. You shouldn't let it stop you.

On top of all of that, reading of Feynman's time at Los Alamos working on The Bomb was a fascinating piece of history. I highly recommend this book.

And watch as much of Feynman on Youtube as you can find. It never ceases to be fascinating.

Review: Logitech Performance MX

Mon, 26 Jul 2010 13:51:27 -0700

I have a long list of companies I won't buy from, due to horrible customer service experiences or shoddy merchandise. On the other hand, my "Must buy from this company" list is awfully short.

Logitech is one company I'm generally OK buying from. (For now...) I go through computer mouses¹ pretty fast, so I end up buying a new one every 4-5 years. Logitech hasn't failed me thus far.

In my opinion, the most important parts of your computer are the ones you interact with: Keyboard, mouse, monitor. I'd rather have a slow computer with a good mouse and keyboard than a fast computer with cheap peripherals. I already have a wonderful keyboard, and I like to have a nice mouse to match.

My latest mouse is the Logitech Performance MX.

How much can you really improve upon the concept of a mouse? It's a round thing you slide around and click. Well, if you believe the marketing on the box, it seems there are at least three areas for improvement:

Add more buttons.
Make it work on more surfaces.
Gimmicks.

None of the above features matter much to me. What I care about:

Is it comfortable to use all day?
Is it going to last me 4 or 5 years?

Let's hit each of these points.

More buttons

The Performance MX has a lot of buttons. If you believe xev, there are somewhere between 10 and 13 (counting the scroll wheel etc.).

I don't need all of these. I like to have a thumb button for middle-clicking, but this mouse has four of them. What am I going to use all of the extras for? Maybe I'll have one open Emacs.

The browser forward and back thumb buttons work out-of-the-box in Linux, which surprised me. I'm used to having to dork around with xmodmap or xbindkeys. Linux has come a long way.

Then there's a thumb button that's supposed to open Expose on OS X or your app-switcher of choice in your OS. It's in a place that's impossible to press consistently with your thumb, unless you're a contortionist. And it does nothing in Linux by default. Similarly there's a vanishingly small "zoom" button near your thumb. I can't even find this button without looking at the mouse, because it's too small.

There's also horizontal scroll, if you can manage to tilt the wheel to either side. My fingers lack the dexterity to do this easily, and the button is awfully stiff, so I'll never use this.

But hey, better too many buttons than too few.

Surfaces

The marketers wrote "This mouse works on glass!" all over the box, so this must be a selling point of the mouse. But my Logitech G500 also worked on glass, once I upgraded the firmware (via Logitech's own website), so I'm not sure why this feature is being pushed so hard.

Gimmicks

On the Performance MX, you can click this little switch and disengage part of the mechanism that makes the scroll wheel "click". This lets you spin the wheel hard and it'll spin freely forever until you stop it manually, or until its inertia runs out.

Do I really need, and I quote, hyper-fast scrolling? The scroll speed of my mouse wheel has never struck me as a productivity bottleneck. Now that I've played with it, sure, it's mildly interesting. If you scroll at just the right speed, you can go all the way from the top of a Reddit thread to the bottom, while still skimming the text. But I don't need this feature.

But the levels to which marketroids will stoop to hype up a product never ceases to amaze me. The name itself is ridiculous. Performance MX. Who comes up with this?

Comfort

My previous mouse was a G500, and it was really quite big. A bit too big. (And there were these removable weights you could use to adjust the weight of the mouse (the "gimmick" feature of that mouse), which were worthless.)

The MX is bigger, but it has more curves and that helps my hand sit on it better. There's a very large thumb rest, which I like. If you have small hands, you might have problems with this mouse. I like the feel so far.

Will it last?

This mouse feels very solid. It has a nice heft to it. It has metal bits (or sturdy metal-like plastic) in places other mouses have cheap plastic and rubber.

The wheel on my G500 wore down very fast. I was some kind of rubbery plastic, which became rounded off, nicked and worn-down a lot over the years. The wheel on the MX feels like some kind of metal, with nice ridges on the top to provide friction. I'm a bit concerned that the wheel disengagement switch might wear down over time though. The more moving parts, the more opportunity for something to break.

The pads on the bottom are huge: a good 1 inch x 1/8 inch. I love this. Using this on my Icemat mouse pad is very comfy. It slides effortlessly, but leaves me with plenty of control.

Battery

I normally prefer corded mouses. This one is rechargeable. Most rechargeable mouses I've used had some kind of stupid dock you had to jam the mouse into. Not only does it take up desk space, it also wears holes into your mouse over time from being inserted and removed from the dock. The battery connectors also wear down fast, and then it doesn't charge right unless you fiddle with it.

The MX can be charged while you use it, via a USB cable, which is a really good idea. It comes with a short cable, and a long extender cable if you need it. The cable looks like it has a non-standard connector though, which is a bad idea. Why not use mini-USB?

The charger cable is awfully heavy though. I wouldn't want to use this thing while it's plugged in for more than a short time.

I don't know what the battery life is like, because it hasn't run out yet, after a couple days. That's good enough for me.

Price

$120. That's a lot. But taken over 4-5 years, it's not that bad.

And the surgery I'd have to have done on my hands after crippling them via a cheap mouse over the next 40 years would be way more than $120.

Should you buy this?

Sure. I like this mouse so far. It works, and it feels like it'll last a while. Props to Logitech for making good products.

Yes, mouses. ↩

Clojure syntax highlighting via SyntaxHighlighter

Wed, 21 Jul 2010 15:17:42 -0700

How do you syntax-highlight Clojure code for display on a website? The best way I can find is SyntaxHighlighter.

Daniel Gómez wrote a brush to give SyntaxHighlighter Clojure support. I tweaked it a bit myself and integrated it into cow-blog. I also converted my favorite color scheme to a SyntaxHighlighter theme. So when I write this code:

(defn- ip
  "Given a request, return the IP.  Looks for an x-forwarded-for
  header, falls back to :remote-addr on the request."
  [request]
  (or (get-in request [:headers "x-forwarded-for"])
      (request :remote-addr)))

You should see something like this:

...unless you're reading this via RSS, or in a browser without Javascript enabled, in which case you'll see plain, depressing black and white. But that's one nice thing about SyntaxHighlighter. It degrades nicely.

One bad thing about SyntaxHighlighter is that it doesn't play nicely with Markdown. Or rather, Markdown isn't powerful enough to let you specify the class of any markdown-generated HTML tags. If you want the <pre class="brush: clojure"> that SyntaxHighlighter requires, you have to write out the HTML by hand. But I hacked Showdown a bit to let me specify classes more easily, so I can avoid having to do that.

The code for all of this is on github with the rest of my blog.

ANTLR via Clojure

Fri, 16 Jul 2010 17:41:04 -0700

ANTLR is a parser-generator for Java. Can you use it from Clojure? Sure. Would you want to? Maybe.

Here's how to do it, start to finish.

For the impatient among you, all of the code below is on github.

git clone git://github.com/briancarper/clojure-antlr-example.git

Setup

I'm going to use leiningen for this project. Let's make a new project called antlr-example.

$ lein new antlr-example

Now edit project.clj to tell lein to fetch ANTLR (and Swank, since I use Emacs). ANTLR is available in Maven Central, so leiningen can grab it for us.

(defproject antlr-example "1.0.0-SNAPSHOT"
  :description "ANTLR Clojure example"
  :dependencies [[org.clojure/clojure "1.2.0-beta1"]
                 [org.clojure/clojure-contrib "1.2.0-beta1"]
                 [org.antlr/antlr "3.2"]]
  :dev-dependencies [[swank-clojure "1.3.0-SNAPSHOT"]])

Then a simple lein deps will download all of the jars and all of ANTLR's dependencies for you. Handy. You end up with this:

antlr-example
├── classes
├── lib
│   ├── antlr-2.7.7.jar
│   ├── antlr-3.2.jar
│   ├── antlr-runtime-3.2.jar
│   ├── clojure-1.2.0-beta1.jar
│   ├── clojure-contrib-1.2.0-beta1.jar
│   ├── stringtemplate-3.2.jar
│   └── swank-clojure-1.3.0-20100502.112537-1.jar
├── project.clj
├── README
├── src
│   └── antlr_example
│       └── core.clj
└── test
    └── antlr_example
        └── core_test.clj

Interactive development? Kind of...

One weakness of ANTLR via Clojure is that development is no longer REPL-based. The ANTLR workflow is essentially a Java workflow:

Write a grammar
Compile the grammar into Java source code
Compile the Java source into Java .class files

Until and unless someone writes a Clojure backend for ANTLR, so that ANTLR can directly produce Clojure source code (and it seems like it should be possible to do so), you're back to a write/compile/debug cycle. Joy!

This also means restarting your REPL every time you alter and recompile your grammar.

But the good thing is that there's ANTLR Works, a free GUI for writing ANTLR grammars. ANTLR Works has an interactive interpreter, which is nice, and it has a compiler/debugger, which is better. These let you test your grammar as you write it, by trying inputs and seeing the resulting AST in graphical form, which is as good as you could hope for. This is actually even a bit nicer than a plaintext REPL.

Plus, it has a nice editor for ANTLR code. Emacs was freaking out over the indentation on a few grammars I tried.

So a decent workflow might be to write and debug your grammar in ANTLR Works, then fire up Clojure afterwards to consume the parser.

A simple grammar

We're going to use a classic textbook example, simple arithmetic expressions, as a proof of concept.

For Java (hence Clojure) to find this code, it has to be named properly and it has to be put into the proper directory on CLASSPATH. I'm going to create the grammar file src/antlr_example/Expr.g. (My grammar file, Java source, and Clojure source will all be jumbled together in src. You can easily do it differently if this offends your sensibilities.)

The first line in the grammar names the grammar (and the classes we'll consume from Clojure):

grammar Expr;

Clojure being a s-exp based language, it might be nice to have ANTLR generate an AST. ANTLR has good support for this:

options {
    output=AST;
    ASTLabelType=CommonTree;
}

Helper tokens for the grammar:

tokens {
    PLUS = '+';
    MINUS = '-';
    MULT = '*';
    DIV = '/';
}

These next lines are important. We want to generate classes antlr_example.ExprLexer and antlr_example.ExprParser, so we have to set our code up in the proper package, antlr_example.

This code includes both a parser and lexer, so you have to set the package for both in ANTLR.

@header        {package antlr_example;}
@lexer::header {package antlr_example;}

And then the grammar itself, which is simple:

expr: term ((PLUS | MINUS)^ term)* ;
term: factor ((MULT | DIV)^ factor)* ;
factor: INT ;

INT :   '0'..'9'+ ;
WS  :   ( ' ' | '\t' | '\r' | '\n')+ {$channel=HIDDEN;} ;

One thing to note is (PLUS | MINUS)^; the ^ here tells ANTLR to treat (PLUS | MINUS) as the parent of the node created for this rule. This means the term on either side will be children of this node. We do the same for MULT and DIV. This will give us a nice tree. Otherwise you end up with a flat list of tokens.

{$channel=HIDDEN;} for the whitespace rule tells ANTLR to ignore whitespace, also useful.

If you use ANTLR Works with this grammar, on the input 1 + 2 * 3 + 4, you can see that it works OK.

That was an awful lot of boilerplate and setup and ugly syntax though. See how much Clojure spoils you? But for a simple grammar, it's not that much code.

For a longer or more complex grammar, you might end up having to embed inline Java code into your ANTLR grammar. But again, that's not the end of the world.

Compile everything

Once you write your grammar (presumably in ANTLR Works), you can generate the Java code and compile it thusly (run from your project's base directory):

$ java -cp 'lib/*' org.antlr.Tool src/antlr_example/Expr.g
$ javac -cp 'lib/*' -d classes src/antlr_example/Expr{Lexer,Parser}.java

The first command should generate src/antlr_example/ExprLexer.java and src/antlr_example/ExprParser.java.

The second command should spew tons of class files into classes/. Leiningen and other tools generally include classes/ on your CLASSPATH, so that's a good place for them. Again putting them here is a fairly arbitrary decision on my part. You can put the source and class files anywhere, as long as the class files end up on your CLASSPATH when you start Clojure.

If you're really so lazy that you can't run these two commands, you could use ANTLR Works' built-in commands for compiling your grammar. The only way I could find to invoke it was by starting a Debug session, which compiled my code as a side-effect. And it spews files into places I don't want them, so I like the command line version better.

You could probably also set up a leiningen task for this if you're re-running these commands a lot, or configure your build tool of choice to do the same. I didn't bother.

Now (re)start your REPL and let's write some Clojure.

Clojure code

It's pretty easy to consume an ANTLR parser from Clojure. I'm putting the code below into src/antlr_example/core.clj

First import the classes. You probably need some ANTLR helper classes to set up the parser; you also need the classes you just generated.

(ns antlr-example.core
  (:import (org.antlr.runtime ANTLRStringStream
                              CommonTokenStream)
           (antlr_example ExprLexer ExprParser)))

Here's a function to parse a string using our new parser class:

(defn parse-expr [s]
  (let [lexer (ExprLexer. (ANTLRStringStream. s))
        tokens (CommonTokenStream. lexer)
        parser (ExprParser. tokens)]
    (.expr parser)))

.expr is the name of the top-level rule we want to invoke from our grammar. The rest is boilerplate; just plug in the proper class names.

So let's see what we've got.

user> (in-ns 'antlr-example.core)
#<Namespace antlr-example.core>
antlr-example.core> (parse-expr "1 + 2 * 3 + 4")
#<expr_return antlr_example.ExprParser$expr_return@3755e508>

Er, OK. A good way to inspect Java objects is via bean, so let's view the guts of this object:

antlr-example.core> (bean *1)
{:tree #<CommonTree +>,
 :template nil,
 :stop #<CommonToken [@12,12:12='4',<8>,1:12]>,
 :start #<CommonToken [@0,0:0='1',<8>,1:0]>,
 :class antlr_example.ExprParser$expr_return}

I see. We set up our grammar above to generate an AST, so :tree on the bean will give us that. This translates to .getTree on the Java object. So we can edit our function to call this for us, since we always want to have the tree.

(defn parse-expr [s]
  (let [lexer (ExprLexer. (ANTLRStringStream. s))
        tokens (CommonTokenStream. lexer)
        parser (ExprParser. tokens)]
    (.getTree (.expr parser))))

Now:

antlr-example.core> (parse-expr "1 + 2 * 3 + 4")
#<CommonTree +>

OK, it appears we have one node of the tree, the root node. Let's peak inside:

antlr-example.core> (bean *1)
{:children #<ArrayList [+, 4]>,
 :childIndex -1,
 :parent nil,
 :text "+",
 :nil false,
 :token #<CommonToken [@10,10:10='+',<4>,1:10]>,
 :class org.antlr.runtime.tree.CommonTree,
 :ancestors nil,
 :tokenStartIndex 0,
 :type 4,
 :childCount 2,
 :charPositionInLine 10,
 :tokenStopIndex 12,
 :line 1}

Looks like our children are in :children on the bean, which translates to .getChildren on the Java object. And we have .getChildCount to count the children, and .getText to get the text of the current node. (We could've learned all of this by reading the javadocs for ANTLR too, but what fun is that?)

Since we're dealing with a tree, we can use tree-seq in Clojure to get a flat list of all the tokens in our text.

tree-seq takes three arguments. First a function that returns true if the node has children, false otherwise. Second a function that returns the children for a node that has children. Third the root node of our tree.

That'll give us a seq of tree nodes. So finally we call .getText on all the resulting nodes in the list, to turn node objects into strings.

Easy enough:

(defn node-seq [x]
  (map #(.getText %)
   (tree-seq #(not (zero? (.getChildCount %)))
             #(.getChildren %)
             x)))

antlr-example.core> (node-seq (parse-expr "1 + 2 * 3 + 4"))
("+" "+" "1" "*" "2" "3" "4")

But that's no good. We lost our tree structure.

We'd rather have something like nested vectors or lists. It's easy enough to roll something by hand. This should do it:

(defn AST [node]
  (if (zero? (.getChildCount node))
    (.getText node)
    (let [children (map AST (.getChildren node))
          txt (.getText node)]
      (if txt
        (cons txt children)
        children))))

antlr-example.core> (AST (parse-expr "1 + 2 * 3 + 4"))
("+" ("+" "1" ("*" "2" "3")) "4")

That's better, but it's a list of strings. These strings all happen to be literal representations of Clojure objects, so a call to read-string in the proper places should give us something we can work with:

(defn AST [node]
  (if (zero? (.getChildCount node))
    (read-string (.getText node))
    (let [children (map AST (.getChildren node))
          txt (read-string (.getText node))]
      (if txt
        (apply list txt children)
        children))))

antlr-example.core> (AST (parse-expr "1 + 2 * 3 + 4"))
(+ (+ 1 (* 2 3)) 4)

Hey look, now we have something we can evaluate.

antlr-example.core> (eval *1)
11

Yeah I kind of planned that ahead. If you had a more complex grammar, you might not get away with something quite this easy.

Conclusion

The advantage of ANTLR is that it's mature, widely used, actively developed, and well-documented. (There's a whole book about ANTLR, The Definitive ANTLR Reference by Terence Parr.) There's also a lot of tooling available for it, not just ANTLR Works, but plugins for other Java IDEs.

The disadvantage is that it's a Java library, and as always, there will be some friction when consuming it from Clojure. But it's not that bad.

For pure-Clojure parser generator alternatives, there are fnparse and clojure-pg, neither of which I've tried much.

Clementine: A triumph of Free Software

Tue, 13 Jul 2010 11:48:51 -0700

Ages ago, in the long-forgotten days of 2008, there was Amarok 1.4. And it was good. Then KDE4 came along and Amarok was rewritten, reshaped, becoming something... different. Something unsettling. Something not altogether pleasant.

Fear not. Today we have Clementine.

I consider Clementine a triumph of Free Software. A great project fell off the rails, so someone else picked up the pieces, forked it and kept the spirit alive.

Features present

Clementine embodies everything good about Amarok 1.4, in a shiny Qt4 package. The layout is eminently pleasant to use. It uses the classic "spreadsheet" playlist view that saw so much success in Amarok 1.4. If you care about cramming as much information about your music as possible onto the screen, this is as good as it gets. It's boring, and that's a good thing. It gets the job done.

Like Amaork, 1.4, in Clementine you can very quickly drill into your music collection, filter it, view recently added tracks, group songs by artist or album or year or genre or a combination of those things. Clementine also handles all of the edge cases correctly: it lists albums with Various Artists exactly how I'd want (exactly like Amarok 1.4). It correctly handles songs with non-Latin tag text.

Clementine detects additions and changes to my music collection instantly, without the massive scan-lags on startup that plague some other music players. Clementine doesn't bat an eye at my 7,000 song collection. There's no MySQL integration, but I don't need it. Clementine's SQLite backend supposedly handles 300k songs without much problem, which is good enough for me.

Clementine has Last.FM integration. It has three different styles of desktop notification. It has visualizations. It handles USB devices. It understands reply gain. It has cross-fading. It has an equalizer. It has a transcoder. It has a cover manager.

I'm tired of listing features. Let's just say it has every useful feature you'd ever want. And if you don't need a feature, it stays out of your way.

And for a program under such active development, it's rock solid. I have yet to see a crash. And speaking of active development, if you follow the activity in Clementine's SVN repo, you will find that this program is updated almost daily. How the devs find the time, I don't know, but I'm grateful. This program has gone from non-existent to awesome in record time.

Clementine can use gstreamer, so it even works cross-platform. I fired it up on Windows 7 the other day and I was amazed at how good it looked and felt. It supposedly also works on OS X.

Clementine doesn't cook your breakfast for you, but that might be in the works.

How to make a good UI

A perfect example of the polish of Clementine's UI: Tagging. How do you tag a whole album worth of music at once? You can select some songs and right click and go into a dialog, like most music players allow.

Or:

Edit a tag for a single song (inline) by clicking the field. Let's say you edit Artist.
Select multiple songs in your playlist. (Click and drag, CTRL-click, Shift-click, CTRL-A, whatever.)
Right click the Artist tag in the song you edited, select Set Artist to "XXXXX", and now all the songs you selected will be updated.

This is the kind of UI innovation that I like. It's simple, it's useful, and it's predictable. You can get things done without going through dialog windows, without a million clicks, without spending a minute scratching your head figuring things out.

(Meanwhile Amarok 2 is busy getting rid of the Stop button and making the volume control circular.)

Features missing

Admittedly, Clementine is missing a couple of features I wouldn't mind having. You can't skin or theme Clementine. You can't rate songs. You can't display song lyrics. You can't "queue" songs. But oh well. I can live without these features because the rest of the program is so darned good. For all I know, these features might pop up next week. I wouldn't be surprised.

The Clementine devs seem to be very friendly and responsive to feature requests and feedback, which is also great.

Clementine is also missing a few features/bloat that I'm glad to see NOT ported from Amarok. Wikipedia integration? Good riddance.

I would pay money for this program.

In November 2009 I had this to say:

(Anyone out there reading this, if you port Amarok 1.4 to Qt4 intact, I will pay you. Seriously. I will pay you money.)

The offer still stands. I will pay money for Clementine. I'm still waiting for a Donate link so I can do so. (Clementine devs, are you reading this?)

Why do I care about this so much? Because I have music playing whenever I'm using this computer, and when you add up work plus free time, I'm at this computer 8-10 hours per day. Music keeps me sane during multi-hour debug sessions. Music is an integral part of my life, and a music app is an integral part of playing music.

It's very important to me that the programs and tools I use all day are comfortable. Otherwise I become cranky. If you were a carpenter, would you want to use a hammer with a wobbly handle all day? I'm a programmer, and I want to use comfortable computer programs.

Clementine is very comfortable.