Reddit and StackOverflow and other websites I frequent are filled to the brim with discussion of Google's Go. The code snippet on the front page is:

package main

import "fmt"

func main() {
  fmt.Printf("Hello, 世界\n")
}

First thoughts that ran through my head as I looked over the site:

• Ugh, look at all that syntax.
• Nice(r) type system (than C++ and Java). I'll stick with multimethods though.
• Concurrency semantics, hmmm... Shared mutable memory between threads? I think I'll stick with Clojure for now thanks.
• Where are the macros?
• It has anonymous functions and closures and sort-of first-class functions? Good. Welcome to the 1960's.
• len is a special operator? Sigh. (Programming language quality is usually inversely proportional to the number of special forms.)
• Cool that they used Japanese in the example though. (That word is sekai, "world", obviously.)

Compared to a Lisp, this language looks indistinguishable to C, Perl, Python, Java etc. It looks like such a small incremental improvement (if it even is an improvement). Yet another imperative, for-loop-wielding, curly-brace-using, pointer-mangling, state-mutating, OOP language.

In fact via Reddit today I read this awesome post to a mailing list which compares Go with ALGOL68, and it gave me a would-you-look-at-that moment. Once you learn a few languages that are significantly different from ALGOL derivatives, all ALGOLish languages start to look eerily similar. Are we really stuck with ALGOL-derived languages being the only viable mainstream languages for all time? How much polish can we possibly apply to the same turd?

Then I realized, I'm turning into a Lisp snob. : ( Learning a Lisp apparently does spoil you for the rest of time. I am without a basis to judge whether this language will be a successful replacement of anything. All I know is I probably won't use it. Honestly I'm much more excited about new things on the horizon in Clojure. And I still have getting better at Haskell on my TODO list.

In yet another step along the path of trying to forcibly morph Emacs into Vim, I started a port of my Vim color scheme Gentooish to an Emacs color-theme.

Then I threw a few lines at the bottom of .emacs to highlight a few more Clojure-specific things when in clojure-mode. Result:

This is inspired partly by parenface.el which dims parens so they don't stand out as much. Lispers have this silly meme where they pretend they can't see parens at all, but being a mere human I want Emacs to help blend them into the background a bit.

I want to dim parens but also make braces and brackets pop out a bit more. Coloring braces, brackets and parens slightly differently helps make a few things easier to read in my opinion, especially in ))]}])) kinds of situations (like at the end of the function above).

This is pretty brute-force and I really wish I could figure out how to highlight #{} (sets) differently than {} (hash-maps). Highlighting (), '(), `(), and #() differently would be awesome too. But that is well beyond my elisp skills at the present. Certain areas of Emacs are all but impenetrable, no matter how much documentation I read and how much banging on the keyboard I do. The morass that is font-lock is one of those areas.

p.s. I find an easy way to keep your dotfiles in git is to make a folder (called dotfiles or something) and symlink your dotfiles from there to your home directory. I also put * in .gitignore so git doesn't slurp up anything sensitive by accident.

Note to self. Given this:

user> (def x [{:foo 2 :bar 11}
              {:bar 99 :foo 1}
              {:bar 55 :foo 2}
              {:foo 1 :bar 77}])

#'user/x

How do you sort by :foo, and where :foo is equal, sort by :bar?

Obviously not like this:

user> (sort x)
; Evaluation aborted.
;; java.lang.ClassCastException: clojure.lang.PersistentArrayMap cannot be cast to java.lang.Comparable

Also not like this:

user> (sort-by #(map % [:foo :bar]) x)
; Evaluation aborted.
;; java.lang.RuntimeException: java.lang.ClassCastException: clojure.lang.LazySeq cannot be cast to java.lang.Comparable

What's a guy got to do to find something Comparable? Well, vectors are Comparable, so we can do this:

user> (sort-by #(vec (map % [:foo :bar])) x)
({:foo 1, :bar 77} {:bar 99, :foo 1} {:foo 2, :bar 11} {:bar 55, :foo 2})

I'm converting one of my websites from Ruby on Rails to Clojure in my spare time. I stupidly put a bunch of RoR-style links inline into certain bits of plaintext content, so in my DB there are a bunch of text fields with <%= link_to ... %> in the middle.

It was easy to fix with a regex though:

(defn clean [txt]
  (re-gsub #"<%=\s*link_to\s+(\"[^\"]+\"|'[^']+')\s*(?:,\s*'([^']+)'\s*)?(?:,\s*image_path\(['\"]([^'\"]+)['\"]\)\s*)?(?:,\s*:controller\s*=>\s*(?::(\S+)|['\"]([^\"']+)['\"])\s*)?(?:,\s*:action\s*=>\s*(?::(\S+)|['\"]([^\"']+)['\"])\s*)?(?:,\s*:id\s*=>\s*(?:(\d+)|:(\S+)|['\"]([^\"']+)['\"])\s*)?\s*%>"
           (fn [[_ s & parts]] (let [href (str-join "/" (filter identity parts))]
                           (str "<a href=\"/" href "\">" (re-gsub #"^[\"']|[\"']$" "" s) "</a>")))
           txt))

And by easy, I mean not easy.

Note to self, try something other than a regex next time.

Note to self, don't bury some framework's funky-syntax DSL in the middle of plaintext content. Next time use HTML or do the conversion from DSL to HTML early rather than late.

Silly how two years ago I thought I'd be using Ruby for that site forever.

Church Numerals are a way of encoding natural numbers using lambda expressions. A number n is represented by a function that composes another function f around its argument x, n times. So:

0 ≡ λf.λx. x
1 ≡ λf.λx. f x
2 ≡ λf.λx. f (f x)
...

These can be implemented very easily in Lisps and other languages with first-class functions. In Clojure, this would be:

0 ≡ (fn [f] (fn [x] x))
1 ≡ (fn [f] (fn [x] (f x)))
2 ≡ (fn [f] (fn [x] (f (f x))))
...

The insane thing is that you can do arithmetic on these things using nothing but more anonymous functions. It's lambdas all the way down.

Once you understand this, your brain will explode, and hopefully little bits of it will land next to the remnants of my own brain to keep it company.

So based on the Wikipedia article, here are some functions that convert numbers to and from Church encoding, and then some functions to do basic arithmetic on Church numerals. I understand this down to exp (via lots of pencil and paper expansions) but I gave up understanding at pred.

(defn church
  "Convert a natural number to a Church numeral."
  [n]
  (loop [n n, acc (fn [f] (fn [x] x))]
    (if (zero? n)
      acc
      (recur (dec n)
             (fn [f]
               (fn [x]
                 (f ((acc f) x))))))))

(defn unchurch
  "Convert a Church numeral to a plain integer."
  [n]
  ((n inc) 0))


(def plus
     (fn [m]
       (fn [n]
         (fn [f]
           (fn [x]
             ((m f) ((n f) x)))))))

(def mult 
     (fn [m]
       (fn [n]
         (fn [f]
          (n (m f))))))

(def exp
     (fn [m]
       (fn [n]
         (n m))))

(defn pred [n]
  (fn [f]
    (fn [x]
      (((n (fn [g]
             (fn [h]
               (h (g f)))))
        (fn [u] x))
       (fn [u] u)))))

(def sub
     (fn [m]
       (fn [n]
         ((n pred) m))))

(def is-zero?
     (fn [n]
       ((n (fn [x] 'FALSE)) 'TRUE)))

And this somehow actually works:

user> (unchurch (fn [f] (fn [x] (f (f (f (f (f (f (f x))))))))))
7
user> (unchurch (church 7))
7
user> (unchurch ((plus (church 2)) (church 3)))
5
user> (unchurch ((mult (church 2)) (church 3)))
6
user> (unchurch ((exp (church 2)) (church 3)))
8
user> (unchurch ((sub (church 7)) (church 3)))
4
user> (is-zero? (church 0))
TRUE
user> (is-zero? (church 7))
FALSE

This is the kind of thing I wish I'd studied more in college. I covered some cool stuff, some small amount of theory of computation and whatnot, but not nearly enough of this. Not enough functional programming, not enough Lispy goodness.

A lot of people say all programmers should learn assembler language so you understand what's happening at a low level. But shouldn't it be just as important to learn lambda calculus? It's another kind of low level, a very important one at that.

So my newest site is about pareidolia which I think is an interesting subject. It's still in its early stages, using a data import from yet another of my old sites that I ran with a friend of mine. This makes three websites I'm running out of the same JVM on my server now. It'll be interesting to see how many more I can cram in there before my VPS explodes. CPU and RAM utilization are still supposedly minimal. I actually plan to redo my old faltering FF1 site entirely in Clojure too, which is probably the largest Clojure project I plan to undertake in the foreseeable future.

I'm still trying to come up with some kind of "how to make a blog in Clojure" tutorial once I nail down a good way to do it. The code for my blog has "grown organically" (i.e. become a mess) beyond the point where I'd inflict it upon the world without substantial cleanup. Probably easier to start over from scratch with something that's more demonstrative.

I'm also not too savvy on many of the "right ways" to do things in Compojure, as per this discussion. That's the good and bad thing about using a Lisp. You can pull off almost anything. There is no underlying structure to the code other than structure you impose. It's easy to write a mess.

I'm making a new website (in Clojure of course) and I have a need to resize uploaded images to make thumbnails. At first I tried to use JMagick because I'm familiar with ImageMagick already and it seemed like an OK library. But on the crusty old OS my VPS uses, I had a really hard time getting it to build, and even once it built it started segfaulting like mad.

Should've gone with something simpler first. Java has built-in libraries for this. It only took a few seconds to adapt this code on Stack Overflow into Clojure code.

(use 'clojure.contrib.java-utils)
(defn make-thumbnail [filename new-filename width]
  (let [img (javax.imageio.ImageIO/read (as-file filename))
        imgtype (java.awt.image.BufferedImage/TYPE_INT_ARGB)
        width (min (.getWidth img) width)
        height (* (/ width (.getWidth img)) (.getHeight img))
        simg (java.awt.image.BufferedImage. width height imgtype)
        g (.createGraphics simg)]
    (.drawImage g img 0 0 width height nil)
    (.dispose g)
    (javax.imageio.ImageIO/write simg "png" (as-file new-filename))))

I posted recently about being dissatisfied with how my blog was playing with MySQL. I was going to try for some kind of file-based storage, but in the end I decided to go with Tokyo Cabinet, which is a very lightweight key/value store.

This simplifies a lot of code, because a blog is pretty much one list of objects (posts) with a bunch of sub-objects (tags, categories, comments). An SQL DB is made for independent objects that are related to each other via keys. So storing a post means deconstructing it into its parts and stuffing all the parts into their own tables, and fetching a post means fetching all the parts and putting it back together. A bit of a bother.

With a key/value store, a post is a hashmap, and it has sub-lists of comments and tags etc., and you serialize the whole thing and stuff it into the DB; when fetching you un-serialize it and you're good. Clojure being a Lisp, there's already a nice serialization format (s-expressions), so there's hardly any work to be done.

I'll give it a few days and if everything actually works, I'll push the code to github. The code is still pretty nasty and ad-hoc; I did this rewrite in the matter of a couple of hours on a weekend. But parts of it may be useful to someone.

I also plan to do some kind of simplified "How to make a blog in Clojure" tutorial in the near future. The code for this blog is bloated with a lot of functionality that is pretty specific to my site and that most people don't need. A more to-the-point tutorial would probably be more helpful.

I had a lot of trouble connecting to an MS SQL Server at work via Clojure. Java 6 comes with a JDBC-ODBC bridge which worked fine from a Clojure REPL at a command prompt, or from inferior-lisp in Emacs, but in SLIME it would hang every time I tried to connect and I'd have to kill Java. Couldn't for the life of me figure out why.

I got it to work eventually by using Microsoft's own JDBC driver, which you can download here.

Once you put the downloaded .jar file on your CLASSPATH (in my case, sqljdbc4.jar) you can connect like this:

user> (def db {:classname "com.microsoft.sqlserver.jdbc.SQLServerDriver"
               :subprotocol "sqlserver"
               :subname "//server_hostname;database=SomeDatabase;user=SomeUser;password=SomePassword"})
#'user/db
user> (use 'clojure.contrib.sql)
nil
user> (with-connection db 
        (with-query-results rs ["SELECT * FROM whatever"] (prn rs)))
... results ...

Posted for the sake of Googlebot and for my own future sanity.

This infamous blog post suggests that someone familiar with a language should be able to name five things they hate about it. "Hate" is a strong word, but I decided to think of five things I find mildyly annoying about Clojure, my favorite language of the moment.

Hashing integers

Clojure automatically converts integers between Integer, Long and BigInteger as needed to prevent overflow. This is good. Integers of the various classes test as equal too. This is also good.

user> (= 123 (int 123) (long 123) (bigint 123))
true

So would you expect this?

user> (hash-map (int 123) :foo (long 123) :bar (bigint 123) :baz)
{123 :foo, 123 :bar, 123 :baz}

Yes, each of the integer classes, though equal via =, do not have the same hash value when put into a hash-map. This is because:

user> (.equals (int 123) (long 123))
false

This is a wart inherited from the JVM. See here for discussion and explanation.

What's more, if you print this map and then try to read it back in, the integers will be read as int, long or bigint arbitrarily depending how big they are. This means you may not get the same class of object back that you output originally.

user> (def x {(bigint 123) :foo})
#'user/x
user> (= x x)
true
user> (def y (read-string (pr-str x)))
#'user/y
user> (= x y)
false
user> (class (first (keys y)))
java.lang.Integer
user> (class (first (keys x)))
java.math.BigInteger

This means that if you ever use integers as hash keys, you must be very careful to cast them all to the same integer type manually.

Metadata doesn't work on everything

Clojure lets you stick arbitrary metadata on various objects. This is higly useful; you can decorate objects with information that doesn't affect the value of the object. However metadata doesn't work everywhere.

user> (with-meta "foo" {:bar :baz})
java.lang.ClassCastException: java.lang.String cannot be cast to clojure.lang.IObj (NO_SOURCE_FILE:0)

You can only stick metadata on certain Clojure objects like Symbols, Vars, Refs, Agents, all of the Clojure collections and so on. You can't stick metadata on, say, a String or an Integer, because those are closed Java classes and can't be touched. It would be nice if you could.

use vs. require vs. import vs. load vs. ...

There are a startling number of ways to import a library into your code in Clojure. You have to choose from load, import, require, use, refer, and so on. Some work on Java classes, some work on Clojure libs. Some of them import symbols into your namespace, some of them don't. Some of them take strings as arguments, some take symbols, some take quoted lists of symbols, some take quoted lists of symbols with sub-lists of arguments. And all of these can be and usually are weirdly inlined into a namespace declaration, with a completely different list-quoting style.

So in Ruby you can do this:

require 'util'
require 'config'
require 'whatever'

Whether it's a gem, or a Ruby source file sitting locally, it all works the same as long as the load path is set up right.

But in Clojure, you do this (actual code from an IMAP library I wrote):

(ns qt4-mailtray.mail
  (:import (java.util Properties)
           (javax.mail Session Store Folder Message Flags Flags$Flag FetchProfile FetchProfile$Item)
           (javax.mail.internet InternetAddress))
  (:use clojure.contrib.str-utils))

This can quickly become unwieldy, especially if you start using the :as or :only or :rename arguments. It's made worse by Java's insane API's full of a billion classes that you need to import to do simple things. (And those things with dollar signs are mangled Java inner class names.) Clojure also lacks the ability to import a whole package worth of classes at once using java.io.* syntax, so you must name all of the classes explicitly.

every? vs some.

This is such a trite pet-peeve that it's barely worth mentioning, but it seems to be brought up repeatedly and endlessly on the Clojure mailing list so at least I'm not the only one bugged by it.

Clojure has a function (every? pred coll) which tests whether every item in a collection tests true via some predicate. To test whether every item in a collection tests false, we have not-any?. And we have a not-every? which tests whether any item tests false.

user> (every? even? [2 4 6])
true
user> (not-every? even? [2 4 6])
false
user> (not-any? even? [2 4 6])
false

Now what would you expect a function to be called which tests whether any item in a collection tests true via some predicate? If you said any? you are wrong! It's some.

Note that some isn't a predicate (hence no question mark in the name); it doesn't return true or false, as above, but rather returns the result of running pred on an item in coll.

user> (some identity [nil 1 2 3])
1

any? is pretty easy to write so it doesn't matter that much. Probably many people have an identical function sitting in some utils.clj file on their systems.

(defn any? [pred coll]
  (when (seq coll)
    (if (pred (first coll))
      true
      (recur pred (next coll)))))

Stack trace madness

Give this function:

(defn foo []
  (throw (Exception. "BARFED")))

What does the stack trace look like in SLIME when you call foo? Like this:

java.lang.Exception: BARFED (NO_SOURCE_FILE:0)
  [Thrown class clojure.lang.Compiler$CompilerException]

Restarts:
 0: [ABORT] Return to SLIME's top level.
 1: [CAUSE] Throw cause of this exception

Backtrace:
  2: swank.commands.basic$eval_region__729.invoke(basic.clj:36)
  3: swank.commands.basic$listener_eval__738.invoke(basic.clj:50)
  4: clojure.lang.Var.invoke(Var.java:346)
  5: user$eval__1506.invoke(NO_SOURCE_FILE)
  6: clojure.lang.Compiler.eval(Compiler.java:4580)
  7: clojure.core$eval__4016.invoke(core.clj:1728)
  8: swank.core$eval_in_emacs_package__336.invoke(core.clj:55)
  9: swank.core$eval_for_emacs__413.invoke(core.clj:123)
 10: clojure.lang.Var.invoke(Var.java:354)
 11: clojure.lang.AFn.applyToHelper(AFn.java:179)
 12: clojure.lang.Var.applyTo(Var.java:463)
 13: clojure.core$apply__3269.doInvoke(core.clj:390)
 14: clojure.lang.RestFn.invoke(RestFn.java:428)
 15: swank.core$eval_from_control__339.invoke(core.clj:62)
 16: swank.core$eval_loop__342.invoke(core.clj:67)
 17: swank.core$spawn_repl_thread__474$fn__505$fn__507.invoke(core.clj:173)
 18: clojure.lang.AFn.applyToHelper(AFn.java:171)
 19: clojure.lang.AFn.applyTo(AFn.java:164)
 20: clojure.core$apply__3269.doInvoke(core.clj:390)
 21: clojure.lang.RestFn.invoke(RestFn.java:428)
 22: swank.core$spawn_repl_thread__474$fn__505.doInvoke(core.clj:170)
 23: clojure.lang.RestFn.invoke(RestFn.java:402)
 24: clojure.lang.AFn.run(AFn.java:37)
 25: java.lang.Thread.run(Thread.java:619)

Yeouch. Now imagine that the above error is coming not from a simple function, but from some random line among hundreds of lines of source code.

Stack traces in Clojure will often tell you little to nothing about what is causing the error, or more importantly, where it's coming from in your code. Clojure functions are translated into Java classes when they're run through the JVM. Often can't even see the name of the function that's throwing the error; names are mangled into things like user$eval__1473.invoke, which is really really confusing when you use anonymous functions.

Per Jason Wolfe and Randall Schulz sometimes you can get a better stack trace if you dig a bit deeper:

user> (.printStackTrace (.getCause *e))

java.lang.Exception: BARFED
    at user$foo__1503.invoke(NO_SOURCE_FILE:1)
    at user$eval__1509.invoke(NO_SOURCE_FILE:1)
    at clojure.lang.Compiler.eval(Compiler.java:4580)
    at clojure.core$eval__4016.invoke(core.clj:1728)
    at swank.commands.basic$eval_region__729.invoke(basic.clj:36)
    at swank.commands.basic$listener_eval__738.invoke(basic.clj:50)
    at clojure.lang.Var.invoke(Var.java:346)
    at user$eval__1506.invoke(NO_SOURCE_FILE)
    at clojure.lang.Compiler.eval(Compiler.java:4580)
    at clojure.core$eval__4016.invoke(core.clj:1728)
    at swank.core$eval_in_emacs_package__336.invoke(core.clj:55)
    at swank.core$eval_for_emacs__413.invoke(core.clj:123)
    at clojure.lang.Var.invoke(Var.java:354)
    at clojure.lang.AFn.applyToHelper(AFn.java:179)
    at clojure.lang.Var.applyTo(Var.java:463)
    at clojure.core$apply__3269.doInvoke(core.clj:390)
    at clojure.lang.RestFn.invoke(RestFn.java:428)
    at swank.core$eval_from_control__339.invoke(core.clj:62)
    at swank.core$eval_loop__342.invoke(core.clj:67)
    at swank.core$spawn_repl_thread__474$fn__505$fn__507.invoke(core.clj:173)
    at clojure.lang.AFn.applyToHelper(AFn.java:171)
    at clojure.lang.AFn.applyTo(AFn.java:164)
    at clojure.core$apply__3269.doInvoke(core.clj:390)
    at clojure.lang.RestFn.invoke(RestFn.java:428)
    at swank.core$spawn_repl_thread__474$fn__505.doInvoke(core.clj:170)
    at clojure.lang.RestFn.invoke(RestFn.java:402)
    at clojure.lang.AFn.run(AFn.java:37)
    at java.lang.Thread.run(Thread.java:619)

This one at least mentions foo by name but you're still going to have a headache after a few hours of those stack traces.

Conclusion

So that's five things. You will notice a common theme. Most of these issues are inherited from the JVM. This is to be expected, I suppose. There's no way you can wrap one language in another without a few compromises.

But these things aren't show-stoppers. They are minor annoyances compared to the benefits you get from using the JVM, i.e. the good performance, tons of libraries, cross-platformness, and so on. Clojure is fun enough to work with and wart-less enough that it took me well over two weeks to write this post.

(If you were expecting me to mention loop/recur and the lack of native TCO in the JVM, you were PAINFULLY WRONG. No one who uses Clojure loses sleep over native TCO. It's largely a non-issue that's endlessly repeated by people looking for an excuse to pass up Clojure in favor of $their_pet_language. To each his own, but I have never found myself caring the slightest about loop/recur.)