Five Things that Mildly Annoy Me in Clojure

This infamous blog post suggests that someone familiar with a language should be able to name five things they hate about it. "Hate" is a strong word, but I decided to think of five things I find mildyly annoying about Clojure, my favorite language of the moment.

Hashing integers

Clojure automatically converts integers between Integer, Long and BigInteger as needed to prevent overflow. This is good. Integers of the various classes test as equal too. This is also good.

user> (= 123 (int 123) (long 123) (bigint 123))
true

So would you expect this?

user> (hash-map (int 123) :foo (long 123) :bar (bigint 123) :baz)
{123 :foo, 123 :bar, 123 :baz}

Yes, each of the integer classes, though equal via =, do not have the same hash value when put into a hash-map. This is because:

user> (.equals (int 123) (long 123))
false

This is a wart inherited from the JVM. See here for discussion and explanation.

What's more, if you print this map and then try to read it back in, the integers will be read as int, long or bigint arbitrarily depending how big they are. This means you may not get the same class of object back that you output originally.

user> (def x {(bigint 123) :foo})
#'user/x
user> (= x x)
true
user> (def y (read-string (pr-str x)))
#'user/y
user> (= x y)
false
user> (class (first (keys y)))
java.lang.Integer
user> (class (first (keys x)))
java.math.BigInteger

This means that if you ever use integers as hash keys, you must be very careful to cast them all to the same integer type manually.

Metadata doesn't work on everything

Clojure lets you stick arbitrary metadata on various objects. This is higly useful; you can decorate objects with information that doesn't affect the value of the object. However metadata doesn't work everywhere.

user> (with-meta "foo" {:bar :baz})
java.lang.ClassCastException: java.lang.String cannot be cast to clojure.lang.IObj (NO_SOURCE_FILE:0)

You can only stick metadata on certain Clojure objects like Symbols, Vars, Refs, Agents, all of the Clojure collections and so on. You can't stick metadata on, say, a String or an Integer, because those are closed Java classes and can't be touched. It would be nice if you could.

use vs. require vs. import vs. load vs. ...

There are a startling number of ways to import a library into your code in Clojure. You have to choose from load, import, require, use, refer, and so on. Some work on Java classes, some work on Clojure libs. Some of them import symbols into your namespace, some of them don't. Some of them take strings as arguments, some take symbols, some take quoted lists of symbols, some take quoted lists of symbols with sub-lists of arguments. And all of these can be and usually are weirdly inlined into a namespace declaration, with a completely different list-quoting style.

So in Ruby you can do this:

require 'util'
require 'config'
require 'whatever'

Whether it's a gem, or a Ruby source file sitting locally, it all works the same as long as the load path is set up right.

But in Clojure, you do this (actual code from an IMAP library I wrote):

(ns qt4-mailtray.mail
  (:import (java.util Properties)
           (javax.mail Session Store Folder Message Flags Flags$Flag FetchProfile FetchProfile$Item)
           (javax.mail.internet InternetAddress))
  (:use clojure.contrib.str-utils))

This can quickly become unwieldy, especially if you start using the :as or :only or :rename arguments. It's made worse by Java's insane API's full of a billion classes that you need to import to do simple things. (And those things with dollar signs are mangled Java inner class names.) Clojure also lacks the ability to import a whole package worth of classes at once using java.io.* syntax, so you must name all of the classes explicitly.

every? vs some.

This is such a trite pet-peeve that it's barely worth mentioning, but it seems to be brought up repeatedly and endlessly on the Clojure mailing list so at least I'm not the only one bugged by it.

Clojure has a function (every? pred coll) which tests whether every item in a collection tests true via some predicate. To test whether every item in a collection tests false, we have not-any?. And we have a not-every? which tests whether any item tests false.

user> (every? even? [2 4 6])
true
user> (not-every? even? [2 4 6])
false
user> (not-any? even? [2 4 6])
false

Now what would you expect a function to be called which tests whether any item in a collection tests true via some predicate? If you said any? you are wrong! It's some.

Note that some isn't a predicate (hence no question mark in the name); it doesn't return true or false, as above, but rather returns the result of running pred on an item in coll.

user> (some identity [nil 1 2 3])
1

any? is pretty easy to write so it doesn't matter that much. Probably many people have an identical function sitting in some utils.clj file on their systems.

(defn any? [pred coll]
  (when (seq coll)
    (if (pred (first coll))
      true
      (recur pred (next coll)))))

Stack trace madness

Give this function:

(defn foo []
  (throw (Exception. "BARFED")))

What does the stack trace look like in SLIME when you call foo? Like this:

java.lang.Exception: BARFED (NO_SOURCE_FILE:0)
  [Thrown class clojure.lang.Compiler$CompilerException]

Restarts:
 0: [ABORT] Return to SLIME's top level.
 1: [CAUSE] Throw cause of this exception

Backtrace:
  2: swank.commands.basic$eval_region__729.invoke(basic.clj:36)
  3: swank.commands.basic$listener_eval__738.invoke(basic.clj:50)
  4: clojure.lang.Var.invoke(Var.java:346)
  5: user$eval__1506.invoke(NO_SOURCE_FILE)
  6: clojure.lang.Compiler.eval(Compiler.java:4580)
  7: clojure.core$eval__4016.invoke(core.clj:1728)
  8: swank.core$eval_in_emacs_package__336.invoke(core.clj:55)
  9: swank.core$eval_for_emacs__413.invoke(core.clj:123)
 10: clojure.lang.Var.invoke(Var.java:354)
 11: clojure.lang.AFn.applyToHelper(AFn.java:179)
 12: clojure.lang.Var.applyTo(Var.java:463)
 13: clojure.core$apply__3269.doInvoke(core.clj:390)
 14: clojure.lang.RestFn.invoke(RestFn.java:428)
 15: swank.core$eval_from_control__339.invoke(core.clj:62)
 16: swank.core$eval_loop__342.invoke(core.clj:67)
 17: swank.core$spawn_repl_thread__474$fn__505$fn__507.invoke(core.clj:173)
 18: clojure.lang.AFn.applyToHelper(AFn.java:171)
 19: clojure.lang.AFn.applyTo(AFn.java:164)
 20: clojure.core$apply__3269.doInvoke(core.clj:390)
 21: clojure.lang.RestFn.invoke(RestFn.java:428)
 22: swank.core$spawn_repl_thread__474$fn__505.doInvoke(core.clj:170)
 23: clojure.lang.RestFn.invoke(RestFn.java:402)
 24: clojure.lang.AFn.run(AFn.java:37)
 25: java.lang.Thread.run(Thread.java:619)

Yeouch. Now imagine that the above error is coming not from a simple function, but from some random line among hundreds of lines of source code.

Stack traces in Clojure will often tell you little to nothing about what is causing the error, or more importantly, where it's coming from in your code. Clojure functions are translated into Java classes when they're run through the JVM. Often can't even see the name of the function that's throwing the error; names are mangled into things like user$eval__1473.invoke, which is really really confusing when you use anonymous functions.

Per Jason Wolfe and Randall Schulz sometimes you can get a better stack trace if you dig a bit deeper:

user> (.printStackTrace (.getCause *e))

java.lang.Exception: BARFED
    at user$foo__1503.invoke(NO_SOURCE_FILE:1)
    at user$eval__1509.invoke(NO_SOURCE_FILE:1)
    at clojure.lang.Compiler.eval(Compiler.java:4580)
    at clojure.core$eval__4016.invoke(core.clj:1728)
    at swank.commands.basic$eval_region__729.invoke(basic.clj:36)
    at swank.commands.basic$listener_eval__738.invoke(basic.clj:50)
    at clojure.lang.Var.invoke(Var.java:346)
    at user$eval__1506.invoke(NO_SOURCE_FILE)
    at clojure.lang.Compiler.eval(Compiler.java:4580)
    at clojure.core$eval__4016.invoke(core.clj:1728)
    at swank.core$eval_in_emacs_package__336.invoke(core.clj:55)
    at swank.core$eval_for_emacs__413.invoke(core.clj:123)
    at clojure.lang.Var.invoke(Var.java:354)
    at clojure.lang.AFn.applyToHelper(AFn.java:179)
    at clojure.lang.Var.applyTo(Var.java:463)
    at clojure.core$apply__3269.doInvoke(core.clj:390)
    at clojure.lang.RestFn.invoke(RestFn.java:428)
    at swank.core$eval_from_control__339.invoke(core.clj:62)
    at swank.core$eval_loop__342.invoke(core.clj:67)
    at swank.core$spawn_repl_thread__474$fn__505$fn__507.invoke(core.clj:173)
    at clojure.lang.AFn.applyToHelper(AFn.java:171)
    at clojure.lang.AFn.applyTo(AFn.java:164)
    at clojure.core$apply__3269.doInvoke(core.clj:390)
    at clojure.lang.RestFn.invoke(RestFn.java:428)
    at swank.core$spawn_repl_thread__474$fn__505.doInvoke(core.clj:170)
    at clojure.lang.RestFn.invoke(RestFn.java:402)
    at clojure.lang.AFn.run(AFn.java:37)
    at java.lang.Thread.run(Thread.java:619)

This one at least mentions foo by name but you're still going to have a headache after a few hours of those stack traces.

Conclusion

So that's five things. You will notice a common theme. Most of these issues are inherited from the JVM. This is to be expected, I suppose. There's no way you can wrap one language in another without a few compromises.

But these things aren't show-stoppers. They are minor annoyances compared to the benefits you get from using the JVM, i.e. the good performance, tons of libraries, cross-platformness, and so on. Clojure is fun enough to work with and wart-less enough that it took me well over two weeks to write this post.

(If you were expecting me to mention loop/recur and the lack of native TCO in the JVM, you were PAINFULLY WRONG. No one who uses Clojure loses sleep over native TCO. It's largely a non-issue that's endlessly repeated by people looking for an excuse to pass up Clojure in favor of $their_pet_language. To each his own, but I have never found myself caring the slightest about loop/recur.)

June 16, 2009 @ 2:56 PM PDT
Cateogory: Programming
Tags: Lisp, Clojure

15 Comments

Fogus
Quoth Fogus on June 16, 2009 @ 11:33 PM PDT

Nice post and I agree mostly with your specific points. Likewise, I agree wholeheartedly about the need to find faults with your language of choice and/or most familiar with. I make it a point to ask such a question when interviewing job candidates as it shows that the person has taken the time to think about their language in more than just a superficial way. You'd be surprised how few people can devise one thing must less five.

-m

Drew Raines
Quoth Drew Raines on June 17, 2009 @ 1:03 AM PDT

You can hit 1 in your sldb buffer to getCause automatically so you don't have to type out printStackTrace. Notice the restart, " 1: [CAUSE] Throw cause of this exception."

Phil
Quoth Phil on June 17, 2009 @ 2:19 AM PDT

1 and 2 are pretty annoying. Metadata is never going to work on core Java classes, and that is a shame, though it should get added for functions soon.

use vs require vs import is pretty straightforward. Is it a Java class? Then import it. Do you want its public vars in your namespace? Then use. Otherwise require. load and refer are low-level functions that you shouldn't be using directly.

Stack traces can be greatly improved from within SLIME by (0) running the latest swank-clojure, which removes the caused-by-swank wrapper exception, and (1) using this code snippet that dims all the irrelevant frames from the trace:

http://github.com/technomancy/emacs-starter-kit/blob/8ddc188e076cd1dea2e907ce04a89c96b6efa25d/starter-kit-lisp.el#L53

It makes the relevant lines jump out much more quickly.

Chas Emerick
Quoth Chas Emerick on June 17, 2009 @ 2:52 AM PDT

You're definitely right about the hashing of ints, but if you want to read stuff in, you should do it right:

user=> (def x {(bigint 123) :foo})
#'user/x
user=> y
"{123 :foo}"
user=> (def y (binding [*print-dup* true]
                (pr-str x)))
#'user/y
user=> y
"#=(clojure.lang.PersistentArrayMap/create {#=(java.math.BigInteger. \"123\") :foo})"
user=> (def z (read-string y))
#'user/z
user=> (= x z)
true
user=> (class (first (keys z)))
java.math.BigInteger
user=> (class (first (keys x)))
java.math.BigInteger

Granted, *print-dup* is not well-known, but it's a great mechanism for ensuring that what you print is what you get (WYPIWYG?). FWIW, I'd read up on it if I were you (search the google group, etc), as there are some caveats (e.g. it doesn't handle arbitrary java objects, etc., but it is fully extensible if you care to do so for particular types).

Brian
Quoth Brian on June 17, 2009 @ 6:40 AM PDT

@Chas: Right. I knew about *print-dup* but didn't even think of it in this case, duh. Thanks for the reminder.

@Phil, Drew: Thanks for the SLIME tips. Didn't know about those.

@Fogus: Yeah if I ever have to interview someone again I'll probably use this question. If nothing else it shows the ability to be objective. People with a religious mindset about their favorite language and an unrealistic view of its perfection can be hard to deal with.

Jon Harrop
Quoth Jon Harrop on June 17, 2009 @ 6:41 AM PDT

No one who uses Clojure loses sleep over native TCO.

Self-selection.

Glen Stampoultzis
Quoth Glen Stampoultzis on June 17, 2009 @ 1:26 PM PDT

The cow at my last post. Sadface.

I love Clojure but the horrible stack trace has to be my main beef. For comparison I just took a look at JRuby and the stack trace, while not perfect, does not include nearly as much crud:

Exception in thread "AWT-EventQueue-0" swing2.rb:12:in "actionPerformed": undefined local variable or method "b" for #<ClickAction:0x169df00> (NameError)
    from :1
    ...internal jruby stack elided...
    from ClickAction.actionPerformed(:1)
    from (unknown).(unknown)(:1)`
Evil One
Quoth Evil One on June 17, 2009 @ 3:11 PM PDT

Lack of "native TCO" in the JVM is NOT the reason for "recur." If Clojure can do tail-recursion with a special form, it could have been designed to do it without a special form. Therefore, the designers of Clojure could have chosen to make the "recur" form implicit, but THEY CHOSE NOT TO, and THAT'S why you have "recur."

mikel evins
Quoth mikel evins on June 17, 2009 @ 10:36 PM PDT

Multimethods seem like a handy mechanism for providing polymorphic APIs, but there is a gotcha: Clojure does not provide a default ordering for inheritance relationships, so it's possible to accidentally write an ambiguous set of inheritance relationships. In that case, you'll see a stack trace until you manually fix the problem with a prefer-method call.

What's more, if you expose a multimethod API in a library, your users can accidentally break your inheritance scheme simply by defining some new tags that inherit from the old ones in your library. They won't notice this problem (and neither will you) until they happen to create an ambiguous inheritance relationship somewhere, and then they'll notice it because code that used to work fine now suddenly causes backtraces. They can fix the breakage, of course, by making the right set of prefer-method calls...if they can figure out what those are. You might want to make sure they have the source code to your library.

You might think, "oh, I'll just write code to compute a stable order for derived types, and call prefer-method automatically, so my users won't have this problem." You can't. derive hierarchies represent inherited types as unordered sets; you cannot use them to get a stable, ordered sequence of inherited types. If you want to make libraries with stable, robust, polymorphic APIs, you'll need to either write those parts in Java or implement your own substitutes for multimethods and derive hierarchies.

The Clojure community, by the way, considers these characteristics of multimethods and derive hierarchies to be features, not bugs.

Doug Clinton
Quoth Doug Clinton on June 19, 2009 @ 9:33 PM PDT

The stack trace comment really hit home for me. I've been playing with Clojure for a couple of weeks now, using it to develop a solution for a real problem that I need to solve, and it seems to me that if there is one thing which will stop Clojure achieving any kind of mainstream acceptance it is the hideous nature of error messages.

Not since working with templates in early C++ compilers have I seen error messages (in clojure's case, in the form of stack traces) which were so totally divorced from the actual cause of the problem. Earlier this week I spent 15 minutes trying to track down a problem in my code which should have taken 15 seconds to fix simply because the stack trace gave me so little information that I ended up having to litter the code with prinlns just to find where the error was being generated. This will be a major drag on productivity in any real-world development and at the moment, putting Clojure in front of an inexperienced programmer will be a disaster.

I had a poke around in the Clojure source and it is clear that the architecture does not lend itself well to providing meaningful error messages, but I believe that by adopting the right conventions for the exceptions that enough information could be collected as the exception propagates back up the stack to put together clear and concise information to describe the error. I may have a go at some particular cases and submit them.

Mark
Quoth Mark on June 21, 2009 @ 6:45 PM PDT

The stack traces in Clojure are truly awful. Is it really so hard to display the lines from my code which contributed to the error? I've tried the various packages that supposedly clean up the stack trace, and they seem to cut the stack trace down quite a bit, but a significant percentage of the time, the important parts of the stack trace relating to my code are removed as well!

Most of your other points I agree with, but I find them easy enough to work around.

I can think of lots of other things that bother me about Clojure. Here are ten:

  1. Clojure code must be arranged in a bottom-up manner, otherwise lots of forward declarations are required, which clutters up the program. Since I like to write code in a top-down manner that is easy to read and understand, this bugs me.

  2. I agree with Mikel's point about multimethods found in the comments. The current design of multimethods, which relies on prefer-method to disambiguate, has some extensibility issues.

  3. Lazy sequences always cache their contents, which is great for sequences whose elements are generated from some computationally-intensive or imperative procedure. But for sequences based off of simple, pure-functional generators, this caching is unnecessary, slowing things down and using up lots of memory. range is implemented internally by Clojure in a way that avoids caching, and it would be nice if Clojure programmers could easily do the same.

  4. Metadata is cool, but adding metadata and working with it can get a bit verbose. #^ is not a synonym for with-meta, but I think a lot of people wish it were, because a special notation for adding/altering metadata would be extremely useful.

  5. I personally don't like nil punning, and although Clojure allows you to avoid nil punning in a lot of instances, it's still considered a desired idiom to use (seq s) instead of (not-empty s), one-armed when clauses that don't specify what happens when the sequence is empty (because the answer just defaults to nil), and other similar examples.

  6. Other than defn-, most defines don't have a private variation. Ditto with doc strings. Would like to see more consistency.

  7. Hard to create structs with "private" information that can only be seen by certain functions.

  8. Hard to modify equality behavior for custom structs or collections. Arithmetic operators are also difficult to customize for new data structures.

  9. I don't like the Clojure philosophy to wrapping Java libraries. Java in Clojure is significantly uglier than pure Clojure, so I'd much rather use wrappers that make a Java library feel more like a native Clojure library.

  10. Clojure inherits from Java a fairly limited stack. Most tree traversals cannot easily be converted to loop/recur, and I really hate having to constantly worry about whether I'm going to blow the stack when building and manipulating trees.

That said, I like Clojure a lot better than the alternatives.

Mark
Quoth Mark on June 21, 2009 @ 6:54 PM PDT

I almost forgot to mention that I don't like the "untagged" nature of records. I prefer Oz's way: circle(x:0 y:1 radius:3 color:blue style:dots) Notice that the struct (aka record) also has a (optional) tag telling you what kind of struct it is (in this case, a circle). I think it's often very useful to know what kind of thing a struct represents.

You can use a :tag field in every record, but then you'll want to create a custom constructor (as opposed to just using struct-map). Also, different programmers will use a different keyword for the "tag". If you iterate through the key-value pairs, you'll end up with :tag as one of the keys, which probably isn't what you want.

Mark
Quoth Mark on June 22, 2009 @ 1:16 AM PDT

Whoops, in number 9 above, I meant I don't like the Clojure culture of NOT wrapping a lot of Java libraries. Would prefer as much Clojure-friendly wrapping as possible.

Brian
Quoth Brian on June 22, 2009 @ 3:43 AM PDT

I agree with much of your list Mark. With regards to 6, you might already be aware but there are private versions of defmacro etc. in clojure.contrib.def. Likewise for 8 there are a bunch of clojure.contrib.generic that provide math functions as multimethods, complex numbers etc. with the downside of being pretty slow compared to native math, obviously.

I would like to see those things built-in to Clojure too personally. The distinction between Clojure builtins and Clojure contrib libraries is confusing and seems kind of arbitrary. Personally I prefer everything to be included in the base language as much as possible, for convenience if nothing else.

Not sure how I feel about 9. Calling Java is so Lispy already that I'm usually not bothered. I am concerned about what's going to happen if the .NET port of Clojure becomes popular. Seems like if you want interoperable libraries, then thin pure Clojure wrappers would be necessary.

Fred
Quoth Fred on June 24, 2009 @ 8:23 AM PDT

I agree with most of your point, except some vs. every. These predicates are thankfully the same as their like-named Common Lisp counterparts. I (non-native speaker) learned in my English class many years ago, that the opposite of some is not any, so these predicates seem to be named right. And some is the only predicate of those 4 which can return a more useful value than simply true, so I think it should do so.

Speak your Mind

You can use Markdown in your comment.
Email/URL are optional. Email is only used for Gravatar.

Preview