user> (let [milli (.getTime (.getTime (java.util.Calendar/getInstance)))
            ts (java.sql.Timestamp. milli)
            d  (java.util.Date. milli)]
        [(= (.getTime d) (.getTime ts))
         (= ts d)
         (= d ts)])
[true false true]

Really, Java? Really? Come on now.

Documenting it doesn't make it any better either:

Note: This type is a composite of a java.util.Date and a separate nanoseconds value. Only integral seconds are stored in the java.util.Date component. The fractional seconds - the nanos - are separate. The Timestamp.equals(Object) method never returns true when passed an object that isn't an instance of java.sql.Timestamp, because the nanos component of a date is unknown. As a result, the Timestamp.equals(Object) method is not symmetric with respect to the java.util.Date.equals(Object) method. Also, the hashcode method uses the underlying java.util.Date implementation and therefore does not include nanos in its computation.

The internet is strewn with the wreckage of people being gnawed upon by this bug.

I'm making a new website (in Clojure of course) and I have a need to resize uploaded images to make thumbnails. At first I tried to use JMagick because I'm familiar with ImageMagick already and it seemed like an OK library. But on the crusty old OS my VPS uses, I had a really hard time getting it to build, and even once it built it started segfaulting like mad.

Should've gone with something simpler first. Java has built-in libraries for this. It only took a few seconds to adapt this code on Stack Overflow into Clojure code.

(use 'clojure.contrib.java-utils)
(defn make-thumbnail [filename new-filename width]
  (let [img (javax.imageio.ImageIO/read (as-file filename))
        imgtype (java.awt.image.BufferedImage/TYPE_INT_ARGB)
        width (min (.getWidth img) width)
        height (* (/ width (.getWidth img)) (.getHeight img))
        simg (java.awt.image.BufferedImage. width height imgtype)
        g (.createGraphics simg)]
    (.drawImage g img 0 0 width height nil)
    (.dispose g)
    (javax.imageio.ImageIO/write simg "png" (as-file new-filename))))

Writing up a blog replacement for Wordpress (in Clojure) is coming along nicely. Clojure + Compojure are awesome. Most fun I've had making a website in a long while.

One problem I've run across is that I want to use Markdown for both post content and visitor commenting. I like Stack Overflow's live Javascript previews, so you can type text in Markdown and see what it'll look like as HTML, as you type it.

As I mentioned before, Markdown is very nice because it (partly) solves one longstanding issue I've had with programming blogs (including my own), namely the proper escaping of HTML and the proper formatting of source code. In Markdown you just put code in backquotes or indent it four spaces and there you go, properly escaped. Markdown is also easy and to type and read, which is a plus. I hate hate hate writing HTML by hand.

Anyways, there is no Markdown parser for Clojure, so I was going to write one. (There is MarkdownJ but it has unresolved issues.) The problem with writing my own Markdown parser in Clojure is that Markdown is not a well-specified language. There is no "official" grammar, just an informal "Here's how it works" description and a really ugly reference implementation in Perl. Most implementations (with the exception of peg-markdown and pandoc and friends) are implemented as a bunch of global regex-replacements passed repeatedly over some text.

The result is that there are a lot of Markdown parsers in a lot of languages, and they all give slightly different results in a lot of corner cases (and a lot of not-so-corner cases). The best I could do in Clojure is pick one implementation and try my best to match it.

Now, for each blog post, the server needs to store both the Markdown text and the HTML text. It needs the Markdown because if someone wants to edit content later, they need to edit the raw Markdown. It needs the HTML so that it can be cached and served to people viewing the website, obviously.

But a consequence of the above mess is that if you use a Javascript Markdown library (i.e. Showdown) to show a live preview, and then use a different Markdown library (my own or any other) to do server-side parsing of the text after it's POSTed, there's a good chance that the preview isn't going to match the real output.

One non-solution to this is to do all the parsing client-side, and POST both the Markdown and the post-Markdown HTML to the server so both can be stored, so no server-side parsing is necessary. Aside from being a horrid idea, it's a huge security risk because it leaves open the possibility of someone POSTing some clean Markdown along with some evil, un-matching HTML.

Another non-solution is to do all the parsing server-side and use AJAX to send the preview back to the client. That wouldn't be nearly as smooth or responsive as I want; on Stack Overflow for example the preview updates instantly after every keyup event in the textarea.

The ideal solution is use the same Javascript library client-side for previews, and server-side for parsing the text. Then the preview and content have a very high chance of matching. This requires some way to run Javascript on the server. Thanks to Clojure and Java and Rhino, this turns out to be trivial.

(ns bcc.markdown
  (:import (org.mozilla.javascript Context ScriptableObject)))

(defn markdown-to-html [txt]
  (let [cx (Context/enter)
        scope (.initStandardObjects cx)
        input (Context/javaToJS txt scope)
        script (str (slurp "showdown.js")
                    "new Showdown.converter().makeHtml(input);")]
    (try
     (ScriptableObject/putProperty scope "input" input)
     (let [result (.evaluateString cx scope script "<cmd>" 1 nil)]
       (Context/toString result))
     (finally (Context/exit)))))

This also saves me from having to write a Markdown parser in Clojure, for which I am thankful.

Once again I'm also thankful we live in times when CPU cycles are cheap and abundant. I'm running a Markdown parser, in Javascript, in Java, via Clojure, and this still runs essentially instantly even for very large input strings. If I had any chance of my blog becoming famous and getting a million hits a day, it might matter, but in real life I'm set.

For my programming entertainment needs, I frequent Slashdot and Reddit, but lately also Stack Overflow. Stack Overflow is turning out to be a good source of information. I've posted some very obscure Emacs questions and gotten great answers in a few hours. Any question you post, no matter how obscure, is likely to be pounced upon by rabid reputation-seeking answerers.

What is it about a slowly-increasing numeric representation of your value as a human being that is so appealing? This is one thing Perl Monks got right early on. People there always say XP points don't matter, but I think it does on some level. It's just a silly number, but even knowing it's silly, on some deep dark level, you look at someone with 10,000 XP differently from someone with 400.

I think one of the best things you can experience as a programmer is for other programmers to think (and say) that you're smart. Maybe that's true of all professions and hobbies, but I know it's true of most programmers. Anything which takes good advantage of that fact of programmer psychology is on track to become successful.

Only bad thing about Stack Overflow is how saturated it is with .NET and Java. If you can wade through that crap, you can find some cool stuff.

I'm still exploring Clojure + Qt Jambi as a nice way to build GUI apps. I have some code to upload, if I can ever figure out how github works. The learning process really never ends for a programmer, does it? Not that I mind it, I actually love it. I love learning new tools and new languages. Git seems interesting and so many people say such good things about it that I almost have to learn it now.

Anyways, anyone who uses Qt Jambi should be aware of issues with memory leaks. Turns out you need to explicitly dispose of toplevel Qt objects or else they're never garbage collected. (This isn't as bad as it might be... most Qt objects aren't toplevel, they have parent objects.) I made a long-running system tray app and I noticed it slowly but steadily increasing in memory usage over the course of a week. I added a few calls to dispose and so far so good.

On that note, while Clojure is awesome and I think a lot of people's "OMG it's on the JVM, run!" reaction is unwarranted, the JVM still does seem to have some memory issues, on my system anyways. You do pay a RAM tax for using the JVM. A simple Clojure Qt4 app that just sits in the system tray and checks my email every couple seconds:

  PID  PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  TTY      COMMAND
 3282  20   0  282m  54m  17m S    0  2.7  11:08.75 pts/3    /usr/lib/jvm/sun-jdk-1.6/bin/java ...

54MB seems excessive. I need to look into ways of reducing that. I've passed in all kinds of parameters to the JVM limiting the heap size etc.; it was actually MORE than that before, which is even sadder.

One of the things that's either really good or really bad about Clojure, depending on your perspective, is that it forces you to learn Java and the JVM better. You don't need much Java knowledge to use Clojure, but it surely helps as soon as you want to interoperate with Java libraries. As much as I dislike Java, I recognize the need to have a job and acquire money with which to purchase such things as food and shelter. Java is good for that kind of thing.

There are some good things about Java. The virtual machine has been refined for quite some time. The garbage collector is likely to perform well. The standard library has gone through many iterations and is very encompassing and complete and amazingly well-documented. The community is enormous. The language is as cross-platform as you could reasonably expect any huge program to be. It has nice GUI frameworks (which nowadays even look native on Windows and Linux, if you use SWT), a good threading library, good socket libraries, and all the things I wish Ruby or Common Lisp had.

The one unignorably bad thing about Java is that you have to write it in Java. It's next to impossible to write Java by hand, and it's still a whole lot of pain even if you use one of the massive Java IDEs that trick you into not noticing the pain. The language is way too verbose. The syntax is busy and full of mandatory brackets and parens and punctuation and bullcrap. The demand that you catch every conceivable exception is tiresome. The ability to abstract all of those things away isn't present. The package / import scheme is way too much typing for any human being. No Lisp-style macros, no easy-to-use anonymous functions, clunky iterators, primitive looping constructs. Everything is forced into a Object Oriented mindset even if it doesn't fit well. And so on.

But the good thing is that nowadays you don't have to write Java in Java. You can write Java in Ruby using JRuby, or write Java in Lisp using Clojure. I gave both of these a try in the past week or so, and both are awesome. You can write the bulk of your program in a nice powerful fun-to-write language, and call out to Java to handle the GUI bits or whatever Java is good at handling. You can write tasty Ruby or Lisp abstractions that hide the horrible mess that is Java's syntax. (There's also Jython, if you swing that way.) You'd think it'd be more effort to write Ruby code that translates to Java than just to write plain old Java, but Ruby is so much better than Java that it actually ends up being easier. For me anyways.

It seems like the lines between programming languages is awfully blurry nowadays. Most languages have some way to interface directly with C code. There are GTK and QT bindings for everything under the sun. We have people writing Lisp interpreters in Javascript, Python interpreters in Lisp, and so on. You have all these VMs (JVM and .NET and Parrot (if it ever gets done)) which let you write the same program in whatever language you feel like. And most of them are cross-platform to some degree or other.

It's an interesting trend which makes a lot of practical sense. No language is great at everything, so why bother limiting yourself to one language per program? Especially with computers being so fast today, we can get away with layering language on top of language. It's a nice situation, if you want to get programs done as fast and with as little effort as humanly possible.

In the past I believe I may have said that Java was god-awful. I was mistaken. Java is far more evil than any god. Java sucks like the black, empty vacuum-void of nothing that existed before time began.

I'm tasked with maintaining and updating some Java code at work. Today I wandered in to work in a chipper mood. The sun was shining, the birds were singing, life was good. Oblivious, I sat at my computer and fired up Eclipse. 20 minutes later it opened and I was looking at some Java code.

Events played themselves out like they usually do As Java flowed across my computer screen, at first it was a slight twinge in the back of my mind that something was wrong. All too quickly that twinge became a dull sort of deep-seated pain. As the minutes passed, my good mood drained from me like my life-blood draining from an open wound. Joy was replaced with horror at the programming atrocities being committed before my eyes. I'm not exaggerating to say that a few times I've looked at my screen wide-eyed, mouth agape, as involuntary mad laughter escaped me.

Whatever my many other faults, I'm pretty fast at reading code and understanding it. Still it took me over an hour today just to locate the place I needed to add five lines of code to get my job done. Generally I'll start at the top of some 20-level-deep hierarchy of classes and interfaces and abstract classes, and work my way down into it until I look back and realize I have no idea where I came from or what I'm even looking for any longer. You see, public abstract class com.someDomain.project.client.model.table.AbstractEditableTableModel, which extends AbstractTableModel and implements EditableTableModel, which itself is a superclass of GenericTableDataModel and MultiTableDataModel, may contain the stub of a method I have a vague notion I may want to look at someday, but the fun thing about Java is that I get to search the whole hierarchy of imaginary insubstantiatable classes top to bottom until I find the overridding method that really implements the logic I need to see. Then I get to re-search the whole tree to find all the methods that method calls, along with the many separate trees of classes of all the objects the first class uses.

The code I'm dealing with is 210 classes and interfaces worth of Java that does little more than fetch some data from a database and display it in a GUI in a couple of grids. Those 210 classes of course extend and implement many bits and pieces of standard classes from the Java's standard library, meaning the number of classes I really have to deal with is far higher. And most of them DON'T DO ANYTHING.

Most of this code is completely without comments of course, and important variables are named things like s and data. And methods have names like good old actionPerformed; so if the program, you know, performs any actions, I guess I'm covered.

But I shouldn't say it's entirely without documentation. Here's one method along with its documentation.

// used to propogate [sic] privilges [sic] down to the lowest node
public void pushPrivileges()
{
    int myPrivileges = getPrivileges();
    Iterator i = tables.keySet().iterator();
    while( i.hasNext() )
    {
        tableModel tbl = tables.get(i.next());
        tbl.addPrivileges(myPrivileges);
        tbl.pushPrivileges();
    }
}

This is one of the hundreds of boilerplate methods in the code I'm dealing with. I often wonder what proportion of code in Java actually directly works to solve your problem, and what proportion just flips buttons and turns knobs on hundreds of objects which in turn flip buttons and turn knobs on other objects, most of which have no need to exist, while Java spins its wheels in the mud. The above method would be one or two lines of straightforward Lisp or Ruby code, if this method even had a need to exist. Java is much like a 100-armed monster, who will put 3 of those arms to work to solve your problem if you very carefully tell it how to use the other 97 arms to masturbate itself.

The highpoint of my day though was finding a method called "isHasDeletePrivilege". Only Java could inspire such madness in a programmer. I very barely managed to keep myself from right-clicking in Eclipse, selecting refactor => rename and typing canHasDeletePrivilege. If you try to read Java code from the perspective of a group of retarded pseudo-sapient cats living on the internet, it actually makes a bit more sense.

The worst thing about Java is that it actually just barely can be used to solve most problems, which only continues to encourage people to use it.

The SBCL download page shows version 1.0.14 released today, and it's already in Portage (masked). The gentoo-lisp list says we got a new Lisp project lead recently. Looks like there's plenty of Lisp going on in the Gentoo world. Personally I am very pleased with the state of Lisp in Gentoo.

Sometimes I wonder what purpose this blog serves. One purpose I found for it today was looking back through my old entries, to see how things have changed over the past couple years. I very strongly believe in introspection for the purpose of refining beliefs to make them more accurately reflect reality. In other words, I know I'm probably wrong about a lot of crap and I really don't like the thought. It bugs me. So I'm always looking for ways to change my perspective on things, if it needs changing.

I first tried Lisp in August 2006, it seems. Some of what I said was somewhat amusing, and wrong. Quoth myself:

My prediction is that Ruby will end up being mostly a superset of Lisp except for a few areas Lisp is specifically targetted at.

Oops! In fact just the opposite is true. Common Lisp is easily a superset of Ruby in all the ways that matter, specifically meta-programming and the flexibility of the object system, to name a few.

Lisp is sadly just a subset of Ruby in terms of the amount of libraries available. Ruby has tons of people writing tons of code for it. But I am finding that Common Lisp actually has a LOT more libraries than you'd guess from a quick glance. The problem is that Common Lisp is so much less "mainstream" than most languages that you have to do some digging to find the libraries you need. Once you do find them, they tend to be of high quality and great utility, from my brief experience.

Then again, Ruby itself is a subset of Perl in this regard. There's a critical mass where you probably have "enough" libraries to get the job done. Perl+CPAN is probably over the top in this regard. Ruby is there. Common Lisp is pretty close.

Quoth myself:

I must admit, properly formatting Lisp seems confusing.

Yeah, I do remember indentation of Lisp code to be pretty confusing back then. In PCL Seibel talks about how "experienced Lispers" use indentation to tell them if something funky is going on with their parens. I realized today I actually do this too now (though I am not an "experienced Lisper"). LET forms look a certain way, IF forms look a certain way, standard function calls look a certain way. You can tell immediately if something fishy is going on if the indentation is being screwy.

Actually I think people probably do this in most languages. If you've written any amount of Ruby, you know that eventually you often end up with 87 end's in a row, some closing if-then statements, some closing iterator blocks, some closing method definitions, some closing class definitions. If you type an end in Vim and it launches all the way to column 1, but you weren't expecting it there, that can tell you that you have something wrong (an extra end somewhere).

But it's much more necessary in Lisp. It seems to me that it would be extremely difficult to write good Lisp code without an editor's help. There are way too many parens for a human to keep track of. That's not to say that Lisp requires a full-blown IDE just to make the language usable (*cough*Java*cough*). Lisp syntax is regular enough that it's REALLY EASY for an editor to very consistently help you keep your parens balanced. The rules are highly logical, simple, and surprisingly standard across the Lisp community (compared to the tabs vs. whitespace, 2 vs. 4 vs. 8 spaces, curly-braces-on-newlines-or-not sorts of wars you'll find in some other languages' communities). If you use paredit or something similar, keeping your parens balanced and indented nicely is a no-brainer.

At night to relax I've still been reading about Common Lisp. There's a lot to like about it.

But there's one thing that drives me crazy:

• Let's define a variable: DEFVAR, DEFPARAMETER.
• Let's define a method: DEFMETHOD.
• Let's define a macro: DEFMACRO.
• Let's define a package: DEFPACKAGE.
• Let's define a class: DEFCLASS.
• Let's define a symbol macro: DEFSYMBOL Whoops, nope, it's DEFINE-SYMBOL-MACRO.
• Let's define a condition class: DEFCONDITION Sorry, nope, it's DEFINE-CONDITION.

...

• Let's access an element of a plist: (GETF SOMELIST :KEY)
• Let's access an element of an array: (AREF SOMEARRAY INDEX)
• Let's access an element of a vector: (ELT SOMEVECTOR INDEX)
• Let's access an element of a hash: (GETHASH SOMEHASH KEY) Wrong! It's (GETHASH KEY SOMEHASH).

For some macros, you have to quote your arguments. For some others, you don't. For some it's a mixture. Sometimes a symbol will evaluate to a variable. Sometimes it'll be read as a symbol, not evaluated. Sometimes it'll be evaluated but you never want it to be evaluated so you always need to quote it.

Sometimes parameters to function calls need to be grouped into lists and sublists. Sometimes you just pass in a big flat bunch of arguments and the method sorts them out. Often it's an arbitrary mix of the two.

Yeah, my problem is consistency. Programming is hard enough without throwing in a requirement for a bunch of rote memorization of the wording and ordering and grouping of things. Especially when those things are arbitrary, and there's no reason NOT to have them be consistent. In Lisp they're probably even more arbitrary than other languages.

This is Lisp, and if I wanted, I could write my own macros that translate new, consistent versions of those things into the inconsistent default versions. Lisp gives you the power to do that easily. But if I do that, I'm no longer writing the same Lisp everyone else* is writing. I'm writing my own derivative where I'm replacing standard things with non-standard things of my own.

There's more to a language than being syntactically correct, after all. There are dialects and idioms, and those are very important. If you learned English from a textbook, but never spoke it aloud with anyone speaks English natively, would you be able to communicate well? Would someone from England or America or India understand you? Likely you would be just barely intelligible to all of the above. It's the same English language, but how you use the language matters as much as being strictly grammatically correct. Your accent, your word choice, your general style, those things matter a whole lot.

It's the same with programming languages. If I wrote Ruby and used C-like for loops and 8-character-wide tab-indentation and CamelCase and ended every line with a semicolon, anyone very familiar with the language would cringe to read it. In Lisp, where you can change pretty much everything about the language, there's even more potential for writing things that are unintelligble to the community at large.**

Lisp does have a beautiful consistency to it overall though.*** Everything is an s-expression. (Well, except LOOP. Oops!) Everything returns a value. Code and data and code that writes code are all the same thing. It's only a few little quirks like the ones I mentioned above that drive me crazy. Consistency is also a large part of why I like Ruby: everything is an object, (almost) everything returns a value. And horrible inconsistency is one reason things like Java and PHP drive me absolutely insane.

The power of programming is in abstraction. When a group of things aren't consistent, there is no abstraction that can represent them. Or if there is, the abstraction is likely to be as nasty as the original problem. That's a bad thing.

( * "everyone else" is probably around 17 people worldwide.)
( ** "at large", for lack of a better term.)
( *** Jokes about how nobody uses Lisp aside, I do like the language plenty.)