Did you ever notice how footnotes make your writing seem more important¹ somehow?

Maybe one reason is that "real" books use footnotes. At a glance, it looks like I have references² backing up everything I say. In reality, I don't, but the connotation carries through somehow³. Now my blog seems scholarly and authoritative.

And if you're like me, you can't resist clicking footnotes to see what they refer to⁴. According to my estimates, by utilizing footnotes, in one fell swoop I have decreased my readers' average reading efficiency by 73%.

In any case, I've added experimental, rudimentary support for footnotes to cow-blog.

I'm loosely copying the syntax from Markdown Extra for this. Markdown is great, except when it isn't. The standard doesn't have support for some useful extensions. I use Showdown for Markdown support, and I'm probably going to work on adding more features of Markdown Extra to Showdown in the near future.

I just dread actually doing it. Showdown (like Markdown itself) is implemented as a series of hackish regex transformations of blobs of text. It's not a proper grammar. Implementing more of Markdown Extra means more regex blobbing. It's brittle and fragile and even getting incomplete support for footnotes was less than enjoyable. But at the same time I find myself wanting to do things that Markdown can't so, so I may have to bite the bullet.

(If there's a Showdown Extra out there already, drop me a URL. It'd be most appreciated. But I couldn't find one.)

Writing up a blog replacement for Wordpress (in Clojure) is coming along nicely. Clojure + Compojure are awesome. Most fun I've had making a website in a long while.

One problem I've run across is that I want to use Markdown for both post content and visitor commenting. I like Stack Overflow's live Javascript previews, so you can type text in Markdown and see what it'll look like as HTML, as you type it.

As I mentioned before, Markdown is very nice because it (partly) solves one longstanding issue I've had with programming blogs (including my own), namely the proper escaping of HTML and the proper formatting of source code. In Markdown you just put code in backquotes or indent it four spaces and there you go, properly escaped. Markdown is also easy and to type and read, which is a plus. I hate hate hate writing HTML by hand.

Anyways, there is no Markdown parser for Clojure, so I was going to write one. (There is MarkdownJ but it has unresolved issues.) The problem with writing my own Markdown parser in Clojure is that Markdown is not a well-specified language. There is no "official" grammar, just an informal "Here's how it works" description and a really ugly reference implementation in Perl. Most implementations (with the exception of peg-markdown and pandoc and friends) are implemented as a bunch of global regex-replacements passed repeatedly over some text.

The result is that there are a lot of Markdown parsers in a lot of languages, and they all give slightly different results in a lot of corner cases (and a lot of not-so-corner cases). The best I could do in Clojure is pick one implementation and try my best to match it.

Now, for each blog post, the server needs to store both the Markdown text and the HTML text. It needs the Markdown because if someone wants to edit content later, they need to edit the raw Markdown. It needs the HTML so that it can be cached and served to people viewing the website, obviously.

But a consequence of the above mess is that if you use a Javascript Markdown library (i.e. Showdown) to show a live preview, and then use a different Markdown library (my own or any other) to do server-side parsing of the text after it's POSTed, there's a good chance that the preview isn't going to match the real output.

One non-solution to this is to do all the parsing client-side, and POST both the Markdown and the post-Markdown HTML to the server so both can be stored, so no server-side parsing is necessary. Aside from being a horrid idea, it's a huge security risk because it leaves open the possibility of someone POSTing some clean Markdown along with some evil, un-matching HTML.

Another non-solution is to do all the parsing server-side and use AJAX to send the preview back to the client. That wouldn't be nearly as smooth or responsive as I want; on Stack Overflow for example the preview updates instantly after every keyup event in the textarea.

The ideal solution is use the same Javascript library client-side for previews, and server-side for parsing the text. Then the preview and content have a very high chance of matching. This requires some way to run Javascript on the server. Thanks to Clojure and Java and Rhino, this turns out to be trivial.

(ns bcc.markdown
  (:import (org.mozilla.javascript Context ScriptableObject)))

(defn markdown-to-html [txt]
  (let [cx (Context/enter)
        scope (.initStandardObjects cx)
        input (Context/javaToJS txt scope)
        script (str (slurp "showdown.js")
                    "new Showdown.converter().makeHtml(input);")]
    (try
     (ScriptableObject/putProperty scope "input" input)
     (let [result (.evaluateString cx scope script "<cmd>" 1 nil)]
       (Context/toString result))
     (finally (Context/exit)))))

This also saves me from having to write a Markdown parser in Clojure, for which I am thankful.

Once again I'm also thankful we live in times when CPU cycles are cheap and abundant. I'm running a Markdown parser, in Javascript, in Java, via Clojure, and this still runs essentially instantly even for very large input strings. If I had any chance of my blog becoming famous and getting a million hits a day, it might matter, but in real life I'm set.

To add yet another language to the ever-expanding list of languages I know well enough to get by but have far from mastered, in the past week or two at work I've needed to play with Javascript a lot. I've long thought of Javascript as the bane of my existence, if only for the horrid browser incompatibilities you have to deal with. I hate hate hate hate writing code that needs to have a bunch of silly conditionals that change the code's behavior based on differences in the underlying platform. In this case of course the platform is a browser, but I hate it equally dealing with differences between operating systems, differences between versions of libraries, and other similar things.

The ideal situation is to use some library where someone has already worked out those platform-dependent quirks. In the case of Javascript, there's JQuery. It really does work wonders. I read recently that Google uses it for some of their sites, and I can probably see why. JQuery gives you some helper functions that let you deal very nicely with event handling and element-selection.

To my surprise, Javascript actually is turning out to be a surprisingly powerful language. It apparently supports closures, and functions are first-order objects. The OOP system in Javascript is oddly different from other languages I've used, but it's workable. I'm doing some relatively funky stuff, and Javascript is handling it well enough. And everything I've done in JQuery so far works in IE6 and Firefox. That's a good sign.

In the course of all this, I'm also using PHP. I had to repress a PHP-induced shudder even as I write this. But this is my first venture into the realm of PHP5, and again it's turning out OK. They made a lot of changes between PHP4 and PHP5, many of which made a difference. The language is still far too verbose for my tastes; Ruby has spoiled me in terms of variable names without the need for Perl-style sigils. And the overload of array()'s in PHP still drives me batty. But this time, I'm making heavy use of some third-party libraries, e.g. Smarty templates for my interface, and Spyc to read some YAML for config and input files. It lets me do stuff in PHP without actually writing much code in PHP. Can't beat that.

Maybe that's the secret of life: to make up for deficiencies in languages I dislike by using tons of third-party libraries. Maybe. Well, no, probably not.

In other non-news, my Gentoo box should arrive at my door via UPS on Thursday. Sweet sweet salvation. I'm going to put this Vista laptop in the closet and bolt the door shut.