This is a read-only archive!

Clojure 1, PHP 0

Goodbye Wordpress

As I mentioned many times, I've been working on replacing Wordpress for my blogging needs. Wordpress has been pretty good for the past three years, but it's time to move on, for a bunch of reasons.

Primarily, the way Wordpress automatically mangles my text is annoying. For example, it turns newlines into paragraphs inconsistently (especially when it comes to pre/code blocks). This blog is mostly about programming, which means being able to post code without having my quotes turned into "smart" quotes and my --flags turned into long-dashes is kind of important. HTML is sometimes automatically escaped, and sometimes not. I can't count how many comments I've gotten where someone posted some code, then posted again to inform me that Wordpress ate the code for dinner. There are plugins to fix some of this, which break every time Wordpress releases a new version, and have never really worked that well for me.

Writing a theme for Wordpress means a mix of PHP and HTML and CSS, which is painful to read and even more painful to write. Aside from the considerable ugliness of PHP itself, there's a lot of weird magic involved with themes, based on naming conventions for files, weird fall-through behavior when certain theme files aren't present and so on. The Wordpress API is enormous and not fun to work with if you want to do something other than the standard Wordpressy kind of blog structure. Static pages aren't too much fun to work with in Wordpress either.

Lately I think I was getting hammered with spam partly because Wordpress is such an easy target. Askimet is nice but it wasn't catching enough lately; maybe 10-15 spams per week were slipping through. And there was always the chance that some widely-known exploit in Wordpress was going to leave my site susceptible to some roving bot.

And so on.

Hello Clojure

Why Clojure? Because it's awesome and fun and powerful and I wanted to learn it better.

Compojure is a web framework for Clojure that made a lot of this very easy. Coming here from a Ruby on Rails background, Compojure has a lot going for it in comparison. Compojure is lightweight and more low-level than Rails. For example Compojure doesn't enforce MVC on you, doesn't force a unit testing framework on you, and doesn't care how you access your data. Compojure just lets you route HTTP requests to Clojure functions based on the URL and request method (RESTfully: POST/GET/DELETE/PUT), and it gives you easy access to the request information, session, GET/POST parameters and cookies.

Under the hood it's all servlets and Jetty, both of which are solid, stable, well-tested, well-documented technologies. However, thankfully, all of that Java stuff is under the hood, and well under it. I didn't have to write a single line of Java or interact with single servlet directly. Everything (session, params, headers) is a Clojure hash-map from the perspective of my code.

Compojure also comes with a domain-specific language for writing HTML, which is similar to CL-WHO and myriad other Common Lisp HTML DSL's. All of which are awesome. I can't say enough how much nicer it is to write (or generate) structured s-exps than to write HTML by hand. More on that below.

Compojure doesn't come with any way to interact with a database, so I had to write one. clojure.contrib has an SQL lib which easily lets you interact with a MySQL database. (Clojure can talk to MySQL via MySQL's JDBC connector, of course.) I used clojure.contrib.sql to write a small (192 lines) library which slurps up a bunch of database tables into Clojure refs, and provides a few functions for basic CRUD operations so that any updates to the ref data is also transparently reflected in the database. The database is essentially only for keeping an on-disk cache of the data in case I need to restart the server. The average number of DB queries per page is zero; everything except posting/editing/deleting data just reads out of a Clojure ref.

With possibly multiple users posting data at once, it's nice to have Clojure's built-in concurrency support. Updating the data refs with new data is always safe from multiple threads simply by throwing a (dosync) around all of the write accesses. This was completely painless to write.

I decided I wanted to use Markdown for posting comments and authoring new pages. This was also very simple to do; I outlined how to get Markdown working in Java and Clojure, in a previous post. The real-time previews for comments are largely inspired by / ripped-off from Stack Overflow, implemented mostly using open-source Javascript libraries like Showdown, JQuery, TypeWatch and TextAreaResizer.

A Brief Comparison: Clojure vs. Wordpress

All of my code including the CRUD library, all of the HTML for the templates and layout, admin controls, and all the glue to put it together is 1,253 lines of code. Wordpress is somewhere over 78,000 lines of PHP depending what you count (doesn't include any themes or layout, but does include Wordpress features I didn't need and didn't implement). It's still a pretty nice reduction in code overall, any way you look at it.

As an example, in my old Wordpress site I had a plugin catcloud to generate a "tag cloud". This plugin itself is 226 lines of PHP, not bad. However, here's the Clojure code to generate a similar tag cloud (which you can see here currently):

(defn tag-cloud []
  (let [tags (sort-by #(.toLowerCase (:name (first %))) (all-tags-with-counts))
        counts (map second tags)
        max-count (apply max counts)
        min-count (apply min counts)
        min-size 90.0
        max-size 200.0
        color-fn (fn [val]
                   (let [b (min (- 255 (Math/round (* val 255))) 200)]
                     (str "rgb(" b "," b "," b ")")))
        tag-fn (fn [[tag c]]
                 (let [weight (/ (- (Math/log c) (Math/log min-count))
                                 (- (Math/log max-count) (Math/log min-count)))
                       size (+ min-size (Math/round (* weight
                                                       (- max-size min-size))))
                       color (color-fn (* weight 1.0))]
                   [:a {:href (:url tag)
                        :style (str "font-size: " size "%;" "color:" color)}
                    (:name tag)]))]
    (block nil
           [:h2 "Tags"]
            (apply html (interleave (map tag-fn tags)
                                    (repeat " ")))])))

This is 10 times less code, which is a good reduction in my opinion. Most of the code is the math to generate a weight logarithmically for each tag so they scale nicely. (all-tags-with-counts) fetches a seq of two-item pairs: the tags themselves (which are hash-maps) and a count of posts for each tag. There are two locally-defined functions in the let which generate the text color and the font size and HTML for each tag.

The vectors that look like [:h2 "Tags"] are input for Compojure's HTML-generating DSL; this would be transformed for example into <h2>Tags</h2>. (block ...) is a macro which wraps its content in HTML for the rounded borders of my layout. (Math/log ...) and friends are calls to standard Java math functions.

This whole function is less code than just the horrible boilerplate array declarations at the top of the Wordpress plugin:

$catcloud_field_data = array(
  array('name' => 'Minimum Font Size', 'option' => 'catcloud_min_font_size', 'size' => '4', 'maxlength' => '3',
       'default' => '9', 'note' => 'Used for the least frequent categories', 'validation' => '/^\d{1,3}(\.\d{1,3})?$/'),
  array('name' => 'Maximum Font Size', 'option' => 'catcloud_max_font_size', 'size' => '4', 'maxlength' => '3',
       'default' => '18', 'note' => 'Used for the most frequent categories', 'validation' => '/^\d{1,3}(\.\d{1,3})?$/'),
  array('name' => 'Font Face', 'option' => 'catcloud_font_face', 'size' => '15', 'maxlength' => '254',
       'default' => '', 'note' => 'Set an optional list of font faces', 'validation' => '/.*/'),
  array('name' => 'Font Units', 'option' => 'catcloud_font_units', 'size' => '3', 'maxlength' => '2',
       'default' => 'pt', 'note' => 'Choose one of em, pt, px or %', 'validation' => '/^(%|em|pt|px)$/'),
  array('name' => 'Color Start', 'option' => 'catcloud_color_start', 'size' => '7', 'maxlength' => '6',
       'default' => '0066CC', 'note' => 'For the least frequent categories. Use a hexadecimal RGB triplet. ie. 0066CC',
       'validation' => '/^[\dA-F]{6}$/i'),
  array('name' => 'Color End', 'option' => 'catcloud_color_end', 'size' => '7', 'maxlength' => '6',
       'default' => 'CC6600', 'note' => 'For the most frequent categories. Use a hexadecimal RGB triplet. ie. CC6600',
       'validation' => '/^[\dA-F]{6}$/i'),
  array('name' => 'Before Category', 'option' => 'catcloud_before', 'size' => '3', 'maxlength' => '20',
       'default' => '[', 'note' => 'Set the character(s) to display before category names', 'validation' => '/.*/'),
  array('name' => 'After Category', 'option' => 'catcloud_after', 'size' => '3', 'maxlength' => '20',
       'default' => ']', 'note' => 'Set the character(s) to display after category names', 'validation' => '/.*/'),
  array('name' => 'Show Top N Categories', 'option' => 'catcloud_top_n_cats', 'size' => '5', 'maxlength' => '3',
       'default' => '', 'note' => 'Show only the top N categories (where N is a number like 10 or 25 or whatever. Set to 0 or empty for no limit.',
       'validation' => '/^\d*$/'),
  array('name' => 'Excluded Categories', 'option' => 'catcloud_excluded_cats', 'size' => '15', 'maxlength' => '254',
       'default' => '', 'note' => 'A comma-separated list of category ids.',
       'validation' => '/^[\d, ]*$/'),

Ugh. As another example, here's the code that handles a POST request to add a new blog page:

(defn do-new-post []
   (let [post (add-post *params*)]
     (sync-tags post (:all-tags *params*))
     (redirect-to "/"))))

It does exactly what it says: Check to make sure the user is logged in, add the post based on the POST params, sync up the tags for that post and redirect to the front page. Lisp lets you say what you want very concisely, with a bare minimum of boilerplate.

How about speed? My Clojure code is actually generating HTML in the most brute-force and wasteful way possible. The HTML for each page is regenerated from scratch, via a cascade of a couple dozen function and macro calls, every time you load a page. But it's still pretty fast, a couple hundred milliseconds for most page requests. This is slightly faster than the Wordpress version of my site. If I ever have performance issues I can switch to another Clojure HTML library, like clj-html which uses the same vector-style syntax but pre-compiles the HTML.

How hard was it to set up on the server? Wordpress is pretty famous for being dirt-easy to deploy anywhere. My Clojure app by comparison was slightly more difficult, as you might expect, but it wasn't brain surgery. My server runs Debian. First I installed the JVM via apt, then I rsynced a bunch of jar's and clj files to the server, then I installed emacs and screen also via apt. Then I put two lines into an Apache config file to proxy-forward traffic to a local port where jetty would be listening. I started Emacs, did (require ', did ( to start everything, and that's about it. Took about 15 minutes to set up from scratch. When I find a bug, I SSH in, re-attach to screen, fix it in Emacs, hit C-c C-c to recompile just the functions I need to update, and then detach from screen again.

I'm pretty pleased with this so far. It was fun to write and has all the features I used from Wordpress, plus more, and the building blocks are there to extend things if I imagine up a new feature I like.

Looks like my blog is still running today in spite of my predictions. Still waiting for the JVM to crash though, I know it's coming. I plan to post the source code for some of this once I'm sure it works.

March 16, 2009 @ 11:50 AM PDT
Cateogory: Programming


Quoth Nicole on March 16, 2009 @ 2:40 PM PDT

inb4 reddit <3

Quoth shaunxcode on March 16, 2009 @ 3:15 PM PDT

Huge fan of lisp myself BUT if you want to play code golf:

class TagCloud{
  function __construct(){
    $this->tags = getActiveRecord('tags')->findAllWithCount()->sortBy('name');
    $this->max_count = minField($this->tags, 'count');
    $this->min_count = maxField($this->tags, 'count');
    $this->min_size = 90.0;
    $this->max_size = 200.0;
    foreach($this->tags as &$tag) $tag = $this->tag($tag);
    return Array(Array('h2', 'with' => 'Tags'),
      Array('div.tag-cloud', 'with' => $this->tags));
  function color($val){
    $b = min(255 - round($val * 255), $this->max_size);
    return "rgb($b, $b, $b)";
  function tag($tag){
    $weight = (log($tag['count']) - log($this->min_count)) /
              (log($this->max_count) - log($this->min_count));
    $size = $this->min_size+round($weight * ($this->max_size-$this->min_size));
    $color = $this->color($wieght * 1.0);
    return Array('a', 'href' => $tag['url'],
      'style' => "font-size:{$size}%; color:{$color};", 'with' => $tag['name']);
Quoth mister on March 16, 2009 @ 3:30 PM PDT

Just wondering what the problem with unit testing is.

Quoth Brian on March 16, 2009 @ 3:41 PM PDT

That's cool, pretty close. The syntax is painful and it'd get much worse very quickly when you start translating deeply nested

[:html [:head [:title "Title"]] [:body [:div [:div [:div ...]]]]]

sorts of things into


It'd be interesting to see if someone could really write a CL-WHO sort of library in PHP, there's a lot more to it than the simple example I have there, as you probably know. Not sure how you'd handle the other macros I'm using either, but it might be doable in PHP5, if much more verbose. Code golf wasn't really my goal though.

Quoth Brian on March 16, 2009 @ 3:49 PM PDT

@mister: No problem with unit testing, but say I like RSpec, then I have to jump through a bunch of hoops to use it rather than the default testing framework. (Which I've done, but it's rather clumsy.)

Or say I want to use testing in some way other than full-blown by-the-book TDD. Rails spews out a test for everything under the sun by default, I don't want or need all of those most of the time.

Rails very strongly encourages you do everything a certain way, which has its benefits, but flexibility is good too sometimes, so I can do things my own way if I want.

Quoth shaunxcode on March 16, 2009 @ 5:05 PM PDT

@Brian: Actually I have come up with a syntax I prefer even beyond cl-who. It is essentially an s-expression but w/o the out the first/last paren and allowing a key: value approach. The other thing is - I am entirely "over" markup and thus a dsl which still largely looks like markup doesn't do a lot for me. I'd rather think in terms of an abstraction of what I am trying to accomplish and then allow it to be rendered for different clients (that could be jquery-ui, hackish html circa 97 tables, pure xhtml+css, flash/flex etc.).

As I am stuck in the world of php for a lot of contracts/work projects I have a php s-expression parser which reads key-value args so I can have views that do things like:

  north: $north_content
  west: (tree with: ((link href: "/leafa" with: "leaf a")
                     (link href: "/leafb" with: "leaf b"))
  center: (block with: ((heading level: 2 with: "Active Node")
                        (block with: $active_content))
  south: $south_content

It is surprisingly easy to parse and can be done with a single line of php.

Quoth Faggins on March 16, 2009 @ 5:57 PM PDT

A couple hundred milliseconds? I barfed. You can do much better.

Quoth Brian on March 16, 2009 @ 6:36 PM PDT

That includes network latency. It's 20-40 milliseconds locally. Can a human being really tell the difference at that point anyways?

Quoth Bob on March 16, 2009 @ 7:34 PM PDT

20-40 ms locally? WTF? Do you reconnect to the database on every request?

Quoth Gabe on March 16, 2009 @ 7:38 PM PDT

I'm a big fan of CodeIgniter, which is a similar REST system for PHP. I did a quick and dirty tag cloud a while back. Here's how it went.

db->get->('posts')->result_array(); $all_tags = array(); foreach ($posts as $post) {$all_tags[] = $post['tag_name'];} $unique_tags = array_unique($all_tags); foreach ($unique_tags as $tag) { $this->db->where('tag_name', $tag)->from('posts'); $unique_tag_count = $this->db->count_all_results(); $total_tag_count = count($all_tags); $ratio = $unique_tag_count / $total_tag_count ; $font_factor = round(6*$ratio); $font_size = 10 + $font_factor; echo "" . $tag . ""; } ?>
Anonymous Cow
Quoth Anonymous Cow on March 16, 2009 @ 8:03 PM PDT

Dude, beware! Markdown can create dangerous HTML:

[XSS pony](javascript:void(document.getElementById('comments'\).innerHTML='<object width="425" height="344"><param name="movie" value=""></param><param name="allowFullScreen" value="true"></param><param name="autoPlay" value="true"></param><embed src="" type="application/x-shockwave-flash" allowfullscreen="true" width="425" height="344"></embed></object>'\))
Quoth wolf550e on March 16, 2009 @ 11:04 PM PDT

Can a human being really tell the difference at that point anyways?<<

Can your blog survive /. (or nowadays I suppose reddit)? 200ms per hit, 5 hits per second, say 4 cores in that box, so 20 hits/second? For real? Also, because of the "Expires: Thu, 01 Jan 1970 00:00:00 GMT" in the HTTP headers, browser and proxy caches won't help you.

Quoth Troels on March 17, 2009 @ 12:30 AM PDT

It's not lisp, but you can use magic methods to build dsl's in php (With some limits, admitted):

class Tag {
  protected $name;
  protected $children;
  function __construct($name, $children) {
    $this->name = $name;
    $this->children = $children;
  function __toString() {
    $str = "<" . htmlspecialchars($this->name) . ">";
    foreach ($this->children as $tag) {
      if ($tag instanceOf Tag) {
        $str .= $tag->__toString();
      } else {
        $str .= htmlspecialchars($tag);
    $str .= "</" . htmlspecialchars($this->name) . ">";
    return $str;

class Builder {
  function __call($name, $args) {
    return new Tag($name, $args);

$x = new Builder();
            "inna div")))));


<html><head><title>title</title></head><body><div><div><div>inna div</div></div></div></body></html>
Quoth Nolan on March 17, 2009 @ 12:34 AM PDT

This looks awesome. It's always tough to come by real-world examples of Clojure, in my experience.

What kind of load do you see as far as memory usage, CPU usage, etc?

Baishampayan Ghose
Quoth Baishampayan Ghose on March 17, 2009 @ 1:50 AM PDT

Impressive stuff. Any chance of open-sourcing the code? I am a Clojure newbie and some working code would help a lot :)

Stuart Halloway
Quoth Stuart Halloway on March 17, 2009 @ 4:08 AM PDT

Very cool! You might simplify tag-cloud a little:

(apply html (interpose " " (map tag-fn tags)))

Quoth Brian on March 17, 2009 @ 4:13 AM PDT

@Bob: What part of "I'm generating HTML in the most brute-force and wasteful way possible" don't you understand? :)

@Anonymous: Thanks, I'll have to be more rigorous in escaping links. Maybe I should look over showdown more closely in general.

@wolf550e: An Expires header is set properly on images and whatnot, just the HTML is reloaded. It survived a small reddit load but I doubt it'd survive Slashdot, no.

@Nolan: The JVM consistently eats a ton of RAM (around 200 MB) which is a pain, but it's pretty steady there . CPU load is low.

@Baishampayan: Yeah I'm going to post some source code once I work out the bugs.

Quoth Brian on March 17, 2009 @ 4:13 AM PDT

@Stuart: Thanks, I figured there must be a built-in function to do that.

PHP Programmer
Quoth PHP Programmer on March 17, 2009 @ 4:49 AM PDT

Sound to me like you guys are a bunch of morons who do not have any idea how to put php to use. It is disgustedly sickening seeing garbage blogs like this.

Quoth Brian on March 17, 2009 @ 4:52 AM PDT

@PHP Programmer: Thanks for sharing. :)

Quoth "GRUNNUR" on March 17, 2009 @ 5:32 AM PDT

@PHP Programmer: Yo dawg, the balls nasty PHP here ain't Brian's bad, it from WordPress. You so gangsta, you go talk on them, fo shizzle.

Quoth ifatree on March 17, 2009 @ 8:52 AM PDT

@"GRUNNUR" - sup dawg. check your math. the author wasn't even complaining about PHP, he was complaining about TinyMCE (a JS product) and blaming PHP for it. Then he compares a web framework to a programming language. Then he goes on to cherry pick a single WP plugin that he's redone to be "better" -- better in that it has no configuration, no internationalization, no features at all! lol. you can't compare a plugin to a function by code size and have it mean anything sensible, people. I don't care if he's bashing PHP or Eiffel or w/e, he's clearly doing it wrong...

Quoth Brian on March 17, 2009 @ 9:23 AM PDT

@ifatree: Nope, I never used the WYSIWYG editor in Wordpress. I'm talking about all the regex replacements that are run on your text after it's posted, e.g. in wp-includes/formatting.php.

Did you read the part where I said Wordpress is longer partly because it includes features I didn't need and didn't want to implement? And then you say I'm "doing it wrong" because the PHP catcloud plugin has features that I didn't implement?

My post clearly is comparing something I wrote using Compojure in Clojure (a web framework in a language), to Wordpress which is written in PHP (a web framework in a language). The post's title is hyperbole, and I wasn't literally perfect with my word choice in a few areas, yes. I think the meaning comes across. It continues to amaze me how great programmers are for nit-picking and over-analyzing things that are as inherently fuzzy as spoken language.

Baishampayan Ghose
Quoth Baishampayan Ghose on March 17, 2009 @ 10:08 PM PDT

Brian: Why waste your time convincing all these fools?

Quoth Brian on March 18, 2009 @ 3:00 AM PDT

I don't mind people disagreeing. I hardly know what I'm talking about half the time anyways. :)

Stuart Miller
Quoth Stuart Miller on March 23, 2009 @ 4:32 AM PDT

Geeze louise Brian, you should've programmed your personal project in the language I prefer using my guidelines. Now you should just gtfo the internet.

In all seriousness, this is really impressive. Congrats.

Quoth rzezeski on March 23, 2009 @ 8:31 AM PDT

Any news when you might let this out in the wild? I'm in the process of starting a blog and I would love to host it on a Clojure solution! It doesn't have to be perfect, and I don't mind bleeding edge.

Quoth Brian on March 23, 2009 @ 9:18 AM PDT

I'll see if I can get it ready for upload this week. A lot of it is hard-coded for my site and probably unusable but it might give someone some ideas.

Quoth rzezeski on March 24, 2009 @ 6:41 AM PDT

Well, maybe I could have a look at it and try to deploy a basic blog on it? That could at least be a good indicator of where it needs some work.

The only two blog solutions I've seen so far that I "like" are Wordpress and Blojsom. I don't want to learn PHP, so that leaves Blojsom. I like that Blojsom is Java based, but Clojure would be better. Plus, I'm pretty sure I just want a lightweight solution. Probably very few people will read my blog, so performance isn't a concern at this point.

I understand if you're busy, just wanted to let you know there is interest.

Quoth Brian on March 24, 2009 @ 7:23 AM PDT

OK, I'll try to upload some code tonight. Note that Java isn't very lightweight when it comes to running it on your server; the JVM is a RAM-hog. But it's lightweight in terms of the amount of Clojure code it takes to run it and add new features.

Quoth rzezeski on March 24, 2009 @ 7:37 AM PDT

Yea, by "lightweight" I was referring to code size and feature set.

I am fully aware of Java's memory footprint. I'm much more comfortable deploying a WAR than I ever would be deploying PHP. It's not hard, but I don't know PHP and don't really want to.

Why are the majority of the blog solutions based on PHP? It's kind of surprising, not the abundance of PHP implementations, but the lack of counter offerings. Maybe I didn't look hard enough?

Quoth Brian on March 24, 2009 @ 8:07 AM PDT

Not sure, probably one reason is that it's included on pretty much every server in the world. Everyone who's using a $5/year host can throw Wordpress on there in a few minutes, no shell access required, little knowledge required, just dump some files somewhere. It also interfaces well with databases ("well" meaning quick and dirty but it works out of the box). PHP is a very simple language, which means it's not very powerful, but that also means it's learnable by non-programmers, so pretty much everyone knows PHP nowadays. PHP is the lowest common denominator.

There's Ruby on Rails, and Django for Python. And a ton of Perl libraries, but I think PHP out-perled Perl when it comes to hacked-together web scripts. There's a blog framework for almost any language, if you look closely enough. Blog engines are the "hello world" of programming languages when it comes to web programming.

Quoth Brian on March 24, 2009 @ 6:09 PM PDT

FYI I started working on posting some code tonight, realized I was running a pretty old clojure-contrib, updated clojure.contrib.sql on my machine and that broke some stuff, so I'm still fixing it up.

Quoth rzezeski on March 25, 2009 @ 12:34 AM PDT

Cool, are you on the latest Clojure as well? At least, post lazy seq?

Quoth Brian on March 25, 2009 @ 3:50 AM PDT

Yeah I'm on bleeding edge Clojure, but not bleeding edge Compojure. James just refactored Compojure to use "ring" and a ton of things changed; I'm still using the pre-ring "stable" version. These things are still moving targets. :(

Quoth rzezeski on March 25, 2009 @ 3:58 AM PDT

Yea, I was about to mention the Compojure changes too, but I figured you were pre-ring. Truthfully, I know very little Compojure; I mainly just follow the mailing list. I could act as a good guinea pig though, and it would finally give me a reason to spend some time with Compojure.

If you want you can contact me at my email (you should have it I guess, I entered it in one of my earlier comments) and let me know if/when you might have something available to look at. Unless, you don't mind blowing-up your blog comments. Also, are you on #clojure at all?

Quoth Brian on March 25, 2009 @ 4:08 AM PDT

I don't mind discussing it here, I know others are interested. I am in #clojure once in a blue moon, nick "briancarper".