Deploying Clojure websites

* This page is related to "Clojure and Compojure to the rescue, again".

On my server I'm running one Java process, which handles four of my websites on four different domains. These are all running on Clojure + Compojure. Some people asked for details of how to do this, so here's a rough outline. For the sake of brevity I'm only going to talk about two domains here, though it scales up to however many you want pretty easily.

This is surely not the only way to do this, and probably not the best way, but it's what I've arrived at after a year of goofing off.

Summary: Emacs + SLIME + Clojure running in GNU Screen; all requests are handled by Apache and mod_proxy sends them to the appropriate Jetty instance / servlet.

Emacs Clojure colors

In yet another step along the path of trying to forcibly morph Emacs into Vim, I started a port of my Vim color scheme Gentooish to an Emacs color-theme.

Then I threw a few lines at the bottom of .emacs to highlight a few more Clojure-specific things when in clojure-mode. Result:

/emacs/gentooish-clojure.png

This is inspired partly by parenface.el which dims parens so they don't stand out as much. Lispers have this silly meme where they pretend they can't see parens at all, but being a mere human I want Emacs to help blend them into the background a bit.

I want to dim parens but also make braces and brackets pop out a bit more. Coloring braces, brackets and parens slightly differently helps make a few things easier to read in my opinion, especially in ))]}])) kinds of situations (like at the end of the function above).

This is pretty brute-force and I really wish I could figure out how to highlight #{} (sets) differently than {} (hash-maps). Highlighting (), '(), `(), and #() differently would be awesome too. But that is well beyond my elisp skills at the present. Certain areas of Emacs are all but impenetrable, no matter how much documentation I read and how much banging on the keyboard I do. The morass that is font-lock is one of those areas.

p.s. I find an easy way to keep your dotfiles in git is to make a folder (called dotfiles or something) and symlink your dotfiles from there to your home directory. I also put * in .gitignore so git doesn't slurp up anything sensitive by accident.

searchpairpos

How did I live this long without knowing about searchpairpos() in Vim? I hate when I write a clumsy, slow reimplementation of something that already exists as a standard function.

The only bad thing about Vim and Emacs both is that the feature list is about a mile and a half long (and that's a bad thing only in the sense of being an overwhelming amount of good things).

I have read almost the entire Vim manual at this point but there are corners that remain unexplored, and sometimes they contain treasure. One thing I love doing is answering Vim questions on Stack Overflow because most of the time I don't know the answer right off the bat, and so looking it up or figuring it out teaches me something new.

Emacs is another story... Emacs remains a mystery to me in many ways, in spite of having used it for about a year now. I very much plan to read the whole Emacs manual. I've already read parts of it but I seem to have barely made a dent. There are things I know should be simple to do or that there are already built-in options for, but I don't know how to do them.

  • How do I kill a word and also kill the whitespace immediately after it so it yanks properly later?
  • When I kill-whole-line and paste that line elesewhere, I lose a newline and screw up indenting. Sometimes it works how I expect and sometimes it doesn't.
  • There are so many things I can do in Vim but can't in Emacs... marks, multiple registers, abbreviations, sensibly configured per-filetype indentation.

etc. etc. I know there are ways to do these things once I have time to just sit down and read the darned manual. And learn elisp's syntax and semantics (which can't be harder than learning Vim script). My ~/.vimrc is currently twice as long as my ~/.emacs, which says a lot.

On a related note, I'm in the process of putting my Vim and Emacs configs on github.

Vim and Emacs modelines

Both Vim and Emacs have "windows", resizeable panes inside the main app "frame". Windows are a very useful feature (aside from the ill-chosen name; should've been "panes" or something). I find windows superior to tabs in my opinion; you can easily view more than one file at a time, view two files side-by-side for comparison, and such. Both Vim and Emacs use the border under the window to show helpful information like the filename and editing mode such; this is called the "status line" or "modeline". Again, very handy.

How do you set a nice custom modeline in Vim? Here's my config from my .vimrc:

set statusline=%f\ %2*%m\ %1*%h%r%=[%{&encoding}\ %{&fileformat}\ %{strlen(&ft)?&ft:'none'}\ %{getfperm(@%)}]\ 0x%B\ %12.(%c:%l/%L%)

Yikes! What a mess. It's just a big string with a bunch of special codes and literal characters all mixed up. If you do :h 'statusline' you can read about all the options. That string is ugly though.

Or so I thought, until I witnessed the trainwreck that is Emacs. The documentation for Emacs' modeline comprises a small encyclopedia. The configuration itself is spread across a couple dozen configuration variables. This is the default value for just one of them:

(#("%[" 0 2
   (help-echo "Recursive edit, type C-M-c to get out"))
 #("(" 0 1
   (help-echo "mouse-1: Select (drag to resize)\nmouse-2: Make current window occupy the whole frame\nmouse-3: Remove current window from display"))
 (:propertize
  ("" mode-name)
  help-echo "Major mode\nmouse-1: Display major mode menu\nmouse-2: Show help for major mode\nmouse-3: Toggle minor modes" mouse-face mode-line-highlight local-map
  (keymap
   (mode-line keymap
          (down-mouse-3 keymap
                (abbrev-mode menu-item "Abbrev (Abbrev)" abbrev-mode :help "Automatically expand abbreviations" :button
                     (:toggle . abbrev-mode))
                (auto-fill-mode menu-item "Auto fill (Fill)" auto-fill-mode :help "Automatically insert new lines" :button
                        (:toggle . auto-fill-function))
                (auto-revert-mode menu-item "Auto revert (ARev)" auto-revert-mode :help "Revert the buffer when the file on disk changes" :button
                          (:toggle bound-and-true-p auto-revert-mode))
                (auto-revert-tail-mode menu-item "Auto revert tail (Tail)" auto-revert-tail-mode :help "Revert the tail of the buffer when buffer grows" :enable
                           (buffer-file-name)
                           :button
                           (:toggle bound-and-true-p auto-revert-tail-mode))
                (flyspell-mode menu-item "Flyspell (Fly)" flyspell-mode :help "Spell checking on the fly" :button
                       (:toggle bound-and-true-p flyspell-mode))
                (font-lock-mode menu-item "Font Lock" font-lock-mode :help "Syntax coloring" :button
                        (:toggle . font-lock-mode))
                (glasses-mode menu-item "Glasses (o^o)" glasses-mode :help "Insert virtual separators to make long identifiers easy to read" :button
                      (:toggle bound-and-true-p glasses-mode))
                (hide-ifdef-mode menu-item "Hide ifdef (Ifdef)" hide-ifdef-mode :help "Show/Hide code within #ifdef constructs" :button
                         (:toggle bound-and-true-p hide-ifdef-mode))
                (highlight-changes-mode menu-item "Highlight changes (Chg)" highlight-changes-mode :help "Show changes in the buffer in a distinctive color" :button
                            (:toggle bound-and-true-p highlight-changes-mode))
                (outline-minor-mode menu-item "Outline (Outl)" outline-minor-mode :help "" :button
                        (:toggle bound-and-true-p outline-minor-mode))
                (overwrite-mode menu-item "Overwrite (Ovwrt)" overwrite-mode :help "Overwrite mode: typed characters replace existing text" :button
                        (:toggle . overwrite-mode))
                "Minor Modes")
          (mouse-2 . describe-mode)
          (down-mouse-1 menu-item "Menu Bar" ignore :filter
                (lambda
                  (_)
                  (mouse-menu-major-mode-map))))))
 ("" mode-line-process)
 (:propertize
  ("" minor-mode-alist)
  mouse-face mode-line-highlight help-echo "Minor mode\nmouse-1: Display minor mode menu\nmouse-2: Show help for minor mode\nmouse-3: Toggle minor modes" local-map
  (keymap
   (header-line keymap
        (down-mouse-3 keymap
                  (abbrev-mode menu-item "Abbrev (Abbrev)" abbrev-mode :help "Automatically expand abbreviations" :button
                       (:toggle . abbrev-mode))
                  (auto-fill-mode menu-item "Auto fill (Fill)" auto-fill-mode :help "Automatically insert new lines" :button
                          (:toggle . auto-fill-function))
                  (auto-revert-mode menu-item "Auto revert (ARev)" auto-revert-mode :help "Revert the buffer when the file on disk changes" :button
                        (:toggle bound-and-true-p auto-revert-mode))
                  (auto-revert-tail-mode menu-item "Auto revert tail (Tail)" auto-revert-tail-mode :help "Revert the tail of the buffer when buffer grows" :enable
                             (buffer-file-name)
                             :button
                             (:toggle bound-and-true-p auto-revert-tail-mode))
                  (flyspell-mode menu-item "Flyspell (Fly)" flyspell-mode :help "Spell checking on the fly" :button
                         (:toggle bound-and-true-p flyspell-mode))
                  (font-lock-mode menu-item "Font Lock" font-lock-mode :help "Syntax coloring" :button
                          (:toggle . font-lock-mode))
                  (glasses-mode menu-item "Glasses (o^o)" glasses-mode :help "Insert virtual separators to make long identifiers easy to read" :button
                        (:toggle bound-and-true-p glasses-mode))
                  (hide-ifdef-mode menu-item "Hide ifdef (Ifdef)" hide-ifdef-mode :help "Show/Hide code within #ifdef constructs" :button
                           (:toggle bound-and-true-p hide-ifdef-mode))
                  (highlight-changes-mode menu-item "Highlight changes (Chg)" highlight-changes-mode :help "Show changes in the buffer in a distinctive color" :button
                              (:toggle bound-and-true-p highlight-changes-mode))
                  (outline-minor-mode menu-item "Outline (Outl)" outline-minor-mode :help "" :button
                          (:toggle bound-and-true-p outline-minor-mode))
                  (overwrite-mode menu-item "Overwrite (Ovwrt)" overwrite-mode :help "Overwrite mode: typed characters replace existing text" :button
                          (:toggle . overwrite-mode))
                  "Minor Modes"))
   (mode-line keymap
          (down-mouse-3 keymap
                (abbrev-mode menu-item "Abbrev (Abbrev)" abbrev-mode :help "Automatically expand abbreviations" :button
                     (:toggle . abbrev-mode))
                (auto-fill-mode menu-item "Auto fill (Fill)" auto-fill-mode :help "Automatically insert new lines" :button
                        (:toggle . auto-fill-function))
                (auto-revert-mode menu-item "Auto revert (ARev)" auto-revert-mode :help "Revert the buffer when the file on disk changes" :button
                          (:toggle bound-and-true-p auto-revert-mode))
                (auto-revert-tail-mode menu-item "Auto revert tail (Tail)" auto-revert-tail-mode :help "Revert the tail of the buffer when buffer grows" :enable
                           (buffer-file-name)
                           :button
                           (:toggle bound-and-true-p auto-revert-tail-mode))
                (flyspell-mode menu-item "Flyspell (Fly)" flyspell-mode :help "Spell checking on the fly" :button
                       (:toggle bound-and-true-p flyspell-mode))
                (font-lock-mode menu-item "Font Lock" font-lock-mode :help "Syntax coloring" :button
                        (:toggle . font-lock-mode))
                (glasses-mode menu-item "Glasses (o^o)" glasses-mode :help "Insert virtual separators to make long identifiers easy to read" :button
                      (:toggle bound-and-true-p glasses-mode))
                (hide-ifdef-mode menu-item "Hide ifdef (Ifdef)" hide-ifdef-mode :help "Show/Hide code within #ifdef constructs" :button
                         (:toggle bound-and-true-p hide-ifdef-mode))
                (highlight-changes-mode menu-item "Highlight changes (Chg)" highlight-changes-mode :help "Show changes in the buffer in a distinctive color" :button
                            (:toggle bound-and-true-p highlight-changes-mode))
                (outline-minor-mode menu-item "Outline (Outl)" outline-minor-mode :help "" :button
                        (:toggle bound-and-true-p outline-minor-mode))
                (overwrite-mode menu-item "Overwrite (Ovwrt)" overwrite-mode :help "Overwrite mode: typed characters replace existing text" :button
                        (:toggle . overwrite-mode))
                "Minor Modes")
          (mouse-2 . mode-line-minor-mode-help)
          (down-mouse-1 . mouse-minor-mode-menu))))
 #("%n" 0 2
   (local-map
    (keymap
     (mode-line keymap
        (mouse-2 . mode-line-widen)))
    mouse-face mode-line-highlight help-echo "mouse-2: Remove narrowing from the current buffer"))
 #(")" 0 1
   (help-echo "mouse-1: Select (drag to resize)\nmouse-2: Make current window occupy the whole frame\nmouse-3: Remove current window from display"))
 #("%]" 0 2
   (help-echo "Recursive edit, type C-M-c to get out"))
 #("--" 0 2
   (help-echo "mouse-1: Select (drag to resize)\nmouse-2: Make current window occupy the whole frame\nmouse-3: Remove current window from display")))

What the hell?

Instead of a string, it's an s-expression, which is better, right? Well no, it's still just a big construct with arbitrary meanings assigned to its contents. Lists mean one thing, strings mean another thing (and those strings, like Vim's, can contain special escape sequences). Symbols mean something else, symbols that are keywords mean something else, numbers mean something else, and so on.

To decipher this I had to learn this mini-language. And also learn about "text properties" and a bunch of elisp stuff. It also required knowledge about a bunch of minor modes and how they tie into the modeline, all of which is essentially a bowl of spaghetti code. And keymaps, and maps to control mouse click events and such. Eventually I figured out that most of that crap is controlling tooltip text.

If you do M-x customize-apropos in Emacs and search for "mode-line", you'll get a helpful list of all of the configuration values and their values. (The default values contain literal tab characters, which you can't even type into the customize text fields without C-qing, because tab jumps you between fields. Ughhhhhhh.)

I gave up even trying to get Emacs to have all the helpful information my Vim modeline has. Even deleting the default crap to pare this down to something readable took some effort. This what I ended up with:

(mode-line-format (quote ("%e--[" mode-line-buffer-identification "]" (vc-mode vc-mode) "  " mode-line-modes global-mode-string " %-")))
(mode-line-in-non-selected-windows t)
(mode-line-modes (quote ("%[" "(" (:propertize ("" mode-name)) ("" mode-line-process) (:propertize ("" minor-mode-alist)) "%n" ")" "%]")))
(mode-line ((((class color) (min-colors 88)) (:background "#333333" :foreground "#bcbcbc" :box (:line-width -1 :color "#333333")))))
(mode-line-highlight ((((class color) (min-colors 88)) nil)))
(mode-line-inactive ((default (:inherit mode-line)) (((class color) (min-colors 88) (background dark)) (:foreground "#8b8b8b" :weight light))))

What does that mean? Don't ask me, I can no longer read it. If customize hadn't produced a lot of that for me, I probably wouldn't have managed. My favorite part is the four-deep nested list of lists of lists of lists for the colors.

Verdict?

  1. Vim's modeline is less powerful than Emacs. But who cares if you can't even read it to edit it? Most of Emacs' modeline features are annoying. (My motivation for editing this to begin with was to turn off all the mouse buttons and mini-menus and crap.)

    Vim's status line is exactly configurable enough. I don't want to build a small GUI app in my modeline. I want it to show certain bits of information about the buffer, that's it. Vim does this.

  2. Vim script is less elegant than elisp. Or is it? Vim's modeline is a custom DSL for formatting modelines. It's hard to think of anything more concise. Concision is a very good thing. Emacs' version is more general, at the expense of horrid verbosity and unreadability.
  3. Vim's modeline is a string, which means you either write it literally, or you construct it by concatenating lots of other strings. This is a faux pas, right? It's like using eval in Ruby or Perl. It's fragile and error-prone. Emacs uses a Lisp, with its macros and quoted lists and code is data and so on.

    But who cares? In this case, a simple string is powerful enough. I don't need a whole Turing-complete programming language to configure a modeline. It's massive overkill and you pay a price for it. The minute a human being is supposed to be keeping track that the second element in the 5-deep nested list means "x", something has gone horribly wrong.

Vim wins this round.

A Sad, Dark Day

Today was a terrible day. I found myself subconsciously trying to use Emacs keystrokes in Vim. I feel dirty. I took a bath but it won't come clean. : (

It just goes to show that you can get used to anything if you do it often enough. Emacs still drives me up the wall but maybe I've achieved a critical mass of enough custom keybindings to let me tolerate it.

Aside from paredit, which has no equal even in Vim, Emacs does have some vaguely non-sucky features. hi-lock is pretty nice (Vim has an equivalent of course). Once I learned a few of the shortcuts for git-emacs I actually found myself using Git much more effectively. Having to drop into a shell to type Git commands is just enough of a disruption to prevent me from doing it often enough. I never got the hang of any version control library in Vim.

I'm almost even getting used to the Emacs buffer model. I find myself C-x bing and flipping back and forth between buffers by name, rather than my Vim practice of opening buffers in certain carefully-placed windows and leaving them there.

On the subject of typing, I broke down finally and ordered a Unicomp Customizer 104 keyboard. I've heard too many hackers say that the old IBM clicky keyboards are good for typing. It should arrive Tuesday, and I'm a lot more excited than anyone should be over a keyboard.

Expect a keyboard review. Try to contain your excitement until then. I know it'll be hard.

Disabling Ctrl-Alt-Backspace

After being reminded the hard way yet again that C-S-Backspace in Emacs invokes the very handy kill-whole-line function, but that C-M-Backspace, while uncomfortably similar to that key-chord, does something very different, I have now officially added to my /etc/X11/xorg.conf:

Section "ServerFlags"
    Option "DontZap" "True"
EndSection

to prevent me from accidentally murdering my X server at the worst possible times.

Juvenile?

I get a cheap thrill out of editing my ~/.emacs file in Vim.

Emacs annoyance #448,546

From the Emacs docs:

Each buffer has a default directory which is normally the same as the directory of the file visited in that buffer. When you enter a file name without a directory, the default directory is used. If you specify a directory in a relative fashion, with a name that does not start with a slash, it is interpreted with respect to the default directory. The default directory is kept in the variable default-directory, which has a separate value in every buffer.

The command M-x pwd displays the current buffer's default directory, and the command M-x cd sets it (to a value read using the minibuffer). A buffer's default directory changes only when the cd command is used. A file-visiting buffer's default directory is initialized to the directory of the file it visits. If you create a buffer with C-x b, its default directory is copied from that of the buffer that was current at the time.

This is extremely annoying. Vim leaves my working directory the hell alone, why doesn't Emacs? Vim lets you set a global working directory, and selectively (and explicitly) change it on a per-buffer basis. This is what I want.

But in Emacs, every time you open a file, the working directory automatically changes to the directory of that file. If you have multiple files open in Emacs (which of course you do), every time you move the cursor between windows, or look at a new file in your current window, your working directory just changed out from under you.

So say you open some files. Then you want to start SLIME. So you C-x 2 to split and open a new window, in which to start SLIME. Except (annoyance #448,547) Emacs doesn't open a window with a new BLANK file, as Vim sanely does via CTRL-W n; instead it puts the file you're looking at into BOTH windows, which is absolutely never what you want. As a result, the working directory of BOTH windows is the directory of whatever file you were looking at. So every time you start SLIME, you have to make sure you M-x cd back to the proper working directory first, because otherwise your Lisp process is randomly going to start in the wrong directory.

But you can change the directory of the SLIME buffer after you start SLIME. Just M-x cd or ,cd. Except if you're unlucky enough to be using SLIME for Clojure, which may have set its classpath based on its working directory when you started it. In that case you have to restart SLIME.

But if you kill SLIME, Emacs (annoyance #448,548) jams another random file into the window where SLIME just was. So your working directory just changed again! Or did it? Depends what Emacs decided to put into your window. You may think that, while SLIME is running, M-x cd and/or ,cd and then ,restart-inferior-lisp may do what you want, but you would be wrong; it always reverts back to the original working directory from when SLIME was first started.

Lost? Confused? tl;dr? Welcome to Emacs. So now I'm looking through the encyclopedic tangled mess of Emacs documentation to try to figure out how to get Emacs not to change my working directory, ever, unless I say so. This is hindered as always by (annoyance #448,549) Emacs' arcane and non-standard terminology. So far, I have committed to memory that "current directory"/"working directory" in Emacs is instead called "default directory". And you don't open files, you "visit" them. Command line? No, "minibuffer".

Vim + screen + REPL = win

Via the Clojure wiki I found a great page describing how you can use GNU screen and some Vim magic to let Vim play nicely with an interactive commandline program like a Common Lisp REPL, Ruby's irb, or Python's, well, python.

That page is a very stripped-down and simpler version of what Limp does for Vim+Lisp. But Jonathan Palardy's version has the benefit of being so simple that you can set it up yourself manually in a second or two. I still have never gotten Limp to work quite right and I don't have the time to debug a big mess of Vim script.

The idea is to start up a named screen session via e.g. screen -S foo -t bar, then start an irb session (or whatever) in there, and then in Vim you can simply yank some text into a named register and send it off to screen via a system call. Download Jonathan's code and see.

It's not a full-blown SLIME; it doesn't have tab-completion or weird interactive debugging windows or such bullcrap. It doesn't capture the output of your command and feed it back into your Vim buffer. But hey, it's pretty good for something you can throw together in 2 minutes, and it works.

So there goes my last reason to ever use Emacs. Good riddance, I must say.

Honestly, Emacs just frustrates the living hell out of me. Oh how I tried to like it. I really did. I've used it on and off constantly over the past year. I have Emacs shortcuts written all over the whiteboard in my office. But its braindead window management, its terrible broken undo/redo system, its finger-crippling key-chord combos, its lack of features I need (like line numbering), its reliance on broken 3rd-party elisp hack scripts for things Vim has built in (like line numbering!), its ugly fonts and GUI elements, and so on and so forth. Vim is such a joy in comparison.