This is a read-only archive!

PAIP; Lisp Lisp Lisp

I ordered PAIP today. Good computer books are so expensive. It hurts me to spend $80 on a book. But there are many worse ways to spend $80 than on something which contains so much knowledge, I guess. I got a $25 gift card for a book store for Christmas, so may as well put it to good use. I've read good reviews of PAIP, so I hope it lives up to them. I had only one AI course in college, and it was taught in C (ugh) by a grad student, and I learned little to nothing from it. It'll be nice to revisit the subject. (And learn more Common Lisp, of course.)

In other non-news, my Common Lisp-powered website is still up and running after a few days, no crashes or spontaneous hard drive reformats or anything, so that's good. Deploying it was interesting. Actually compared to the one Ruby on Rails site I deployed, Hunchentoot was far easier to deploy in many ways.

When you deploy Rails you have the problem that Rails is a beast of a framework, and Ruby is slow as a dog to begin with. So there are all kinds of silly nasty ugly hacks to get it to run with acceptable performance. Personally for my Rails site I have to run three Mongrel servers and use Apache's mod_proxy_balancer to send traffic to them. Some other people use fastcgi or such CGI-caching scripts, which I never could get working well. Just to run Rails apps at all, there are all kinds of Apache mod_rewrite hacks going on to get routes to work. Etc. etc. etc. It's a huge mess and I pray my Rails server never dies because I sure don't want to have to go through to pain of re-deploying it now that it's up and running.

With Hunchentoot on the other hand, you start Lisp once and it runs forever. No Apache to screw with means happy times are had by all. In fact my Hunchentoot-run website does exactly what Rails routes do, all via a single function that parses the request URI and dispatches based on it. With no .htaccess wizardry or cryptic Rails config files anywhere in sight.

Every request to Hunchentoot is apparently handled in a Lisp thread, so concurrency issues are generally all taken care of for you, no worries. In fact since you have Lisp running forever you can get away with crazy things like using Lisp objects as a sort of cache for data instead of hitting your database for every request, which I imagine probably works wonders for performance. I'll never know for sure, because my site will never ever be popular enough to put it to the test, but one can dream.

There were a few snags deploying my Hunchentoot server, none of which were the fault of Lisp, more the fault(?) of Linux. One was that I wanted to bind Hunchentoot to port 80. As you may know, only root has the right to bind a program to a "privileged" port like 80. I certainly did not want to have a Lisp image permanently running on my server as root, for security reasons. Thankfully Debian has a drop-in solution for this in the form of authbind, which lets normal users bind programs to privileged ports.

The second problem is that Common Lisp via SBCL is a sort of interactive program (see also, the REPL), and isn't really made to be run as a daemon, at least not the way I usually run it. There are many solutions to this however, and the solution I chose was detachtty, which does exactly what the name says. As the linked site describes, it's like a very stripped-down sort of GNU Screen.

Then there's the issue of how I can log into that running Lisp image from my home computer to run commands. I could easily SSH to the server, and use attachtty (which comes with detachtty) to attach to SBCL directly and type commands there. But SBCL's built-in REPL is a bit primitive. SLIME would be nicer. The solution here is to start Swank on the server. then use Slime running on my local machine to connect to the remote Swank.

This is, again, extremely simple. In a previous entry I linked to a movie that describes in great detail how this can be done, start to finish. You can see another more thorough example of how to do it here. Essentially you run SWANK:CREATE-SERVER in the remote SBCL; Swank binds only to localhost on the server, so to use it from home, you then have to make an SSH tunnel. Local Emacs can slime-connect to the remote Swank via the tunnel.

So that's it. authbind starts detachtty, which runs SBCL, which --loads a little 5-line Lisp script that imports some libraries and my photo blog source code, then starts Swank, and then starts my server. A little init script to make this happen by default every time the server reboots, and there's nothing more to be done. There's no real need to restart the server from now on, since I can log into the REPL and run commands and update the running program on-the-fly. But if I had to restart it or redeploy it entirely, it would be no problem.

This strikes me as an example of one of the great things about Linux. Many small tools that all do a single job very well. A tool to let non-root users bind port 80, a tool to let you run any program as a daemon, the glue to make those things work together with SBCL, and a great text editor to talk to SBCL after it's running. It all works beautifully, and there are no mysteries anywhere in the process start to finish.

January 30, 2008 @ 6:31 PM PST
Cateogory: Programming

18 Comments

Jeremiah
Quoth Jeremiah on January 30, 2008 @ 10:30 PM PST

So, I have this website that I want to implement - I've got most of the design work done, and I've been trying to decide whether I want to do it in RoR or CL.

The types of things the site needs to do would be easiest (for me at least) to implement in CL, minus a caveat that has kept me leaning towards using RoR:

I need to be able to interface with the OS (linux). As in I need to be able to call commands in /usr/bin (or a subset thereof, probably from within a chroot environment) and work with files and directories. I am currently unaware of a good means of doing this in CL - but then again, I'm rather new to the lisp scene anyhow.

Also, I'm rather poor at the moment, and I'll need cheap hosting for a while. I'll probably end up hosting off my own box, but I don't think my ISP is going to like that if I generate much traffic, and I'm not currently living in an area that has much in the way of ISP choice.

I'm aware that this might not be the most appropriate place to look for answers on, but do you have any good advice?

foo
Quoth foo on January 30, 2008 @ 11:13 PM PST

The origami site looks nice. Well done!

Ben
Quoth Ben on January 31, 2008 @ 1:29 AM PST

Hi Brian,

Maybe it's that I'm only 2/3 through my morning coffee, but I can't find an RSS link anywhere around here. Any hints for a guy who wants to subscribe to your posts?

Thanks, Ben

Aticus
Quoth Aticus on January 31, 2008 @ 3:26 AM PST

Jeremiah, dont take my advice very heavily, as I am too a lisp sort-of-newbie, but I have read that SBCL has a means to call foreign programs. Even if that's not true, you could always use CFFI (this time I know for sure that SBCL provides it) to call a C function that will do the work...

Jeremiah
Quoth Jeremiah on January 31, 2008 @ 3:43 AM PST

Aticus, yes some more research on my end landed me at sb-ext:run-program, which supports almost exactly what I need.

That's what I get for posting questions prematurely, I guess.

Peter Christensen
Quoth Peter Christensen on January 31, 2008 @ 3:54 AM PST

I also got myself PAIP for Christmas and from what I've read so far, it's a great book. Chapter 1 is the best explanation of Lisp I have read anywhere and Chapter 2 is the best non-trivial demonstration of Lisp programming technique and style. I'll keep an eye on your blog to see what you think of it.

I think you might have jinxed your origami site, because I can't connect to it now.

Red
Quoth Red on January 31, 2008 @ 3:56 AM PST

Jeremiah, Every Common Lisp implementation I'm aware of has functions for calling external programs. In SBCL, it's sb-ext:run-program (see http://www.sbcl.org/manual/Running-external-programs.html#Running-external-programs ). There's even trivial-shell, a Common Lisp library which abstracts the whole thing across implementations.

Of course Common Lisp has plenty of file and directory related functions. There's also CL-FAD (Files And Directories) which goes beyond that (see http://weitz.de/cl-fad/)

Brian
Quoth Brian on January 31, 2008 @ 4:52 AM PST

@Jeremiah: I second Red's recommendations. I use CL-FAD for my site for a lot of things. It wasn't hard to write a few functions to map URLs to file paths for example. And my origami blog globs directories full of photos and thumbnails to figure out what to display.

@Peter: Yeah, the whole server died last night. All the sites I host on that server went down. Great timing!

@Ben: http://briancarper.net/feed/atom/ (I should add an RSS link instead of relying people to use the Firefox RSS icon in the toolbar I guess.) If you mean an RSS feed for the origami site, I plan to add one soon.

Gustavo Barrancos
Quoth Gustavo Barrancos on February 01, 2008 @ 11:41 AM PST

Nice! I'm really facing some language dilemma right now.. Just curious about a bunch of them and little time to hack them all (Common Lisp,Haskell,Clean,Ruby). Your post gave me courage to give CL a try on Web Development. Looking forward to see more posts about it from you :)

Brian
Quoth Brian on February 01, 2008 @ 3:24 PM PST

Haskell still gives me nightmares. Good luck with CL. :)

Gustavo Barrancos
Quoth Gustavo Barrancos on February 05, 2008 @ 10:27 AM PST

Thanks! I'm looking forward to become a a daily-basis-lisper... lol. I'm using Scheme in a game project(RPG stuff) and i want to play with CL for web stuff initially. Ps: Congrats for the Origamis. The Cactuar really kicked my ass! (i'm a big fan of Final Fantasy btw ..lol)

Brian
Quoth Brian on February 05, 2008 @ 1:47 PM PST

If I can do learn Lisp well enough to make a website out of it, anyone can.

Thanks for looking at my origami. If you like FF origami, take a look at Satoshi Kamiya and/or Brian Chan. Both have made FF-related models, far better than mine.

Nikhil Prabhakar
Quoth Nikhil Prabhakar on February 07, 2008 @ 2:02 AM PST

Very good article!

Ken
Quoth Ken on March 21, 2009 @ 7:25 AM PDT

"I certainly did not want to have a Lisp image permanently running on my server as root, for security reasons."

What I did was run Hunchentoot on an unprivileged port, and then ran nginx redirecting port 80 to that. This wasn't to get around the port restriction -- that was just a nice side effect. nginx can do HTTP compression on my outgoing data stream, which saves a ton of bandwidth for me.

Brian
Quoth Brian on March 21, 2009 @ 11:56 AM PDT

I'll have to look into nginx, thanks for the info.

Steve
Quoth Steve on January 04, 2011 @ 2:09 AM PST

Nice website, Brian.

To use Clojure, is it necessary to know much about Java? I know almost nothing about it.

Steve

Brian
Quoth Brian on January 04, 2011 @ 4:20 AM PST

You can get a long way without Java, but you will probably need to learn it eventually. Same with any other language, you need to understand the platform you're running on.

Java is not hard to learn, thankfully. Probably a lot simpler to learn than Clojure itself.

Steve
Quoth Steve on January 05, 2011 @ 12:51 AM PST

Are you using the same tools mentioned above with the substitution of Clojure for SBCL?