Out of memory... ouch

I've written before about how I'm running four Clojure-driven websites out of a single JVM on my VPS. No problems for many months, but today I tried to make a blog post and got all kinds of out-of-memory errors. Hopefully I didn't lose any / many user comments on this blog in the past couple days, but it's possible.

I restarted the JVM and gave it a bit more RAM to play with, I imagine this will fix things. But we'll see. It occurs to me now that there may be such a thing as too much caching.

This post is related to Deploying Clojure websites
January 20, 2010 @ 4:12 AM PST
Cateogory: Programming

Virtualbox looking good

I blathered on a bit about VMWare a while back, and lots of people recommended VirtualBox. I'm trying VirtualBox 3.1.2 with Windows 7 host and Linux guest, and it works surprisingly well. I've used it successfully to hack on a bunch of projects while I'm stuck on a Windows laptop (shudder).

Installing was essentially self-explanatory. I never read any docs, except when it came to installing those "Guest Addition" programs to allow better mouse-handling. And I had to look up how to go about sharing folders between host and guest. But the documentation was clear, I found a short description how to set up the share and then and a command to run to mount the host folder:

sudo mount -t vboxsf ShareName /mnt/mountpoint

I'm using Gnome in my guest, because I haven't used it in a long time and I was curious what'd changed. I'm amazed even Compiz works in the guest. I recall a time in the very recent past when my computer couldn't even handle Compiz natively, let alone in a virtual machine.

Perhaps the best part about VirtualBox compared to VMWare is that there is one product called "VirtualBox" and one download link that took me a matter of seconds to find. Fancy the thought.

This post is related to VMware: What's in a name?
January 20, 2010 @ 4:08 AM PST
Cateogory: Linux

2009 in review

2009 sucked because I was living in a different country than my wife, thanks to months of Canadian immigration paperwork and bureaucracy. This situation is going to be changing in the immediate future, which means 2010 will not suck.

I did have a lot of time to learn things, which is good. I got all kinds of things accomplished at work, learned some supervisory skills (shudder), wrote some code that was put to good use etc. My websites grew in popularity slightly. I learned Clojure and had lots of fun banging out a few apps. I tried to learn Haskell and failed. I feel like I advanced in origami a bit. I inched ahead slightly learning Japanese. I figure in another 50 years I'll know Japanese enough to say "Hello, I know Japanese but I'm too old to use it for anything now".

I read a gratuitous amount of books. I got into Asimov for the first time; usually I dislike sci-fi, but his stuff is good. I found Neal Stephenson, and wish I'd have found him earlier. I read more programming books than I can remember. I found some interesting books on psychopathy and other psychology-related topics, and read plenty of Richard Dawkins and other sciency and atheismy books.

There just isn't enough time in the day to learn everything I want to learn. I come home from a day of writing code all day at work, goof off on the internet a bit, talk to my wife, and then I read books and write code until 3 or 4AM, and it's still not enough time.

I have apps I want to code, drawings I want to draw, origami I want to fold, video games I want to play, movies I want to watch, music I want to listen to, and the list of books I want to read keeps growing faster than I can read them, even given that I already read 4 or 5 books per month. If I had a social life, I can't imagine how little time I'd have for these things.

This year I almost want to slam the brakes on, spend a lot of time with my wife, and let my brain settle. I will definitely do that to some degree, but I can't stop learning in the meantime. I'm running out of years. 29 years old, only four or five good decades left, if I'm lucky, and my brain will be deteriorating the whole time. At least I have plenty to keep me busy.

January 04, 2010 @ 1:18 PM PST
Cateogory: Rants

Deploying Clojure websites

On my server I'm running one Java process, which handles four of my websites on four different domains. These are all running on Clojure + Compojure. Some people asked for details of how to do this, so here's a rough outline. For the sake of brevity I'm only going to talk about two domains here, though it scales up to however many you want pretty easily.

This is surely not the only way to do this, and probably not the best way, but it's what I've arrived at after a year of goofing off.

Summary: Emacs + SLIME + Clojure running in GNU Screen; all requests are handled by Apache and mod_proxy sends them to the appropriate Jetty instance / servlet.

January 04, 2010 @ 12:42 PM PST
Cateogory: Programming

Clojure and Compojure to the rescue, again

I haven't posted here much recently because I've been hacking on another recently-sort-of-completed website. One of my favorite hobbies is old 8-bit video games. The first thing I ever programmed was a website about Final Fantasy for the old NES, and I've fiddled with it for the past 10 years or so.

A while back I decided to rewrite the whole thing using Clojure + Compojure with data in mysql. This went really well. I know lines of code isn't that great a metric, but it can give a rough estimate: this whole website is done in 3,400 lines of Clojure, which includes all of the HTML "templates" and the DB layer I had to write. And it's turtles Clojure all the way down. The only thing not written in Clojure are a couple bits of Javascript here and there and the stylesheet.

I suspect the target audience of this blog and the target audience of that website don't overlap that much, but I figured someone might be interested in some of the detail of how it's implemented. A few things I learned...

January 03, 2010 @ 7:57 AM PST
Cateogory: Programming

Comments work again

I broke the ability to leave comments a couple days ago. Thanks to everyone who let me know. It's fixed now.

I broke it while uploading yet another website I finished a couple days ago. It's yet another Compojure/Clojure site, this time a bit more ambitious than my humble blog. I plan to write about that whole experience once I have a bit of time.

December 24, 2009 @ 4:20 AM PST
Cateogory: Programming

Let's parse

Is there anything more fun than parsing strings? I submit to you that there is not. I'm currently reading my way through Parsing Techniques - A Practical Guide, which has a first edition free online. (I'm hoping Santa brings me a copy of the 2nd edition this year.)

This is a good book, with enough math to be rigorous but not so much that it's completely unreadable. It starts from the absolute basics ("What's a grammar?") and goes through the Chomsky hierarchy and then dives into parsing techniques in great detail, in a language-agnostic way.

Languages and grammars are fascinating. In high school I studied Spanish, French, Latin and German, largely in my spare time. When I was 16, if people asked what I wanted to do for a living, I said "translator".

The plan to become a translator failed partly because the quality of my early education was horrendous and partly because mastering a language is extremely difficult and at 16 I wasn't motived enough. And then computers showed up in my life, which gave me a never-ending supply of languages to play with, while being fun (and profitable) in so many other ways. But I still took two years of Japanese classes in college for no reason other than enjoyment, and I'm still trying (and failing) to learn Japanese in my spare time 8 years later.

Perl was my first favorite language probably for no reason other than regular expressions. I can understand how people call PCRE syntax line-noise, but to me it's beautiful line noise. I live and breathe regular expressions nowadays. My favorite CS class in college was one where we went through and laboriously built finite-state automata and pushdown automata and Turing machines. Seeing the equivalence of these simple machines with the different classes of grammars was a huge epiphany. Such a simple concept with such huge consequences.

Dijkstra said:

Besides a mathematical inclination, an exceptionally good mastery of one's native tongue is the most vital asset of a competent programmer.

I strongly agree with that sentiment. People tell me at times that I'm good at written communication. I have my doubts, and anyways I find it funny because I'm so terrible at verbal communication. I think if I have any success at writing, it's because I view writing as a mechanical process.

I told a prof in college once that I felt like my papers wrote themselves once I had an idea in mind. There are rules of grammar and style, and you learn them and follow them, or break them deliberately if you have a good reason to. You write some prose, then you debug it until it "works" mentally. I don't care about typos and I split infinitives and comma-splice on purpose, but ambiguous or awkward phrases usually stand out to me like compiler bugs in my brain.

What's more important than language? Few things. Language is important enough to be nearly hard-wired into our brains. Children learn it instinctively. Human beings can still easily and effortlessly out-perform the best supercomputer at the task of parsing and interpreting speech. We think in words. The programming languages computers understand are dirt-simple by comparison, but writing code still feels like writing "thoughts for the computer" sometimes.

There are very few times you'll hear me say "What a wonderful world we live in". But one of those times is when I have the opportunity to explore an area of study like language. It's such an enjoyable experience to struggle and try to master such a thing. It's an amazing universe where we have these weird little rules and they work and we can understand them and manipulate them and produce things with them.

December 15, 2009 @ 12:46 PM PST
Cateogory: Programming

Clojure funding

Rich Hickey works on Clojure full-time for free, and he's asking people who get something out of Clojure to contribute some cash. $100 is less than I spend on random books and crap over the course of a year, so I gladly chipped in.

Clojure doesn't make me any money, I use it for hobby websites that actually cost me money every month just to keep going. But it's so much fun I think $100 is small price to pay, if it keeps Clojure development going.

December 14, 2009 @ 8:49 AM PST
Cateogory: Programming
Tags: Clojure

Lame comment spam management that works

It's been nine months since I ditched Wordpress and moved to a blog system I wrote from scratch (in Clojure). This was a great move in so many ways. One of those ways is comment spam. My site is as popular now (or maybe slightly more popular now) as it was when I was running Wordpress, so I think comparing before and after is valid.

With Wordpress, every morning I'd do the ritual of deleting overnight spambot droppings. Typically I got between 1 and 5 every night. I had a default Wordpress install and all I used for spam filtering was Askimet. Askimet did a surprisingly good job, catching literally if not thousands of spams every week which otherwise would've been ruining my site. But inevitably some would still get through. And what's worse, there were a lot more false positives than I could tolerate.

Since I started counting with my new system, which is around 6 months, to the best of my knowledge I've gotten zero spambot-produced comments that made it through my filters. This is pleasant, to say the least.

The system I'm using is stupid. None of it is stuff I thought of myself, I got ideas from other lots of other blogs or articles I read, but the implementation is mine and it's not sophisticated. It would take a bot author a few seconds to work around it. But no one has bothered. Why bother writing a bot for my one-man blog, when you can write a bot for Wordpress and have it work on tens of thousands of blogs? And I can change my system to defeat the bots with a few lines of code just as easily as they can work around it.

So here's why I think it's working.

December 05, 2009 @ 6:34 PM PST
Cateogory: Programming

Respect is earned

Here's my theory of respect.

Everyone deserves some level of basic respect, just for being a fellow human being. This kind of respect means that I won't step on your toes, I'll smile and nod when you talk, I'll hold the door for you. This kind of respect isn't worth much.

The important kind of respect is the one where I will listen carefully to things you say and give your words a lot of consideration and weight. When we disagree, I will sometimes give you the benefit of a doubt (where applicable) and believe your opinion over my own. I will consider you to be a good person. I will want to emulate you. I will make every effort to treat you especially well. Your opinion of me will matter to me. The dictionary uses the words "deference" and "esteem" and that's accurate.

This is the kind of respect that's worth a lot. This is the kind of respect I hope to earn from people (though I don't know how much I succeed), and the people I respect in this way are the ones I seek out in life, the people I want to be around.

December 03, 2009 @ 4:16 PM PST
Cateogory: Rants
Tags: Rants