This is a read-only archive!

Out of memory... ouch

I've written before about how I'm running four Clojure-driven websites out of a single JVM on my VPS. No problems for many months, but today I tried to make a blog post and got all kinds of out-of-memory errors. Hopefully I didn't lose any / many user comments on this blog in the past couple days, but it's possible.

I restarted the JVM and gave it a bit more RAM to play with, I imagine this will fix things. But we'll see. It occurs to me now that there may be such a thing as too much caching.

This post is related to Deploying Clojure websites
January 20, 2010 @ 4:12 AM PST
Cateogory: Programming


Quoth Brian on January 20, 2010 @ 4:12 AM PST

Hello Brian, I'm testing the comment system. Sincerely, Brian.

Andres F.
Quoth Andres F. on January 20, 2010 @ 7:39 AM PST

Too much caching? Blasphemy.

Au contraire, you probably just need more of it!

DC Jobs
Quoth DC Jobs on January 21, 2010 @ 12:21 AM PST

Until I read this post, I didn't ever realize the potential effect of caching on memory. I guess the more features a site has, the more that can go wrong.

Martin Jul
Quoth Martin Jul on January 21, 2010 @ 7:24 PM PST

For caching, you will be fine if you just remember to use weak references so that the garbage collector can reclaim the memory of the cached items when memory is short.

I have seen the same problem frequently in systems where people just stuff things into an unbounded key-value hash with hard references that never releases infrequently used entries or gives up cached instances when memory pressure is high.

Quoth Rick on February 26, 2010 @ 11:22 AM PST

Does your cache take advantage of Java's WeakReferences? That might be an out. You also may want to to try the JRockit JVM.

Anyway, you site looks clean. I really tried doing web development in Common Lisp and gave up due to the library madness (lots of libraries, but the current versions of each don't work together - and most are underdocumented at best). You stuff is almost inspiring me to give Clojure a whirl, even though I really am anti-Java these days.

Quoth Brian on February 26, 2010 @ 11:44 AM PST

No weak references, just Clojure data structures. Probably not the best idea but it's gotten me by for over a year so far. I think I've had to restart the server maybe 4 times.

Yes, I did a site in CL too and it was not a lot of fun to get running or keep running. Java sucks, but Java libraries are good. Well-documented and complete and widely tested. The Java libs you need for web development are generally wrapped in Clojure by now anyways, e.g. Compojure which wraps Jetty, so you don't get your hands dirty.

Think of Java as the assembler language for Clojure. Not sure where I picked up that phrase, but it's a great analogy. No one likes Java, but it's there if you need it for speed.