Making an RPG in Clojure (part one of many?)

What do you get when you combine old-school Final Fantasy-style RPGs with Clojure? Fun times for all. Well, for me at least.

I'm working on a sort of RPG engine in Clojure so I can make my own RPG. Click the thumbnail for a very preliminary video demo (6.5 MB) showing the engine in action.

RPG Demo

All I do in the video is walk around, and eventually start adding random NPCs to the map to test collision detection. Not all that exciting, but I'm proud nonetheless.

To forestall questions, yes I'll eventually post the source code, but no, not yet. It barely works. Just a proof of concept so far.

Right now I can walk around a map while NPCs also randomly walk around the map, not much more. So there isn't much to talk about. But not bad for 4 days and 600 lines of code (one tenth of which is ASCII art... more on that later). Keep in mind that I have no idea what I'm doing.

Collision detection works so people don't walk through walls or each other, and after endless tweaking I got all the animations to be very smooth, even when I add a few dozen NPCs to the map (as I do in the video). The video is a bit jerky but it looks better in person. All of the admittedly poor artwork is also created by myself, thanks to the GIMP and some hastily-read tutorials on pixel art.

It all runs on plain old Swing in Clojure. Here's some of what went right and what went wrong so far.

Background

I've played a lot of RPGs, but I've never programmed a game more complex than Pong. I knew what double-buffering is, and that's as far as my knowledge of game programming went when I started.

My first idea was to use a plain old Swing JFrame. I'd make a bunch of sprites saved as PNG files, read them all in and draw them on the JFrame, as many times per second as I could manage. Then there's some global state to keep track of where everything is. Simple enough.

(PS I have no idea what I'm doing.)

Failures

My first version used Clojure agents (i.e. threads) for everything. The game logic was a thread, the renderer ran in a thread, every NPC was its own thread. The world itself was a single ref that all of these agents banged on. So the NPCs would tell the ref "I want to move down one square", another thread might say "Brian just pushed 'left' on the keyboard, so start scrolling the map". The world-ref would kindly oblige while preventing two NPCs from standing on each other or letting the PC walk through the walls.

Clojure is awesome in letting you do this in a completely safe and coordinated way. Everything worked well. But even for the crappy 2D graphics I'm using, all those threads caused way too much lag. I could get a good framerate if everyone was standing still, but if I was walking while the NPCs were walking, I'd get lots of lag. It was even worse on a slower computer.

My best guess is that the reason for the lag was the constant restarting of canceled transactions due to multiple threads trying to edit the world ref 50+ times per second. My sucky 2-core CPU couldn't keep up. It isn't surprising that this failed, in hindsight.

OpenGL?

Next, I decided to try out OpenGL. There are multiple options for OpenGL in Java. One is JOGL and another is lwjgl.

I program in Linux, and my video card is ancient. I can barely get OpenGL to work in the best of times. Installing JOGL was a slight chore (Gentoo doesn't even include it in its repo). JOGL is not just a JAR you throw onto CLASSPATH, you need some native extensions, obviously. I got it running somehow, but I wouldn't want to explain to someone else how I did it.

I did get jwjgl to work too, eventually, which was nice. There is a really nice Java 2D game framework called Slick which uses lwjgl. Some good games were created using this, for example Stickvania, which recently hit Reddit recently. I got Slick up and running in short order.

Unfortunately Slick doesn't play nicely with a Clojure REPL. I could build and start a game, but once the game is stopped, it never runs properly again without restarting the REPL. Slick caches images to try to be speedy, and it seems like the cache is either corrupted or destroyed when you close down your game, for one thing. This is not conducive to Clojure REPL-style development, and I didn't want to spend a lot of time fixing it.

Another issue is that I'd really like this game to be cross-platform and available to non-hackers, and the thought of trying to tell the average gamer how to install JOGL or lwjgl was daunting, given the bullcrap I had to go through. Not sure if I can just throw JOGL into a JAR and distribute it, maybe I can, but I didn't want to bother reading about it. Swing on the other hand runs everywhere with no effort.

My main problem is that I know even less about OpenGL programming than I do about Swing, and don't have a month to learn. Back to the drawing board.

Success

It turns out I don't need OpenGL anyways. All I need is program more intelligently. (Did I mention I have no idea what I'm doing?) Instead of dozens of threads, my current (working) version has one thread. It updates the game logic, then renders the game, then waits 10 milliseconds or so, then repeats this (forever).

With the single-threaded version, the logic is actually more complex than the multi-threaded version. Agents and refs were really nice and braindead-easy to work with. But such is life.

Once I learned about keeping track of things in realtime by counting milliseconds instead of counting frames or using timeouts to do logic/render updates, things worked better. (Did I mention I have no idea what I'm doing?) The game loop looks like this now:

(defn game-loop [#^Canvas canvas]
  (loop [last-time (get-time)]
    (let [curr-time (get-time)
          delta (- curr-time last-time)]
      (do-logic delta)
      (do-render canvas)
      (when @RUNNING
        (Thread/sleep 10)
        (recur curr-time)))))

The code to actually start the game, to give you an idea:

(defn start-world [world]
  (let [#^JFrame frame (doto (JFrame. "Game")
                         (.addWindowListener (proxy [WindowAdapter] []
                                               (windowClosing [e] (stop)))))
        #^JPanel panel (doto (.getContentPane frame)
                         (.setPreferredSize (Dimension. REAL-WIDTH REAL-HEIGHT))
                         (.setLayout nil))
        #^Canvas canvas (Canvas.)]
    (doto canvas
      (.setBounds 0 0 REAL-WIDTH REAL-HEIGHT)
      (.setIgnoreRepaint true)
      (.addKeyListener (proxy [KeyAdapter] []
                         (keyPressed [e] (handle-keypress e))))
      (.addMouseListener (proxy [MouseAdapter] []
                           (mouseClicked [e] (handle-mouse e))))
      )
    (.add panel canvas)
    (doto frame
      (.pack)
      (.setResizable false)
      (.setVisible true))
    (.createBufferStrategy canvas 2)
    (dosync (ref-set RUNNING true)
            (ref-set PAINTER (agent canvas))
            (ref-set WORLD world))
    (send-off @PAINTER game-loop)))

That's about it for the Swing side of things, aside from scribbling on the Canvas in the render function.

The agent I wrap around the Canvas controls the thread that runs the game loop. The agent helpfully keeps track of any exceptions that happen during the loop, and I can view those exceptions via agent-error, which is handy for debugging.

Note how little code it is to set up a keyboard event handler, thanks to proxy:

    (doto canvas
      ...
      (.addKeyListener (proxy [KeyAdapter] []
                         (keyPressed [e] (handle-keypress e))))

A couple lines of Clojure for what would be a lot of senseless boilerplate in Java. Notice how you the keyboard handler calls a normal Clojure function handle-keypress. Clojure / Java interop really is seamless.

Maps

My maps are made using ASCII art. (Did I mention I have no id-... never mind.) Here's the code for the test map, for example. Map here is a deftype (available in bleeding-edge Clojure), which takes a map of tiles, a "pad" tile, the map, and then a mask showing walls / tiles where the player shouldn't be allowed to walk. My Map type acts like a Clojure hash-map most of the time, but it also lets me name the fields that all maps should share, and has a proper "type", among other things.

(deftype Map [tileset pad tiles walls]
  clojure.lang.IPersistentMap)

(def test-map (cache-map
               (Map {\  (tile "ground")
                     \/ (tile "ground-shadow-botright")
                     \< (tile "ground-shadow-left")
                     \d (tile "dirt")
                     \| (tile "wall_vertical")
                     \- (tile "wall_horizontal")
                     \1 (tile "wall_topleft")
                     \2 (tile "wall_topright")
                     \3 (tile "wall_bottomleft")
                     \4 (tile "wall_bottomright")
                     \u (tile "below-wall")
                     \v (tile "below-wall-shadow")
                     \w (tile "wood-floor")
                     \c (tile "cobble")
                     \b (tile "bush")}

                    (tile "ground")

                    ["                 1----2                            "
                     "                 |vuuu|                            "
                     "1----------------4<bbb3--------------------------2 "
                     "|vuuuuuuuuuuuuuuuu/   uuuuuuuuuuuuuuuuuuuuuuuuuuu|<"
                     "|<1------------2               cccccccc d        |<"
                     "|<|vuuuuuuuuuuu|<       d      c      c          |<"
                     "|<|<           |< cccccccccccccc    d ccccccc d  |<"
                     "|<3------ -----4< c                         ccccc|<"
                     "|<uuuuuuu/uuuuuu/ c  1-------2    1------------2 |<"
                     "|<1---2 bcb       c  |vuuuuuu|<   |vuuuuuuuuuuu|<|<"
                     "|<|vuu|< c  d     c  |< 1-- -4<   |<           |<|<"
                     "|<3- -4<bcb       c  |< |vucuu/   3------ -----4<|<"
                     "|<uu/uu/ c        c  3--4< c      uuuuuuu/uuuuuu/|<"
                     "|<  c  dbcb       c  uuuu/ cccc         bcb      |<"
                     "|<  cccccccccccccccccccccccc  cccccccc   c   d   |<"
                     "|<  d        d                     d ccccc       |<"
                     "|<                    1--------------------------4<"
                     "3---------------------4uuuuuuuuuuuuuuuuuuuuuuuuuuu/"
                     "uuuuuuuuuuuuuuuuuuuuuuu/                           "]

                    ["                 xxxxxx                            "
                     "                 x    x                            "
                     "xxxxxxxxxxxxxxxxxx xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx "
                     "x                                                x "
                     "x xxxxxxxxxxxxxx                                 x "
                     "x x            x                                 x "
                     "x x            x                                 x "
                     "x xxxxxxx xxxxxx                                 x "
                     "x                    xxxxxxxxx    xxxxxxxxxxxxxx x "
                     "x xxxxx x x          x       x    x            x x "
                     "x x   x              x  xxx xx    x            x x "
                     "x xx xx x x          x  x         xxxxxxx xxxxxx x "
                     "x                    xxxx                        x "
                     "x       x x                             x x      x "
                     "x                                                x "
                     "x                                                x "
                     "x                     xxxxxxxxxxxxxxxxxxxxxxxxxxxx "
                     "xxxxxxxxxxxxxxxxxxxxxxx                            "]
                    )))

This lets me make all of my tiles be simple PNG files with sane names. I could dork around with sprite sheets, but why bother? Emacs or Vim column-editing and overwrite modes make it easy enough to make a map this way. Swing thankfully handles transparency for me if I use PNGs, so it's a no-brainer to make multiple map layers later, which I'll need later. The code for reading in PNGs is brain-dead simple thanks to Java's ImageIO:

(defn tile [name]
  (ImageIO/read (File. (str "img/" name ".png"))))

The cache-map function (below) iterates over the ASCII art (via seqs on the Strings), using each character as a key into the map of real images, and builds a BufferedImage out of it. This is cached, since the renderer needs to re-draw the background every frame. In this function I'm also drawing a "padding" layer to sit under the background, so that I see endless fields of grass when I walk close to the edge of the map, instead of garbled, smeared-out graphical artifacts.

(defn cache-map [amap]
  (let [height (count (:tiles amap))
        width (count (first (:tiles amap)))

        #^BufferedImage img (BufferedImage. (tile-to-real width) (tile-to-real height) BufferedImage/TYPE_INT_ARGB)
        #^Graphics2D g (.createGraphics img)

        #^BufferedImage pad (BufferedImage. (tile-to-real width) (tile-to-real height) BufferedImage/TYPE_INT_ARGB)
        #^Graphics2D padg (.createGraphics pad)]
    (doseq [[y row] (cseq/indexed (:tiles amap))
            [x tile] (cseq/indexed row)]
      (.drawImage padg (:pad amap) (tile-to-real x) (tile-to-real y) REAL-TILE-SIZE REAL-TILE-SIZE nil)
      (if-let [#^Image img ((:tileset amap) tile)]
        (.drawImage g img (tile-to-real x) (tile-to-real y) REAL-TILE-SIZE REAL-TILE-SIZE nil)
        (throw (Exception. (str "Missing tile " tile ".")))))
    (assoc amap
      :map-image img
      :pad-image pad
      :max-map-x width
      :max-map-y height)))

This code is nasty, mostly due to converting between tile-based coordinates and pixel-based coordinates. This code needs to be cleaned up a bit. But that's about the most complex code you'll find in my program so far.

Who needs mutable state?

Clojure is a mostly functional language, in the sense of strongly discouraging unnecessary use of mutable state, and this program is no different. I'm sometimes amazed how far I can get before I need mutable state at all. The vast majority of my functions take a world value (a plain old hash-map) as an argument, and return a new world value after making changes to it. The current state of the world is whatever value is currently in the global WORLD ref.

The render loop grabs a snapshot of the world from the ref on each iteration, and then draws it. Thanks to Clojure refs, the snapshot of the world is guaranteed to be consistent (e.g. no NPC objects in the middle of mutating themselves) and persistent (the world value sticks around as long as the renderer needs it, even if the WORLD ref is changing in another thread). Once it's been drawn, the renderer throws the world snapshot away and it's garbage-collected later.

This all happens around 50-100 times per second in my game, and there's no noticeable lag. So that's a good thing.

Conclusion

That's about it. I'll post the full source code once it doesn't suck as much.

In my opinion, exploring a new area of study is some of the best fun a person can have. It's a flood of information and a surprise every 10 seconds.

Making a game has been like that. There's a huge wealth of knowledge about this kind of programming that I never knew existed. Everything I read on this topic is new to me and fascinating. The saddest(?) part of this whole thing is that I'm going to have more fun programming this game than I usually have playing games I buy in the store.

February 20, 2010 @ 8:27 AM PST
Cateogory: Programming

27 Comments

Stu
Quoth Stu on February 20, 2010 @ 10:03 AM PST

Interesting write-up. I think the only real stumbling point might be choosing to write and have it run in the Java VM. Not really a killer, for something simple and 2D at least, but I think it'd be easier to run into performance headaches as a result.

If you're interested in doing something like this and want to get practice with Lua, try LOVE2D. It's a cross-platform Lua interpreter that uses SDL and is based on the idea of making 2D games rapidly. After I got over the hurdle of figuring out Lua's syntax, I've been able to throw stuff together in it remarkably fast - mostly proof of concept stuff that I lost interest in at some point, but still, stuff that works and with minimal headaches.

Timo Mihaljov
Quoth Timo Mihaljov on February 20, 2010 @ 7:06 PM PST

Thank you for writing this post! It's interesting to see a game developed in a functional language, as at their core games are all about updating a huge blob of global state—the virtual world. Trying to break a game into (purely) functional pieces seems to be quite a challenge for programmers used to thinking in Object-Oriented terms. To make the culture shock even worse, many game concepts map very naturally to OO: you have a Player collecting Items into an Inventory and so on, so the temptation to write Java in Clojure is great for newbies like me.

Here are a few thoughts I had while reading this post (don't take them as criticism—they just popped in my mind and I thought I'd share them):

  • Sleeping 10 ms each frame makes the game run slower, but the duration of each frame, and thus the timestep of your simulation, is unpredictable. If you sleep just enough to make each frame take 1/60 s (capping your framerate to 60 FPS), many things become slightly cheaper and easier; for example, your characters can move at the fixed speed of n pixels per frame, you can advance the animations by a fixed amount of frames per update, etc.

  • The term "map" has quite a lot of meanings in your codebase: the function, the data structure, and the set of tiles in the world. And I assume that at some point your player might even find a map of a dungeon. To avoid confusion, it might be good to distinguish between the meanings by using names like "hash maps", "tile maps" and "map items" in comments and documents.

  • I don't think that threading the whole world through your functions is a good idea. It's almost equivalent to storing everything in global variables that any function can modify. I think it would be better to only give each function just enough of the world as several explicit arguments, and then explicitly merge the result with the previous state of the world with assoc-in or update-in.

  • Even though it's still WIP, and written for C++, you might find the book Game Programming Patterns interesting. If nothing else, it gives you a glimpse into the way pros think about game development (AFAIK, the author works for EA).

Good luck with the project, and please do keep us updated as your game progresses!

-- Timo

Chas Emerick
Quoth Chas Emerick on February 20, 2010 @ 9:52 PM PST

Seeing this sort of stuff in clojure is great. How could you possibly get any actual work done when you have a hobby project like this?! :-)

I hate to be the guy who leaves a (almost) purely technical comment, but it seems like the agent-based approach would be absolutely ideal, and should not lead to a 1:1 mapping of agents to threads. Q: were you using send or send-off to dispatch actions to your agents? The latter could lead to your thread count approaching the number of agents you have, while the latter would be working in a fixed-size threadpool matching the number of CPU cores you have (or perhaps it's 2x, I don't recall off the top of my head). I just wince a little bit seeing that hardcoded 10ms sleep.

John Cromartie
Quoth John Cromartie on February 21, 2010 @ 12:20 AM PST

I think you're wise to stick with the Java2D API for graphics as long as your game is a simple 2D one. I've seen great performance for tile-based games using this API. You could hardly do better with pure OpenGL, mostly because Java2D is hardware accelerated anyway. Good luck, and work hard on your RPG so we can all play it some day :)

James Hofmann
Quoth James Hofmann on February 21, 2010 @ 1:19 AM PST

I don't think Timo caught it, but I can see that you're already using the time delta for movement - though the additional sleep time is a little worrisome. Delta time is a good and commonplace method, but also not perfect in all cases because timing becomes non-deterministic - you can't reproduce a run of the game unless you also reproduce the time delta of every frame (deltas could also be recorded, but that only solves one problem. In situations like multiplayer games, you still have to have a method of synchronizing to the server clock. If the timestep grows too large, glitches in collision will result. And so on. There's plenty of complexity once you get deep into the concept of "timestep." A fixed-length timestep isn't as robust or convenient as a variable one, but the logic is less mind-bending.)

ASCII-art maps are a tradition of video games going back to the microcomputer BASIC era, so it's good to see them back in action. But I want to point out that there are multiple ways to reason about tiles:

Tileset mapping makes it possible to swap the look of environments by swapping tilesets. This could be done as a retrofit of the existing system. It has the added benefit of playing nicely with spritesheets. More on that later.

Using materials instead of images in the data encourages you to deal with the juxtaposition of tiles in your code and use a more abstract representation for editing. Eg: you wouldn't place the shadow tiles, your code would run through the map and generate them based on the relative locations of wall and grass tiles. You don't have a lot of tiles now so the savings here aren't great, but it can simplify the process if you start having a lot of "jigsaw" work in your maps.

You don't seem to have a story for entity layout yet. You can place them as another type of tile, or in their own list with x/y coordinates. And if you list instances separate from tiles, it opens the option of defining entities with details beyond "archetype ID." Depending on how detailed you want level design to be, you may want to develop more in this direction, with customizable property lists and targeting/sequencing mechanisms for things like traps or switch puzzles.

Last thing is asset management. You say "why bother with sprite sheets," but the problem you're overlooking is that when sprites are one-to-one with files, it becomes tiresome to maintain and work on them. Do you really want to have to give a name to "below-wall-shadow"? Or create five different bushes and call them "bush" through "bush5"? Sprite sheets, even if they're a simplified format, let you do more cut+paste to kickstart a new animation, expand a tileset, or make a variation on an existing sprite. Same reasons you're drawn to using the REPL - it's scaffolding that helps you iterate. You know you're going to be making more pixel art, so take a stab at a format, and make it as convenient as you can think of for your future self some weeks or months down the line.

None of these things are things you actually have to worry about now, they're just "non-obvious consequences."

Carl Smotricz
Quoth Carl Smotricz on February 21, 2010 @ 1:39 AM PST

Great work Brian, and many thanks for sharing that with us!

I dabble a bit in Clojure from time to time, and in game-related software at other times. Your report here is an inspiration to plug away at this some more. Also a nice endorsement for the many good features of Clojure.

Chris Lewis
Quoth Chris Lewis on February 21, 2010 @ 1:48 AM PST

This is very cool! My own fumblings in Clojure have come up short, but seeing a game, which almost everyone will think of as being purely mutable state, written immutably is awesome.

It seems that your own tests and conclusions have ended up with a game that looks a lot like Infinite Mario Bros. architecture (link to my copy of the source with a better timing mechanism than the original). You may find inspiration there.

I would definitely be interested to see if Chas' recommendation of send-off, ahem, pays off. While threading is not a traditional way (and your settled way is the same way IMB renders), I would think that this is as much as thread co-ordination overhead as anything else. As Clojure makes such things easy... it seems like there must be some idiomatic Clojure way. It's a shame that agents didn't immediately pay off, my reading of the docs leads me to believe that Clojure would be implicitly doing some interleaving, which wouldn't result in the constant transaction problem you mention. That said, my reading of the docs is usually wrong :)

I don't necessarily agree that working on the JVM is going to make it easy to run into performance headaches, but the biggest warning alarm in my head is garbage collection: if you're throwing away large amounts of world state every frame, there's going to be a lot of garbage to pick up, which could end up in stalling the game. You'll probably have to think of some sort of GC strategy that occurs during natural pauses in the game, such as talking to an NPC, changing a map tile, switching to a battle mode, or even the user pressing pause.

All in all, I think this is great! Do you have any ideas what game logic might look like in idiomatic Clojure? Is there any power you think you'll buy yourself?

Brian
Quoth Brian on February 21, 2010 @ 3:03 AM PST

Thanks all for the encouragement.

Stu: This is really a Clojure learning project, so I want to stick with it. The JVM has kept up with my needs so far. I've heard good things about Lua though.

Timo: Yeah, "map" means a lot of things in Clojure already, let alone when you add dungeon maps. At least the tile maps are capital Map in the code. In the code it isn't confusing, but talking about it is a chore.

The "world" for now really only consists of a map and entity locations, so I may as well pass it all around because everything needs to look at it all. e.g. NPCs need to know where all the other NPCs are to avoid walking into them, need to know where the walls are etc. I'm doing a lot of assoc-in on the world. It works out OK. Still a lot of refactoring to be done though. Thanks for the book recommendation.

Chas: Hmm. I was using send-off. Not sure what I was doing wrong. I might have to revisit the agent-based version. I can actually take out the call to sleep in my version, and it still works, but my CPU is probably going to outrun the refresh rate of my monitor, so there's no point going that fast. Looping and sleeping and using the real time delta to adjust the animation appropriately seems to be a standard way to do this, but I'm still learning.

James: I don't care too much about making the game deterministic right now, but that's an interesting thought. I did have a thought about having a "game clock" tick regularly in the background somehow and syncing against that, but I didn't see any benefit and I have no idea how to make that work. This game is never going multiplayer, single player is killing me enough already.

The auto-placement of shadows did occur to me as I was tediously drawing them in. To be honest, my "real" maps are probably going to be much less regular than this, and hand-placement of tiles will be essential (think FF6), so I don't think it'll be a problem. I may just draw the maps as single .PNG files. There are some good map editor programs that I could use to export to PNG.

Chris: Ah, Infinite Mario, why didn't I think to look at that? 've heard of that before but forgot about it. That would be great source code to learn from. Thanks. Haven't had any lag from the GC yet, but the JVM has concurrent garbage collection nowadays via some JVM flags, if I need it.

Not sure what you mean about the game logic. Most of the game logic is just a big state machine. Characters are either :standing or :walking so far, but later I'll have an :in-dialog state, :in-battle state, etc. It's relatively straightforward.

Chris Lewis
Quoth Chris Lewis on February 21, 2010 @ 3:33 AM PST

Brian:

By game logic I just mean the code that dictates the game design, rather than the game implementation. eg. pressing a button to swing a sword is a game implementation issue; how much damage that does is game design.

I am somewhat interested in this. I believe that there are new paradigms to be found about how we go about encoding game designs, and I was wondering if you had any vision about whether functional programming brings something to the table.

Brian
Quoth Brian on February 21, 2010 @ 3:43 AM PST

Oh, I see. I am still brainstorming the battle mechanics. I don't see how FP is going to make it any easier or harder to code, to be honest. The same kinds of things need to end up happening either way. Haven't gotten there yet though, we'll see.

Chris Lewis
Quoth Chris Lewis on February 21, 2010 @ 3:54 AM PST

That was my feeling as well, but I figured you knew more about it than I did, judging by the technologies you use to power your site ;)

Brian
Quoth Brian on February 21, 2010 @ 4:06 AM PST

Perhaps you missed the part where I point out that I have no idea what I'm doing. :)

David Nolen
Quoth David Nolen on February 21, 2010 @ 4:37 AM PST

The Clojure+OpenGL story is actually a lot nicer now. Did you look at Penumbra, http://wiki.github.com/ztellman/penumbra? It provides a very, very idiomatic interface to OpenGL via LWJGL. It's also works well with Leiningen, all dependencies, including native code will downloaded for you and you don't have to configure your paths. This makes it easy for other Clojurians to try out and hack on your code.

Penumbra also make it really easy to work with your code at the REPL. You can redefine fns and the display will update immediately.

Anyways, check it out if you decide to give another go at OpenGL with Clojure. I've been using it a bit recently and it's really fantastic.

Chouser
Quoth Chouser on February 21, 2010 @ 5:56 AM PST

Looks like a fun project. A fun project in a great language, what could be better? :-)

I think your performance issues with the initial version were just because you had only a single ref (if I understand you correctly). Where Clojure's STM and refs shine is when you have multiple threads accessing multiple refs in an ad-hoc ways that are hard to plan out ahead of time. In this case, perhaps each tile or coordinate could have its own ref. Something moving from one to another would check and update both the old and new tile ref in a single dosync. Clojure's STM would make sure there was never a race condition there, but most movements wouldn't conflict with most other movements allowing the separate threads to plow ahead unrestricted.

Anyway, it's a thought. Glad you found some way to make progress!

Brian
Quoth Brian on February 21, 2010 @ 6:05 AM PST

David: Darnit, wish I'd seen that a week ago. Thanks for the link. If Java 2D turns out not to work, that'll be the next thing I try. I think even with that library, OpenGL ends up being slightly more low-level than I'm used to with Java 2D, but I might need that level of control if my current approach fails.

Chouser: Hmm, one tile per coordinate. That might actually work out well, if each tile could be in charge of keeping track of its own state rather than one huge hash-map ref trying to manage all the tiles at once. Thanks for another option to try.

I'm glad I posted about this when I did, lots of good ideas here.

Jason Baker
Quoth Jason Baker on February 21, 2010 @ 6:06 AM PST

This makes me wonder... Is there any way to run clojure on the iphone? I only mention this to say that I'd pay $1.99 for it. :-)

Andrew
Quoth Andrew on February 21, 2010 @ 6:28 AM PST

Brian, re:Chouser's suggestion of 1 ref per tile, I believe Rich Hickey's 'ants' example from 2008 uses just such a system. He talks about it in detail on one of his videos on vimeo.

Ian
Quoth Ian on February 21, 2010 @ 7:06 AM PST

This is very cool, Brian; I really enjoyed reading about your development process and how you tackled all of this.

One note. You should just post the source code now. There's a temptation to wait until it's perfect or even "good enough," but the reality is that what you've already done will be useful, so why not put it out there for people to look at, contribute to, or fork as they may?

Eric
Quoth Eric on February 21, 2010 @ 1:08 PM PST

Do you know about vsync? It's an elegant alternative to your 10ms sleep.

Once you enable it, your swap-buffer calls will start blocking for just the right length of time.

Eric
Quoth Eric on February 21, 2010 @ 1:24 PM PST

Also, I recently read some interesting stuff about how to handle the boundary between state and pure functions (look for the parts about "named state" -- especially figure 12). It sounds like you are already doing what the author recommends but I figured you might like to check it out.

Dennis
Quoth Dennis on February 21, 2010 @ 9:11 PM PST

Ironically, I ran across this not a few days ago: http://ideolalia.com/creating-a-simple-game-in-clojure

Which, more poignantly led me to this: http://github.com/ztellman/penumbra

Which is, as the description states, '...an idiomatic wrapper for OpenGL in Clojure, by way of LWJGL.'

Don't know if it helps, but it's always cool to find another tool for your favorite language.

John Cromartie
Quoth John Cromartie on February 21, 2010 @ 10:23 PM PST

Eric: Java2D currently forces vsync. Some people complain that you can't turn it off, but I hardly see that as a bad thing, since it results in a smooth display.

Penumbra is nice, but I believe it only supports immediate mode right now, which ends up getting pretty slow pretty quickly. Until Penumbra supports vertex arrays and VBOs it won't be a serious contender in the performance department. For code clarity and ease of development it can't be beat, though. I can't imagine a better way to write OpenGL code.

Eric
Quoth Eric on February 22, 2010 @ 2:52 AM PST

I'm reading this blog entry in chunks and just read the "OpenGL?" section. My own experience with interfacing to graphics from an exotic language (I'm using Go for a game of similar scope) has led me to go with a multiprocess solution. I have an SDL process written in C which spawns the Go process with which it communicates over a pipe. Of course, you have to serialize the events and the display commands this way but I haven't found that to be a big nuisance. And after measuring pipe speed on my machine, I decided that the context switching and system call overhead was not a problem. After having played a bit with GLUT and Cocoa and Qt, I've found SDL to be relatively nice for writing a small game.

dasuxullebt
Quoth dasuxullebt on February 22, 2010 @ 4:12 AM PST

Nice. Well, I was considering to port my Jump'n'Run-Game (which is not finished and atm doesnt proceed for the lack of time to code) to Clojure, too, as lispbuilder-sdl for Common Lisp is comparably hard to use, since its an actively developed library, and SDL itself is, too, while J2D keeps being constant, and is already ported to every platform.

What kept me from doing that is - on the one hand - my animosity against clojure, and - on the other hand - that I didnt want to restart writing the engine once again (already did so a couple of times).

Anyway, your work looks interesting.

cgrand
Quoth cgrand on February 28, 2011 @ 7:55 PM PST

@Chouser the problem with one ref per tile is snapshotting in the render loop. The rendering may busy retry because of a lack of history :-/

It requires some fine tuning, especially on low-end hardware.

I think one ref per tile is too fine-grained for the STM. At somepoint I had a prototype of a refmap type. sigh yet another thing to work on as soon as I recover bandwidth...

Dave
Quoth Dave on February 28, 2011 @ 8:53 PM PST

For 3D game programming, if you're using java, check out http://jmonkeyengine.com/

Tim Azzopardi
Quoth Tim Azzopardi on February 28, 2011 @ 10:00 PM PST

The clojure+swing ants demo (one ref per tile) source code is here:

http://clojure.googlegroups.com/web/ants.clj?gda=hYoBxzoAAABoLitVpBTEcNIQc_NHg39S3ktrIN1HaghvROCePDY5Lu9OU0NQiFWgQuhmPR7veGf97daDQaep90o7AOpSKHW0

Rich talks about it in detail on this 2 1/2 hour video http://blip.tv/file/812787

the ants are agents and basically only lock the cells (Refs) that they move to and from.

However as cgrand says, the "problem with one ref per tile is snapshotting in the render loop"

In the code, that happens here:

(defn render [g] (let [v (dosync (apply vector (for [x (range dim) y (range dim)] @(place [x y]))))

This gets a consistent world view in one transaction for the purposes of rendering the screen and can of course conflict with the ants transcations and require retries etc. In practice the code works fine, although you should question the scalability on a larger map with more ants.

If you are happy with a possibly inconsistent world view then you could change the rendering code so that the dosync is around one cell at a time. As the screen refreshes n times per second, an occasionally inconsistent world view may not be a problem. A possible happy compromise might be: in the area around the player, you have a consistent world view (i.e. dosync on the cells around the player), but elsewhere (where you don't care so much) just dosync on individual cells. That would reduce contention and make it scale. Maybe!

Good luck.

Speak your Mind

You can use Markdown in your comment.
Email/URL are optional. Email is only used for Gravatar.

Preview