briancarper.net (λ) (Tag: PHP)

Adminer, where have you been all my life?

Sat, 28 Nov 2009 17:47:41 -0800

How do you view and edit data in a mysql DB? Lots of ways.

There's always commandline mysql. This is how I do it around 50% of the time. But it could be better. ASCII-art table dumps are not the easiest things to read, and readline-based history and editing only gets you so far.

phpMyAdmin is what I grew up with. It's pretty good but it's way too heavyweight. It also has all kinds of funky Javascript, to the point where I actually use Greasemonkey to remove some of it. Auto-selecting query text when you try to edit it in particular drives me crazy. Auto-selecting ANYTHING on Linux, where selecting text usually equals clobbering the X clipboard, is a really really bad idea. Last I tried it's a one line of Javascript in Greasemonkey to fix this by the way:

document.getElementById('sqlquery').removeAttribute('onfocus');

But phpMyAdmin may have changed since last I used it, it's been a while.

There's the native mysql GUI, called mysql-gui-tools in Gentoo. It's a standalone app that's pretty good, but the Linux version is gimped up compared to the Windows version for some strange reason. In any case it seems to be discontinued or something. There's some new MySQL Workbench thing coming, which I'll probably try once it hits Gentoo, but it looks like overkill for my simple needs.

SQuirreL SQL is another cross-platform GUI app. It connects to lots of different kinds of DBs (pretty much any DB that Java can talk to), so at work where I have get my mysql server to talk to someone else's MS SQLServer (ugh) I use this. But it's very heavyweight and not the most enjoyable interface.

Last week I chanced upon something pretty good. Adminer is a single PHP file you throw on a server and there you go. It gives you something vaguely similar to phpMyAdmin but far more lightweight. There's very little Javascript messing with my query-writing and the styling is minimal and easy to read. I don't know how secure it is, so I don't plan to put it on any public servers, but on my test server doing web development it's good. I love bouncing back and forth between a the database in one browser tab and my website in another tab. This is what I use now.

Clojure 1, PHP 0

Mon, 16 Mar 2009 18:50:00 -0700

Goodbye Wordpress

As I mentioned many times, I've been working on replacing Wordpress for my blogging needs. Wordpress has been pretty good for the past three years, but it's time to move on, for a bunch of reasons.

Primarily, the way Wordpress automatically mangles my text is annoying. For example, it turns newlines into paragraphs inconsistently (especially when it comes to pre/code blocks). This blog is mostly about programming, which means being able to post code without having my quotes turned into "smart" quotes and my --flags turned into long-dashes is kind of important. HTML is sometimes automatically escaped, and sometimes not. I can't count how many comments I've gotten where someone posted some code, then posted again to inform me that Wordpress ate the code for dinner. There are plugins to fix some of this, which break every time Wordpress releases a new version, and have never really worked that well for me.

Writing a theme for Wordpress means a mix of PHP and HTML and CSS, which is painful to read and even more painful to write. Aside from the considerable ugliness of PHP itself, there's a lot of weird magic involved with themes, based on naming conventions for files, weird fall-through behavior when certain theme files aren't present and so on. The Wordpress API is enormous and not fun to work with if you want to do something other than the standard Wordpressy kind of blog structure. Static pages aren't too much fun to work with in Wordpress either.

Lately I think I was getting hammered with spam partly because Wordpress is such an easy target. Askimet is nice but it wasn't catching enough lately; maybe 10-15 spams per week were slipping through. And there was always the chance that some widely-known exploit in Wordpress was going to leave my site susceptible to some roving bot.

And so on.

Hello Clojure

Why Clojure? Because it's awesome and fun and powerful and I wanted to learn it better.

Compojure is a web framework for Clojure that made a lot of this very easy. Coming here from a Ruby on Rails background, Compojure has a lot going for it in comparison. Compojure is lightweight and more low-level than Rails. For example Compojure doesn't enforce MVC on you, doesn't force a unit testing framework on you, and doesn't care how you access your data. Compojure just lets you route HTTP requests to Clojure functions based on the URL and request method (RESTfully: POST/GET/DELETE/PUT), and it gives you easy access to the request information, session, GET/POST parameters and cookies.

Under the hood it's all servlets and Jetty, both of which are solid, stable, well-tested, well-documented technologies. However, thankfully, all of that Java stuff is under the hood, and well under it. I didn't have to write a single line of Java or interact with single servlet directly. Everything (session, params, headers) is a Clojure hash-map from the perspective of my code.

Compojure also comes with a domain-specific language for writing HTML, which is similar to CL-WHO and myriad other Common Lisp HTML DSL's. All of which are awesome. I can't say enough how much nicer it is to write (or generate) structured s-exps than to write HTML by hand. More on that below.

Compojure doesn't come with any way to interact with a database, so I had to write one. clojure.contrib has an SQL lib which easily lets you interact with a MySQL database. (Clojure can talk to MySQL via MySQL's JDBC connector, of course.) I used clojure.contrib.sql to write a small (192 lines) library which slurps up a bunch of database tables into Clojure refs, and provides a few functions for basic CRUD operations so that any updates to the ref data is also transparently reflected in the database. The database is essentially only for keeping an on-disk cache of the data in case I need to restart the server. The average number of DB queries per page is zero; everything except posting/editing/deleting data just reads out of a Clojure ref.

With possibly multiple users posting data at once, it's nice to have Clojure's built-in concurrency support. Updating the data refs with new data is always safe from multiple threads simply by throwing a (dosync) around all of the write accesses. This was completely painless to write.

I decided I wanted to use Markdown for posting comments and authoring new pages. This was also very simple to do; I outlined how to get Markdown working in Java and Clojure, in a previous post. The real-time previews for comments are largely inspired by / ripped-off from Stack Overflow, implemented mostly using open-source Javascript libraries like Showdown, JQuery, TypeWatch and TextAreaResizer.

A Brief Comparison: Clojure vs. Wordpress

All of my code including the CRUD library, all of the HTML for the templates and layout, admin controls, and all the glue to put it together is 1,253 lines of code. Wordpress is somewhere over 78,000 lines of PHP depending what you count (doesn't include any themes or layout, but does include Wordpress features I didn't need and didn't implement). It's still a pretty nice reduction in code overall, any way you look at it.

As an example, in my old Wordpress site I had a plugin catcloud to generate a "tag cloud". This plugin itself is 226 lines of PHP, not bad. However, here's the Clojure code to generate a similar tag cloud (which you can see here currently):

(defn tag-cloud []
  (let [tags (sort-by #(.toLowerCase (:name (first %))) (all-tags-with-counts))
        counts (map second tags)
        max-count (apply max counts)
        min-count (apply min counts)
        min-size 90.0
        max-size 200.0
        color-fn (fn [val]
                   (let [b (min (- 255 (Math/round (* val 255))) 200)]
                     (str "rgb(" b "," b "," b ")")))
        tag-fn (fn [[tag c]]
                 (let [weight (/ (- (Math/log c) (Math/log min-count))
                                 (- (Math/log max-count) (Math/log min-count)))
                       size (+ min-size (Math/round (* weight
                                                       (- max-size min-size))))
                       color (color-fn (* weight 1.0))]
                   [:a {:href (:url tag)
                        :style (str "font-size: " size "%;" "color:" color)}
                    (:name tag)]))]
    (block nil
           [:h2 "Tags"]
           [:div.tag-cloud
            (apply html (interleave (map tag-fn tags)
                                    (repeat " ")))])))

This is 10 times less code, which is a good reduction in my opinion. Most of the code is the math to generate a weight logarithmically for each tag so they scale nicely. (all-tags-with-counts) fetches a seq of two-item pairs: the tags themselves (which are hash-maps) and a count of posts for each tag. There are two locally-defined functions in the let which generate the text color and the font size and HTML for each tag.

The vectors that look like [:h2 "Tags"] are input for Compojure's HTML-generating DSL; this would be transformed for example into <h2>Tags</h2>. (block ...) is a macro which wraps its content in HTML for the rounded borders of my layout. (Math/log ...) and friends are calls to standard Java math functions.

This whole function is less code than just the horrible boilerplate array declarations at the top of the Wordpress plugin:

$catcloud_field_data = array(
  array('name' => 'Minimum Font Size', 'option' => 'catcloud_min_font_size', 'size' => '4', 'maxlength' => '3',
       'default' => '9', 'note' => 'Used for the least frequent categories', 'validation' => '/^\d{1,3}(\.\d{1,3})?$/'),
  array('name' => 'Maximum Font Size', 'option' => 'catcloud_max_font_size', 'size' => '4', 'maxlength' => '3',
       'default' => '18', 'note' => 'Used for the most frequent categories', 'validation' => '/^\d{1,3}(\.\d{1,3})?$/'),
  array('name' => 'Font Face', 'option' => 'catcloud_font_face', 'size' => '15', 'maxlength' => '254',
       'default' => '', 'note' => 'Set an optional list of font faces', 'validation' => '/.*/'),
  array('name' => 'Font Units', 'option' => 'catcloud_font_units', 'size' => '3', 'maxlength' => '2',
       'default' => 'pt', 'note' => 'Choose one of em, pt, px or %', 'validation' => '/^(%|em|pt|px)$/'),
  array('name' => 'Color Start', 'option' => 'catcloud_color_start', 'size' => '7', 'maxlength' => '6',
       'default' => '0066CC', 'note' => 'For the least frequent categories. Use a hexadecimal RGB triplet. ie. 0066CC',
       'validation' => '/^[\dA-F]{6}$/i'),
  array('name' => 'Color End', 'option' => 'catcloud_color_end', 'size' => '7', 'maxlength' => '6',
       'default' => 'CC6600', 'note' => 'For the most frequent categories. Use a hexadecimal RGB triplet. ie. CC6600',
       'validation' => '/^[\dA-F]{6}$/i'),
  array('name' => 'Before Category', 'option' => 'catcloud_before', 'size' => '3', 'maxlength' => '20',
       'default' => '[', 'note' => 'Set the character(s) to display before category names', 'validation' => '/.*/'),
  array('name' => 'After Category', 'option' => 'catcloud_after', 'size' => '3', 'maxlength' => '20',
       'default' => ']', 'note' => 'Set the character(s) to display after category names', 'validation' => '/.*/'),
  array('name' => 'Show Top N Categories', 'option' => 'catcloud_top_n_cats', 'size' => '5', 'maxlength' => '3',
       'default' => '', 'note' => 'Show only the top N categories (where N is a number like 10 or 25 or whatever. Set to 0 or empty for no limit.',
       'validation' => '/^\d*$/'),
  array('name' => 'Excluded Categories', 'option' => 'catcloud_excluded_cats', 'size' => '15', 'maxlength' => '254',
       'default' => '', 'note' => 'A comma-separated list of category ids.',
       'validation' => '/^[\d, ]*$/'),
)

Ugh. As another example, here's the code that handles a POST request to add a new blog page:

(defn do-new-post []
  (check-login
   (let [post (add-post *params*)]
     (sync-tags post (:all-tags *params*))
     (redirect-to "/"))))

It does exactly what it says: Check to make sure the user is logged in, add the post based on the POST params, sync up the tags for that post and redirect to the front page. Lisp lets you say what you want very concisely, with a bare minimum of boilerplate.

How about speed? My Clojure code is actually generating HTML in the most brute-force and wasteful way possible. The HTML for each page is regenerated from scratch, via a cascade of a couple dozen function and macro calls, every time you load a page. But it's still pretty fast, a couple hundred milliseconds for most page requests. This is slightly faster than the Wordpress version of my site. If I ever have performance issues I can switch to another Clojure HTML library, like clj-html which uses the same vector-style syntax but pre-compiles the HTML.

How hard was it to set up on the server? Wordpress is pretty famous for being dirt-easy to deploy anywhere. My Clojure app by comparison was slightly more difficult, as you might expect, but it wasn't brain surgery. My server runs Debian. First I installed the JVM via apt, then I rsynced a bunch of jar's and clj files to the server, then I installed emacs and screen also via apt. Then I put two lines into an Apache config file to proxy-forward traffic to a local port where jetty would be listening. I started Emacs, did (require 'bcc.blog.server), did (bcc.blog.server/go) to start everything, and that's about it. Took about 15 minutes to set up from scratch. When I find a bug, I SSH in, re-attach to screen, fix it in Emacs, hit C-c C-c to recompile just the functions I need to update, and then detach from screen again.

I'm pretty pleased with this so far. It was fun to write and has all the features I used from Wordpress, plus more, and the building blocks are there to extend things if I imagine up a new feature I like.

Looks like my blog is still running today in spite of my predictions. Still waiting for the JVM to crash though, I know it's coming. I plan to post the source code for some of this once I'm sure it works.

Practicality: PHP vs. Lisp?

Mon, 22 Sep 2008 01:17:25 -0700

Eric at LispCast wrote an article about why PHP is so ridiculously dominant as a web language, when arguably more powerful languages like Common Lisp linger in obscurity.

I think the answer is pretty easy. In real life, practicality usually trumps everything else. Most programmers aren't paid to revolutionize the world of computer science. Most programmers are code monkeys, or to put it more nicely, they're craftsmen who build things that other people pay them to create. The code is a tool to help people do a job. The code is not an end in itself.

In real life, here's a typical situation. You have to make a website for your employer that collects survey data from various people out in the world, in a way that no current off-the-shelf program quite does correctly. If you could buy a program to do it that'd be ideal, but you can't find a good one, so you decide to write one from scratch. The data collection is time-sensitive and absolutely must start by X date. The interface is a web page, and people are going to pointy-clicky their way through, and type some numbers, that's it; the backend just doesn't matter. For your server, someone dug an old dusty desktop machine out of a closet and threw Linux on there for you and gave you an SSH account. Oh right, and this project isn't your only job. It's one of many things you're trying to juggle in a 40-hour work week.

One option is to write it in Common Lisp. You can start by going on a quest for a web server. Don't even think about mod_lisp, would be my advice, based on past experience. Hunchentoot is good, or you can pay a fortune for one of the commercial Lisps. If you want you could also look for a web framework; there are many to choose from, each more esoteric, poorly documented and nearly impossible to install than the last. Then you get to hunt for a Lisp implementation that actually runs those frameworks. Then you get to try to install it and all of your libraries on your Linux server, and on the Windows desktop machine you have to use as a workstation. Good luck.

Once you manage to get Emacs and SLIME going (I'm assuming you already know Emacs intimately, because if you don't, you already lose) you get to start writing your app. Collecting data and moving it around and putting it into a database and exporting it to various statistics packages is common, so you'd do well to start looking for some libraries to help you out with such things. In the Common Lisp world you're likely not to find what you need, or if you're lucky, you'll find what you need in the form of undocumented abandonware. So you can just fix or write those libraries yourself, because Lisp makes writing libraries from scratch easy! Not as easy as downloading one that's already been written and debugged and matured, but anyways. Then you can also roll your own method of deploying your app to your server and keeping it running 24/7, which isn't quite so easy. If you like, you can try explaining your hand-rolled system to the team of sysadmins in another department who keep your server machine running.

Don't bet on anyone in your office being able to help you with writing code, because no one knows Lisp. Might not want to mention to your boss that if you're run over by a bus tomorrow, it's going to be impossible to hire someone to replace you, because no one will be able to read what you wrote. When your boss asks why it's taking you so long, you can mention that the YAML parser you had to write from scratch to interact with a bunch of legacy stuff is super cool and a lovely piece of Lisp code, even if it did take you a week to write and debug given your other workload.

Be sure to wave to your deadline as it goes whooshing by. If you're a genius, maybe you managed to do all of the above and still had time to roll out a 5-layer-deep Domain Specific Language to solve all of your problems so well it brings tears to your eye. But most of us aren't geniuses, especially on a tight deadline.

Another option is to use PHP. Apache is everywhere. MySQL is one simple apt-get away. PHP works with no effort. You can download a single-click-install WAMP stack nowadays. PHP libraries for everything are everywhere and free and mature because thousands of people already use them. The PHP official documentation is ridiculously thorough, with community participation at the bottom of every page. Google any question you can imagine and you come up with a million answers because the community is huge. Or walk down the hall and ask anyone who's ever done web programming.

The language is stupid, but stupid means easy to learn. You can learn PHP in a day or two if you're familiar with any other language. You can write PHP code in any editor or environment you want. Emacs? Vim? Notepad? nano? Who cares? Whatever floats your boat. Being a stupid language also means that everyone knows it. If you jump ship, your boss can throw together a "PHP coder wanted" ad and replace you in short order.

And what do you lose? You have to use a butt-ugly horrid language, but the price you pay in headaches and swallowed bile is more than offset by the practical gains. PHP is overly verbose and terribly inconsistent and lacks powerful methods of abstraction and proper closures and easy-to-use meta-programming goodness and Lisp-macro syntactic wonders; in that sense it's not a very powerful language. Your web framework in PHP probably isn't continuation-based, it probably doesn't compile your s-expression HTML tree into assembler code before rendering it.

But PHP is probably the most powerful language around for many jobs if you judge by the one and only measure that counts for many people: wall clock time from "Here, do this" to "Yay, I'm done, it's not the prettiest thing in the world but it works".

The above situation was one I experienced at work, and I did choose PHP right from the start, and I did get it done quickly, and it was apparently not too bad because everyone likes the website. No one witnessed the pain of writing all that PHP code, but that pain doesn't matter to anyone but the code monkey.

If I had to do it over again I might pick Ruby, but certainly never Lisp. I hate PHP more than almost anything (maybe with the exception of Java) but I still use it when it's called for. An old rusty wobbly-headed crooked-handled hammer is the best tool for the job if it's right next to you and you only need to pound in a couple of nails.

RSS feed (Common Lisp)

Mon, 18 Feb 2008 18:26:58 -0800

I made an RSS 2.0 feed for my origami gallery the other day. Thanks to CL-WHO this is trivial. Here's the complete code:

;; RSS
(defun rss ()
  (cl-who:with-html-output-to-string (s nil :prologue "<?xml version=\"1.0\" encoding=\"ISO-8859-1\" ?>")
    (:rss :version "2.0" :|xmlns:atom| "http://www.w3.org/2005/Atom"
          (:channel (:title "An Origami Gallery")
                    (:link "http://origamigallery.net")
                    (:|atom:link| :href "http://origamigallery.net/feed" :rel "self" :type "application/rss+xml")
                    (:description "A photo gallery.  Of origami models.")
                    (loop for model in (all-models)
                          do (htm (:item (:title (str (fullname model)))
                                         (:link (str (absolute-url model)))
                                         (:guid  (str (absolute-url model)))
                                         (:description (str (clean-remarks (remarks model)))))))))))

There are other Common Lisp libraries to make RSS feeds, which may have made this even easier, but I like CL-WHO.

Interesting about this code is how short it is. CL-WHO is an example of embedding another language in Common Lisp (e.g. a domain specific language), in this case, XML or HTML.

XML in particular happens to map very nicely onto s-expressions, allowing nested tags and attribute/value pairs and whatnot. Except that s-exps are far less verbose than XML, far nicer to type, and far easier to read in my opinion. One quirk here is that CL-WHO uses keyword symbols for XML tag names (CL keyword symbols begin with a colon), and XML uses colons for namespaces. To make a keyword symbol with a colon in the middle in CL, you have to surround it in pipes. No big deal though.

This kind of thing isn't entirely possible or quite as elegant in most other languages. (Ruby does pretty good though.) In PHP, for example, you can mix up PHP and HTML content, but the PHP interpreter doesn't parse the HTML or do anything interesting with it. It just treats it as a bunch of strings; embedding PHP in HTML is just a shortcut for string concatenation.

You could write a library in PHP to generate XML code (and I'm sure such libraries exist) but they're either going to have some kind of functional / OO interface, which will make it very verbose and probably clunky to use in comparison to CL, or else it's going to be something like Smarty which actually isn't PHP at all but rather a separate language with its own parser and interpreter or compiler that itself happens to be written in PHP. What you could not easily do is write PHP code that turns other PHP code into an XML document.

But that's largely what CL-WHO does. Because CL-WHO parses the s-exps that represent my XML data, if I forget a closing paren, Lisp will yell at me, so my XML is guaranteed to be well-formed. CL-WHO doesn't verify that I'm using proper RSS tags in the proper places or anything but it could easily be extended to do that. See PCL for an HTML interpreter/compiler which can verify that you only use valid HTML tags, for example.

CL-WHO also lets me change whether my XML attribute values are in single or double quotes, or whether I want to allow empty tags (as in old HTML) or require them all to be closed, and other such things. CL-WHO can nicely indent my XML, or generate it unindented.

And since this is all just Common Lisp, you see that I can embed a loop right in the middle of the XML s-exps, or do conditionals, or I could write a function that generates some or all these s-exps for me programatically. Or any number of other things. I have all of Common Lisp at my fingertips to create these s-exps.

See also "The Nature of Lisp" which is a nice long discussion of how Lisp is a better XML than XML. (And I do mean looong. I haven't read the whole article myself, but enough to know it's pretty good.)

One benefit of a Lisp-driven website

Mon, 11 Feb 2008 22:06:23 -0800

My Common Lisp-powered origami photoblog is still up and running smoothly so far. (I posted more models, go look, shameless plugs and so on and so forth.) No major problems to report in the past couple weeks.

One huge benefit (in my opinion) of a site run on Common Lisp is the way you can solve the "admin control panel" problem. Most web site frameworks / blog engines / message boards have some control panel interface, for example the one I'm using to type this blog entry.

Why are control panels necessary? You could SSH to your server and run SQL queries directly to input your blog posts. One reason we don't do this is because it bypasses the logic of your website. We use a PHP form so that it can complain if you type a post with no title, or run a filter to convert your line-breaks to HTML tags, or do spell-checking, or whatever. It does all these things before entering your data into the DB, and then when the DB does need to be updated, it updates it in a consistent way. If multiple tables need to be updated, it doesn't forget to do them all. Etc. etc.

The problem I have with control panels is that due to limitations of web languages and how the internet works, these control panels must be run as web pages just like any other. Meaning they're open and accessible to the world if you visit the right URL.

To solve this we start heaping on the passwords. Above and beyond SQL client access and FTP access and shell account access, we make a brand new custom layer of user accounts and permissions. Only we implement it using .htaccess files and SQL hacks and fragile HTTP connections and cookies and sessions. And then we get to test how well we did by letting every script kiddie and crawler-bot in the world hammer on it at will.

I agonized about how to make a "control panel" for my photo blog. These things aren't easy to get right. Eventually, I realized it wasn't necessary. The REPL is my control panel. I can implement the necessary logic as simple Lisp functions. When I want to post a new model, I SSH to my server, fire up Emacs, connect to the Lisp running on my running server via local SLIME, and run an ADD-MODEL function. Simple. Lisp automatically timestamps my posts, and checks for empty-string model names, and all the other good stuff you'd want from a control panel. (Plus if it fails, I get a debugger.)

What about authentication? I get that for free. SSH is my authentication. Authentication is what SSH is made for and it's going to be far better at it than some web-page SQL-based hack-job I come up with. I don't even have to remember a password, since I have certificates set up for passwordless login. And there's no URL that points to my photo blog control panel, which is nice.

What about letting multiple people log into the "control panel"? Anyone you don't trust to SSH into a shell account usually shouldn't be trusted to log into a Wordpress or VBulletin control panel either in my opinion. Run Lisp as a non-privileged user (probably should be doing that anyways, I am) and run SSH in a chroot if you really care.

What about the typical web gallery feature of letting you upload an image and then having it automatically thumbnailed? I wrote a two line shell script that uses ImageMagick (which is likely what a PHP-run gallery would be using anyways) to thumbnail all my photos locally, then rsyncs them to my server. Why re-invent the wheel?

Couldn't Perl or Ruby or PHP do the same thing? Well, Perl/Ruby/PHP don't run persistently on the server. So it'd be a bit different. Couldn't you write some standalone scripts to run from the commandline to insert new posts into your blog? I suppose, but it surely wouldn't be as nice an interface as Slime in Emacs, unless you enjoy using bash as your text editor. And have fun with quoting / escaping. (EDIT: As some readers reminded me, yes they can run persistently via Apache hackery. And Rails has a "console". I stand corrected.)

The REPL obviously isn't a solution to every problem. If your authentication requirements are complex enough, you'll have to build something yourself. And it's a problem if your users don't know Lisp (and yeah, that right there kills it for 99.999998% of the world). Web-based control panels have the benefit of being "so easy anyone can use them".

But for my needs, and probably the needs of a great many websites run by one or two trusted Linux-savvy people who just need to be able to securely update the site once in a while, a few Linux tools + the REPL works beautifully.

sigh

Thu, 10 Jan 2008 03:54:05 -0800

I spent four hours working on my photo-blog in Common Lisp that I've mentioned earlier.

Why does Common Lisp hate me? After four hours, I'm not yet even to the point where I can reliably, successfully load all the libraries I need to load. How pathetic is that? I haven't even gotten to the point where I could, you know, actually write my program.

This article explains how to set up an ASDF system file. I'm using ASDF because you almost HAVE to use it if you're utilizing any 3rd-party libraries, because so many 3rd-party libraries use ASDF themselves and because I wouldn't know where to begin to write my own dependency-resolution scripts from scratch.

So my ASD file has all the right dependencies, so all the libraries load in the right order and Lisp knows they exist. Then I have a packages.lisp that defines my package. Then I have a main source file with all my code in it, which is IN-PACKAGE my package. This all appears to work.

As long as I fully-qualify every function that comes from another package, it works fine. Problem is that if I try to (:use :cl-who) in my package, to avoid having to endlessly repeat myself every time I use a function from CL-WHO, I get a flood of warnings saying:

undefined variable: PRINC

Funny, because PRINC is a built-in function in Common Lisp. And PRINC is defined in my package. And if I IN-PACKAGE to CL-WHO, it's accessible from there too.

Can I use ASDF and still have my package inherit functions from other packages? This is the great mystery which I have been completely unable to solve. Is this some compile-time vs. runtime mess that only exists in Common Lisp? Am I accidentally interning symbols and obliterating other packages' symbols? Am I just mentally retarded?

This is extremely frustrating. I don't care how wonderfully great your programming language is: if you have to arm-wrestle the thing for a week to get your environment set up, what's the point? How can something so simple as importing functions from other packages be so difficult?

I could've had everything done already if I'd have used Ruby. I could gem install what I need, and then require it and wow, it works. Hell, I could've had this done in PHP or Perl by now.

If I ever get CL working on my machine, then I have to worry about installing SBCL and all these libraries and mod_lisp and whatnot on my Debian server. God almighty, I don't even want to think about that.

False

Thu, 13 Dec 2007 19:55:25 -0800

In Ruby, the only things that are false are nil and false. This is a nice change over other languages where other things evaluate to false in boolean tests. Like C and many others, which considers the integer 0 to be false. So is 0.0 false? How about 1e-999999? Or there's Perl, which also considers an empty string to be false, and auto-casts between numbers and strings, which can make things fun. And of course Perl has Perlish insanity like 0E0, "zero but true".

However all languages pale in comparison with PHP. They provide these handy charts to help you figure out what's true and what's false.

That really sums up PHP well. You have three standard functions (isset, is_null, empty) with completely inconsistent names. Two equality test operators whose meaning completely changes between PHP versions. And then a couple of 12x12 grids of crap necessary to figure out how they behave.

"0" == 0? True.

0 == ""? True.

"0" == ""? False...

null == array()? True....?

Really makes you appreaciate Ruby.

Better

Tue, 04 Dec 2007 02:41:42 -0800

Today I turned my 750+ lines of ugly PHP into 200 lines of much nicer PHP, and the shorter version works much better than the longer version. So I'm feeling happier with myself than I was yesterday. I did finally manage to make my parser recursive and remove all the special-case handling code. I had an interesting experience where I was diligently hacking away at my code, and then I noticed everything was actually working and had been working for quite some time without my noticing.

Still if I was a better programmer it wouldn't have taken me a month to figure this out.

My only worry now is that if anyone but me ever has to use or maintain these scripts, they're going to have to understand recursion and understand what I was going for here. That's a scary thought. One guy who interviewed me for a job a few years ago said that he asks a lot of people to explain recursion to him during job interviews, largely because a lot of programmers don't even know what the word means.

Sigh

Sun, 02 Dec 2007 03:20:45 -0800

Part of what Ableson and Sussman keep saying in these lovely lecture videos is that a good way to solve a problem is to divide it into layers of abstraction, where one layer of abstraction gets its work done by using a "language" that you wrote in the abstraction layer immediately below it. They say that a good way to solve a problem is to write something powerful enough to solve the general KINDS of problems that you're trying to solve, because you're probably going to need something that powerful later anyways. In these videos they tend to say in a very clear, direct, rigorous way things that I've heard said many times in other places by other people.

In particular, in the videos they stress trying to keep your "custom languages" written in such a way that they can be used recursively. My grand PHP project that I've been working on for some time at work is disturbingly similar to some of the examples they use in those lectures; namely, my project seems to be a good example of how NOT to do things.

My project is a set of scripts to build forms to conduct an interview. It's basically an enormous HTML form, but the form is split into pages, and the pages contain questions, which are groupings of HTML controls and text labels and Javscript and other things. To borrow language from SICP: questions are really my primitives. In addition to questions there are "groups" of questions that behave like single questions in certain circumstances but not others. And some questions have "sub-questions" that depends on them in hard-to-handle ways. So in SICP terminology: Questions can be combined by forming groups, or forming question/sub-question pairs, or otherwise just by listing them sequentially. The groupings can be into rows, or into columns, or into both, or into other kinds of patterns.

The program works by having someone write the survey in YAML, which is a very easy plaintext-ish way to write config files. Then my scripts parse the YAML, build my question objects and render them. The end. What I have though is essentially a custom language, with YAML as its syntax, used for building arbitrary surveys. It works fine and I was able to write our 20-page interview in this survey-language very easily. Ableson and Sussman would hopefully be proud (if highly disgusted by the details of my implementation in PHP).

When I was designing these scripts I do remember having the thought of whether I should make the questions in my language behave fully recursively. In other words I could simply make EVERY question have the ability to store references to one or more sub-questions. Then there would be only one kind of question, which may or may not have sub-questions. When I render a question, I'd just have it render itself and then render all its children recursively.

I gave that a try but gave up. In practice (so I thought at the time), questions should never really need to nest more than one or two levels deep, and the questions should only be combined into groups in a very limited couple of ways. So really, a fully recursive language wasn't necessary. So I wrote my language such that grouping questions together was a special case handled with extra code. Likewise sub-questions for normal questions were a special case handled with even more extra code. This worked OK.

If only I'd have seen these videos a month ago. This week I came upon a situation where the survey-building language I came up with isn't powerful enough to handle a certain type of grouping construct which I need. My language isn't arbitrary enough to let me do what I need to do. To make matter worse, looking at my code now, I realized that all the special-case code required to handle all the things I thought wouldn't be a problem has accumulated far beyond what I'd have ever expected.

Live and learn, I suppose. I think I was wandering down the right path, even if I fell off into the ditch before I reached the end of it.

Some other guy built a similar survey system for a different group at work a while back, and I had a chance to look at his code. That code consisted of one long listing of intermingled PHP, SQL and HTML, hard-coded throughtout to work specifically for the survey the group was conducting. I wanted to use those scripts myself but they were essentially unusable because they were so non-generic. I at least did better than that.

Lisp vs. Smarty

Fri, 16 Nov 2007 21:28:40 -0800

I started writing a post called "Lisp vs. Smarty" two nights ago but lost my train of thought before I could finish. Lately I see lots of blog entries about PHP and in particular this one by Sean Potter about Smarty made me want to finish typing this.

In Practical Common Lisp, in two chapters (30 and 31) and a few hundred lines of code, Peter Seibel produces a library he calls FOO that can compile (or interpret) s-expressions into HTML. FOO is essentially an interpreter/compiler for a little mini HTML-sexp language, which can compile the HTML-sexp language into Lisp code (that can be run later as a Lisp script) or can use the HTML-sexp code to produce HTML output directly. The book is there on his site for free in its entirety, feel free to read those chapters.

This FOO library is immensely powerful, and the things that make FOO more powerful than Smarty are the things that make Lisp more powerful than PHP in general. Personally I found it a great example to help me understand some of the differences between Lisp and non-Lisp languages.

Both Smarty and FOO are libraries that have at least two purposes: To produce dynamic HTML based on parameterized input (i.e. feed it values that are inserted into the final HTML) and to produce HTML without having to write a thousand static files manually by using control and looping constructs and by allowing you to have re-usable modular templates.

So, you can embed Lisp code into FOO to act as simple text filters. You can refer to Lisp variables in FOO scripts. You can use control structures like if/then/else, or looping constructs, or what have you. That's nothing different than Smarty can do though. Smarty is good at those things.

But since this is Lisp, you can easily write macros that simplify, abstract, or otherwise help you write the FOO HTML-sexps. In other words, you can write Lisp code that largely writes your HTML-sexps for you.

With Smarty, you're much more limited in what kinds of abstractions are available to you. You're probably going to write the Smarty templates mostly by hand. You're usually not going to do the equivalent of Lisp and write PHP code that writes your Smarty templates. If you do, it's going to be in certain limited ways Otherwise it would probably be more verbose and more of a pain than writing the Smarty templates yourself. After all, the point of Smarty is to help you NOT to have to write PHP. Trying to programatically produce Smarty templates using PHP is a mess largely because there's no easy way for PHP to understand Smarty templates other than as strings. Using PHP to produce Smarty and using PHP to produce HTML are equally nasty problems to try to solve.

So instead you often use Smarty to "write" Smarty. But not really. You write template files, split them up, and then you import various bits and pieces of some template files into the insides of other template files. You can throw Smarty looping and control constructs around your Smarty include statements, so this all works pretty well. But the Smarty template language is obviously rather restricted in what tools it gives you for this kind of meta-templating. And templates are about the finest granularity you're going to get.

And note, the Smarty templates surely aren't going to write the HTML tags for you. They aren't going to auto-close your open HTML tags, letting you choose how to do it based on your preference for HTML or XHTML. Smarty templates aren't going to make sure you put quotes around your attribute lists. They aren't going to (optionally) properly indent your HTML code. FOO does all of these things and more. FOO can also be used to make "templates" much like Smarty, but it gives you far greater control than that.

So let's talk extensibility. Smarty gives you hooks you can use to write plugins that do various things. The Smarty plugin documentation is around 15 pages of documentation largely telling you what you CAN'T do. Smarty plugin files for example must be named certain things, and put in certain folders. Your tags must look and act in certain ways. When your functions are called, they receive parameters X, Y, and Z, which Smarty provides, and then you must return exactly Q, which Smarty expects from you.

There are a limited number of very specific KINDS of functions you can write. You can write a function to filter some text to change it into other text. You can write a function that gets a big string full of raw un-parsed template text and returns a big string back after you do something to it. You can write a function that gets a big string full of already-parsed template text and return back a big string of text after you do something to it. Etc. Key word here: BIG STRINGS.

But let's say for example that the {if}{else}{/if} construct was missing from Smarty, and you wanted to write it. Could you do it? Smarty gives you no hook to write something like that. So you could try to use what Smarty dose give you. You could make up three tags ( {if}, {else}, and {/if} ). And then you could maybe write some functions that take big strings of the template text that's in between those tags, and parse it yourself. Maybe you could write (in PHP) a sort of state machine, global enough to last between calls to the custom functions you wrote. Would it handle an arbitrary number of {else}'s per {if}? Would it work recursively? That might take some more work.

If you're resorting to such a thing, you're almost re-inventing Smarty yourself. So hey, you could just go and edit the Smarty compiler PHP code directly to add this functionality.

I just looked at the Smarty_Compiler.class.php file that comes with Smarty. Oh the horror. Inside are line after line of regular expressions designed to take tokenize Smarty template text. Once it's been tokenized, there's an enormous switch/case statement to deal with the allowed tags. Some tags are translated directly into PHP code, which is output as a bunch of our friend, the plain old string, later to be presumably eval'ed. Some tags rely on other tags, so there's a stack to which tags are pushed and popped as the file is parsed. It really is a full-fledged compiler: A "Smarty template language"-to-PHP compiler, written in nice slow PHP.

I tried to track down the code responsible for implementing the {if}{else}{/if} construct in this Smarty compiler file. The 40+ lines of swtich/case that handle the {if}-related tags make a couple calls to, among other things, a function called _compile_if_tag. This one function is 163 lines long. Line count isn't a good example of complexity, of course. So here are just a few of those 163 lines:

/* Tokenize args for 'if' tag. */
preg_match_all('~(?>
    ' . $this->_obj_call_regexp . '(?:' . $this->_mod_regexp . '*)? | # valid object call
    ' . $this->_var_regexp . '(?:' . $this->_mod_regexp . '*)?    | # var or quoted string
    \-?0[xX][0-9a-fA-F]+|\-?\d+(?:\.\d+)?|\.\d+|!==|===|==|!=|<>|<<|>>|<=|>=|\&\&|\|\||\(|\)|,|\!|\^|=|\&|\~|<|>|\||\%|\+|\-|\/|\*|\@    | # valid non-word token
    \b\w+\b                                                        | # valid word token
    \S+                                                           # anything else
)~x', $tag_args, $match);

...

if(preg_match('~^' . $this->_func_regexp . '$~', $token) ) {
        // function call
        if($this->security &&
           !in_array($token, $this->security_settings['IF_FUNCS'])) {
            $this->_syntax_error("(secure mode) '$token' not allowed in if statement", E_USER_ERROR, __FILE__, __LINE__);
        }
} elseif(preg_match('~^' . $this->_var_regexp . '$~', $token) && (strpos('+-*/^%&|', substr($token, -1)) === false) && isset($tokens[$i+1]) && $tokens[$i+1] == '(') {
    // variable function call
    $this->_syntax_error("variable function call '$token' not allowed in if statement", E_USER_ERROR, __FILE__, __LINE__);                      
} elseif(preg_match('~^' . $this->_obj_call_regexp . '|' . $this->_var_regexp . '(?:' . $this->_mod_regexp . '*)$~', $token)) {
    // object or variable
    $token = $this->_parse_var_props($token);
} elseif(is_numeric($token)) {
    // number, skip it
} else {
    $this->_syntax_error("unidentified token '$token'", E_USER_ERROR, __FILE__, __LINE__);
}

And so on, and so forth.

Suffice it to say, in the Smarty compiler, the code that implements {if}{else}{/if} is at least a couple hundred lines of code that looks like the above. It would take me quite a bit of time and effort even to figure out what that code does, let alone write it myself and debug it.

Now, that code is part of the Smarty compiler itself, written by the people who wrote Smarty, i.e. the people in the world who know Smarty best. If I was trying to write my OWN code that gave me a {if}{else}{/if} structure in Smarty, my code would by necessity be FAR MORE complex than that. If written as a plugin, my code wouldn't have direct access to the guts of the Smarty compiler. Instead my code would have the very limited access that Smarty provides to plugins and custom functions. Even if I tried to hack the compiler msyelf, my code would be written by me, who certainly doesn't have the expert knowledge necessary or understand the inner workings of Smarty well enough to get this code working without a good deal of effort.

So let's contrast this with FOO. If an if-then-else sort of construct was missing from FOO, could you write it yourself? It turns out you could. Peter Seibel calls this a "trivial example" and mentions it in passing. It turns out you could add this construct to the FOO mini HTML-sexp language using two lines of Lisp.

Using TWO LINES.

TWO LINES!

(define-html-macro :if (test then else)
  `(if ,test (html ,then) (html ,else)))

There's around 50 lines of Lisp that puts everything in place to you allow you to write "plugins" like the above or far more complex than the above. 50 lines of Lisp that let you pretty much arbitrarily extend the HTML-sexp mini language itself, to add new tags including entirely new control constructs, that the HTML-sexp compiler understands.

Lisp already has a perfectly fine if-then-else construct after all, as does PHP. Somewhere in the PHP compiler itself, there's logic (maybe written in C? Maybe assembler?) that implements if-then-else. Why can't we use that compiler code? Why did we have to write our own Smarty compiler to do it all for us?

Rather than write a new templating language, and then write a tokenizer and lexer and parser for that language, and then slurp up a bunch of strings and translate it all back to a plain old PHP if-then-else, which is then transformed by the PHP compiler itself into native code that implements if-then-else... in FOO, we use Lisp. Lisp can already read s-expressions and parse them properly, and give us access to the result as native Lisp objects. Lisp has if-then-else, which we can use directly. That's all we need. Aside from the wonderful simplicity of it all, imagine the difference in performance.

The key is that the HTML-sexp language is s-expressions. Lisp itself is s-expressions. The compiler that translates the FOO code into Lisp is s-expressions. The code that will run the FOO compiler is s-expressions. The macro code that helps you abstract all of the above is s-expressions. The macros that help you write THOSE macros are s-expressions. Everything is s-expressions. And Lisp is VERY GOOD at reading, writing, and arbitrarily manipulating s-expressions. Which means that Lisp is VERY GOOD at reading, writing, and arbitrarily manipulating mini-languages like FOO, mini-language-interpreters and compilers, Lisp code, Lisp code that runs Lisp code, and Lisp code that writes Lisp code.

One guy wrote FOO, and then very nicely explained start-to-finish every single line of code that makes it work, in two short chapters of a beginners' Lisp book. Compare this with thousands of lines of PHP written by a team of programmers over the course of apparently six years, to produce a library that at the end of the day does far less than FOO can do.

PHP namespaces? Nope!

Tue, 13 Nov 2007 01:53:29 -0800

Today, I found out that PHP 5 still doesn't support namespaces. This is kind of extremely crap. I wrote a bunch of classes whose names are simple things like Scale and Numeric. These are good names because non-programmers have to work with this code, and certain names have meanings in those people's field of study which would be helpful to re-use for their sake.

But now I have to rename my classes because I'm sure they're going to clash with something, having such simple and generic names. So I'm likely going to have to append a string to the front or end of each one, e.g. ScaleQuestion and NumericQuestion.

In other words, now I get to implement a hacked-up version of namespaces myself, in nice slow PHP code rather than native compiled PHP-interpreter code, using manual string manipulation of class names. The only thing that's going to allow me to do this is the fact that PHP lets you use strings as symbolic references, which itself is incredibly evil.

Which is worse, living with a deficiency of the language and dumbing down / un-abstracting my code, or making up for its deficiencies by essentially constructing and eval'ing a bunch of strings?

Lisp quibbles

Fri, 09 Nov 2007 20:50:55 -0800

At night to relax I've still been reading about Common Lisp. There's a lot to like about it.

But there's one thing that drives me crazy:

Let's define a variable: DEFVAR, DEFPARAMETER.
Let's define a method: DEFMETHOD.
Let's define a macro: DEFMACRO.
Let's define a package: DEFPACKAGE.
Let's define a class: DEFCLASS.
Let's define a symbol macro: ~~DEFSYMBOL~~ Whoops, nope, it's DEFINE-SYMBOL-MACRO.
Let's define a condition class: ~~DEFCONDITION~~ Sorry, nope, it's DEFINE-CONDITION.

...

Let's access an element of a plist: (GETF SOMELIST :KEY)
Let's access an element of an array: (AREF SOMEARRAY INDEX)
Let's access an element of a vector: (ELT SOMEVECTOR INDEX)
Let's access an element of a hash: ~~(GETHASH SOMEHASH KEY)~~ Wrong! It's (GETHASH KEY SOMEHASH).

For some macros, you have to quote your arguments. For some others, you don't. For some it's a mixture. Sometimes a symbol will evaluate to a variable. Sometimes it'll be read as a symbol, not evaluated. Sometimes it'll be evaluated but you never want it to be evaluated so you always need to quote it.

Sometimes parameters to function calls need to be grouped into lists and sublists. Sometimes you just pass in a big flat bunch of arguments and the method sorts them out. Often it's an arbitrary mix of the two.

Yeah, my problem is consistency. Programming is hard enough without throwing in a requirement for a bunch of rote memorization of the wording and ordering and grouping of things. Especially when those things are arbitrary, and there's no reason NOT to have them be consistent. In Lisp they're probably even more arbitrary than other languages.

This is Lisp, and if I wanted, I could write my own macros that translate new, consistent versions of those things into the inconsistent default versions. Lisp gives you the power to do that easily. But if I do that, I'm no longer writing the same Lisp everyone else* is writing. I'm writing my own derivative where I'm replacing standard things with non-standard things of my own.

There's more to a language than being syntactically correct, after all. There are dialects and idioms, and those are very important. If you learned English from a textbook, but never spoke it aloud with anyone speaks English natively, would you be able to communicate well? Would someone from England or America or India understand you? Likely you would be just barely intelligible to all of the above. It's the same English language, but how you use the language matters as much as being strictly grammatically correct. Your accent, your word choice, your general style, those things matter a whole lot.

It's the same with programming languages. If I wrote Ruby and used C-like for loops and 8-character-wide tab-indentation and CamelCase and ended every line with a semicolon, anyone very familiar with the language would cringe to read it. In Lisp, where you can change pretty much everything about the language, there's even more potential for writing things that are unintelligble to the community at large.**

Lisp does have a beautiful consistency to it overall though.*** Everything is an s-expression. (Well, except LOOP. Oops!) Everything returns a value. Code and data and code that writes code are all the same thing. It's only a few little quirks like the ones I mentioned above that drive me crazy. Consistency is also a large part of why I like Ruby: everything is an object, (almost) everything returns a value. And horrible inconsistency is one reason things like Java and PHP drive me absolutely insane.

The power of programming is in abstraction. When a group of things aren't consistent, there is no abstraction that can represent them. Or if there is, the abstraction is likely to be as nasty as the original problem. That's a bad thing.

( * "everyone else" is probably around 17 people worldwide.)
( ** "at large", for lack of a better term.)
( *** Jokes about how nobody uses Lisp aside, I do like the language plenty.)

Javascript forays, PHP adventures

Wed, 07 Nov 2007 00:39:42 -0800

To add yet another language to the ever-expanding list of languages I know well enough to get by but have far from mastered, in the past week or two at work I've needed to play with Javascript a lot. I've long thought of Javascript as the bane of my existence, if only for the horrid browser incompatibilities you have to deal with. I hate hate hate hate writing code that needs to have a bunch of silly conditionals that change the code's behavior based on differences in the underlying platform. In this case of course the platform is a browser, but I hate it equally dealing with differences between operating systems, differences between versions of libraries, and other similar things.

The ideal situation is to use some library where someone has already worked out those platform-dependent quirks. In the case of Javascript, there's JQuery. It really does work wonders. I read recently that Google uses it for some of their sites, and I can probably see why. JQuery gives you some helper functions that let you deal very nicely with event handling and element-selection.

To my surprise, Javascript actually is turning out to be a surprisingly powerful language. It apparently supports closures, and functions are first-order objects. The OOP system in Javascript is oddly different from other languages I've used, but it's workable. I'm doing some relatively funky stuff, and Javascript is handling it well enough. And everything I've done in JQuery so far works in IE6 and Firefox. That's a good sign.

In the course of all this, I'm also using PHP. I had to repress a PHP-induced shudder even as I write this. But this is my first venture into the realm of PHP5, and again it's turning out OK. They made a lot of changes between PHP4 and PHP5, many of which made a difference. The language is still far too verbose for my tastes; Ruby has spoiled me in terms of variable names without the need for Perl-style sigils. And the overload of array()'s in PHP still drives me batty. But this time, I'm making heavy use of some third-party libraries, e.g. Smarty templates for my interface, and Spyc to read some YAML for config and input files. It lets me do stuff in PHP without actually writing much code in PHP. Can't beat that.

Maybe that's the secret of life: to make up for deficiencies in languages I dislike by using tons of third-party libraries. Maybe. Well, no, probably not.

In other non-news, my Gentoo box should arrive at my door via UPS on Thursday. Sweet sweet salvation. I'm going to put this Vista laptop in the closet and bolt the door shut.

nofollow

Thu, 04 Oct 2007 15:51:25 -0700

Someone recently mentioned that author comment links in my blog were all given a nofollow attribute. So I wrote a plugin that removes nofollow from comment author links. If you search around it's not hard to find a million other plugins that do the same thing. Putting something like this into /wp-content/plugins (and activating the plugin) should probably do it:

<?php
/*
Plugin Name: Remove nofollow
Plugin URI: http://briancarper.net
Description: Remove 'nofollow' from comment author links
Version: 1.0
Author: Brian Carper
Author URI: http://briancarper.net
*/

function remove_nofollow($text) {
    return preg_replace('/ rel=\\'.*?\\'/', '', $text);
}

add_action('get_comment_author_link', 'remove_nofollow');

?>

There is apparently some amount of controversy about nofollow. Who woulda thought. I personally don't see much need for it. Links are links. If I didn't want links to be followed, I wouldn't post them or let other people post them in the first place.

Updated cows

Thu, 13 Sep 2007 15:30:46 -0700

I've read that good website layouts evolve slowly over time, and it's better to change and improve the layout you've already got rather than redo the whole thing from scratch. Probably sage advice. I tweaked my layout a bit to take more advantage of a bunch of empty space where nothing much was happening, and also to add more cow. My girlfriend drew me a cow silhouette for my blog layout. (By the way, when your girlfriend is willing to take the time to draw you a cow graphic for your blog layout, you know it's true love. <3) I used a sort of cliche'ed "reflective" effect, thanks to the GIMP, but such is life. I couldn't think of anything better to put up there, and this may be the first time it's ever been used on a purple cow, so maybe I'm breaking new ground.

I think there is a science to certain aspects of layout design. For example should you have your links underlined or not? It's likely that many people (especially people who've been using the internet for a long time) associate underlined text in webpages with links. Having links that aren't underlined may slip people's notice; on the other hand, having text underlined that isn't a link may make people click around on non-links before they notice. It may only break someone's concentration for a split second, but sometimes that can still be annoying, especially if it happens lots of times. To that end I try to make all my links underlined, and nothing else. An exception is that category cloud thing on the right, because I figure people are probably smart enough to figure out to click there, and the underlines cluttered it so much that it was hard to read. In reality I have no idea what I'm talking about, but these kind of things probably matter.

To my amazement, someone actually expressed interest in my putting this layout up for others to download, so I plan to once I remove all the places I've hard-coded the thing to work with my website. It's not the prettiest code in the world, but PHP never is.

I've said it many times, and I'll never stop saying it: you simple can't go wrong with cow-based web page layouts.

Ruby, Rails, etc.

Wed, 25 Apr 2007 22:55:44 -0700

I tried out Ruby on Rails for the first time today. My spare time (what little remains of it) has been divided between playing with my websites, and playing with Ruby. So I figured why not combine the two.

It's somewhat infuriating when I can start from scratch and make something in 15 minutes in Rails that works as well as the huge mess of PHP crap that it took me over a week to write. My previous web host didn't have Ruby, so I was forced to use PHP by necessity. PHP4, of course. Blech. Is there anyone on this planet who uses PHP by choice?

I briefly toyed with the idea of using Smarty. It seems like a nice templating system, with its only weakness being that you have to use PHP to use it.

On a different topic, this week's Ruby Quiz was amusing. The problem is essentially to split a string in every possible way (with varying split length, with some upper bounds on split length), transform the parts, recombine them, and then print or process the result. A recursive solution is very straightforward: split the string into a little left part and a big leftover right part, transform the left part, then recursively process the right part in the same way. I wrote up a solution that's about 20 lines long. But it's slow as heck.

Reading other people's solutions, I noticed a neat thing about recursion in Ruby. There are two different ways of doing this recursively. One way (the slow way and memory-intensive way, the way I did it) is to recurse all the way to the base case (run out of leftovers to process), leaving your transformed bits of string scattered all over your call stack. Then return all the parts back up the call stack putting them together as you go. When you've hit the very top of the stack again, you'll have a huge accumulated list (array) of all the results.

But another way is to pass the partly-processed and unprocessed parts of the string both down the call stack as parameters in your method; each method call eats a bit of the unprocessed part and adds a bit to the processed part and then both are passed along again. The base case is once your unprocessed part is empty. If you also pass a Ruby block along as you recurse, then when you hit the bottom of the call stack, you can yield your accumulated fully-processed result to the block. This way you don't have to collect all the results into an array before returning them. You can yield the results as soon as you compute them, one-by-one. I thought it was neat.

PHP design patterns?

Tue, 23 Jan 2007 18:44:48 -0800

As one of my professors used to say, "There's no problem that can't be solved by adding another layer of abstraction". He was joking, but there's a little bit of truth to that.

I've been working on my other website for a while. It's all PHP4, not entirely by my choice but I'm limited what can be installed on my server. PHP4 turned out to be a lot more powerful than I'd initially thought. Namely the OOP options available are mostly adequate and in a few places surprisingly OK. PHP has some nice meta-programming sorts of functions like get_func_args() and call_user_func() which let you play around with doing weird things dynamically at runtime, which were almost essential in putting all this crap together. It lacks the completeness or elegance of Ruby, and PHP's syntax is generally horrific and painful, but it'll do.

I recently read Head First Design Patterns which is quite a good book. I was able to use plenty of what I learned there in designing a sort of templating system for my other website. Basically I have tons and tons of tables of different sorts of data. CSS goes a long way to making the presentation of the tables easy to change given that they are properly marked-up, but actually marking them up properly and consistently was the problem.

So I put together a bunch of classes that generate HTML tables. One of the things from Design Patterns that really makes sense when you hear it is to lean toward composition rather than inheritance. In other words, an object that has another object as an instance variable is often easier to deal with than an object that inherits the same functionality from a parent class; the reason being that you can use polymorphism to your advantage, and have your object contain a member of a GENERIC type, and then you can give your object an object of any subtype of that generic type you want. So your class then does not depend on the type of its instance variable at all, so long as all possible objects that end up in that variable have the same interface. Loose coupling and all that.

I have used this to great advantage in adding flexibility in marking up table cells. A cell will be a different class (actually a different composition of lots of classes) depending on the kind of data it stores. A TextCell holds text, so I know to pad it and format it (differently than an ImageCell, or a cell containing an unordered list). I can composite TextCells with other more specific types; a value that represents a percentile is a PercentileCell; the class handles formatting the numeric value and sticking a % on the end. I could composite it with an ImportantCell which will give it a special background color. It's as simple as $cell = new TextCell( new ImportantCell( new PercentileCell( 56 ))); The nifty thing is that I can composite the cells arbitrarily in any order or combination, because each simply holds a generic "cell" member and calling a method on an object only causes it to call the same method on its child object, all the way down the stack. Also, if I later decide to take the % off the end of all my percentile values, I can do it by changing a single class definition. That's the kind of thing you can't handle well with HTML/CSS alone. Before, it'd take me a month and half to go mucking around in static HTML files fixing it all. (Or an hour worth of Perl regexing, but still, no fun.) Or if I want my HTML to validate as XHTML Strict, I can change the HTML for all my tables at once by screwing with the base Cell class. If I want to turn it into XML or some other markup, I could do that too, in theory.

At this point I'm essentially writing most of my pages entirely in PHP, never needing to write any raw HTML by hand at all, which is a very nice change of pace. I wish they'd have taught more of this sort of thing in college. We learned how to write searching and sorting algorithms until blue in the face. But we never really touched on how to design software at a very high level. I still think I'm not too good at it in many ways, but I'm learning, at least.

PHP makes me sad; programming makes me happy

Fri, 24 Nov 2006 20:21:40 -0800

My other site is about Final Fantasy (the original video game of the series, for the NES). That site is how I learned HTML, a good 7 years ago. It's still my baby and (perhaps sadly) one of what I consider my greatest accomplishments. I wish I had more time to work on it nowadays, but such is life.

Oh how the internet has changed. I started out on that site with plain old static HTML files. It used to take me months to do or change anything. Eventually I learned about server-side includes and I started using those. A few years later I delved into PHP and started using it as a more sophisticated "include" mechanism and templated a good bit of the site.

Fast forward to today, and I'm trying to get most of my data in a mysql database. I learned a lot from work about how to properly design a database. That part I think I did pretty well at, on my FF1 site. The PHP frontend is where I started having problems. PHP is just such an un-fun language to work with. I hear PHP5 is nicer than PHP4, but PHP4 is what I'm stuck with. At first I tried a purely functional solution, and it very quickly become such a tangled mess of spaghetti code that it was nearly impossible to use. Today I rewrote that crap using the extremely weak OOP constructs PHP4 has to offer, and weak though they may be, my code is ten times shorter and probably ten times easier to read.

Sometimes I'll be sitting here banging out code, and it will be not working. The grunt-work of programming is debugging. You write something you think will work, and 10 times out of 10, it will fail the first time. So you find the first bug, fix it, rerun it. Failure. Find the next bug, fix it, run it, failure. Find, fix, run, failure. And you repeat this over and over and over. But eventually you hit that point where you find the very last show-stopper bug, and you fix it, and run your program, and it all of a sudden it WORKS. You almost never expect it to work. Success flies out of nowhere and hits you in the head, and it's really exciting. You get screen after screen of delicious output, all thanks to that last little tweak. That's the moment that makes the slog of computer programming all worth it.

Lisp, part 2

Mon, 14 Aug 2006 00:22:47 -0700

Everyone I know who does Lisp says "Learn Lisp and it will change how you look at all other programming languages!" I don't know if that's true or not, but I think I'm starting to learn a bit.

Chapter 8 of Practical Common Lisp discusses macros. As has been pounded into my brain by the author, macros in Lisp aren't like macros in other languages. Nosirree. Nope. Not like them. I definitely won't forget that Lisp macros are different than macros in other languages. And you won't forget that either. If you read this book.

Macros in Lisp let you write code that writes code. Quoth one of my professors in college, "Code that writes code is some of the neatest code there is!". I'll agree with that. Writing a whole database or a whole test unit framework in Lisp in 50-odd lines is pretty impressive. Lisp doesn't even seem like that concise a language, compared to Ruby for example. But it's powerful. I'm reminded of Haskell, where everything is recursion. When you start layering recursion on top of recursion, things tend to branch out and get "big" very quickly; that "bigness" equals power and efficiency, if used properly. That seems to be one of Lisp's strengths; it lets you do that pretty easily.

Well, a lot of languages let you write "code that writes code". Perl has eval() which can take a custom-built string of code and execute it somewhat dynamically. The thing is though, eval sucks. If it dies, it dies at runtime, not compiletime. And you can only put things together via string manipulation. Lisp's code-building macros are more "built into the language", for lack of a better term.

Ruby lets you do some neat things too though. I found this article to be a pretty good description. Ruby lets you throw blocks of code around all over the place, and the more I learn of Lisp, the more I'm reminded of Ruby, or the more I think of ways I could be doing things Lisply in Ruby. Ruby has the added advantage of letting you do all kinds of horribly powerful things at runtime. Chopping up classes and methods and rebuilding them as you see fit, etc. The book hints that this is something Lisp lets you do too, since so much of Lisp is actually written IN Lisp.

To draw a nice contrast, today I was forced to do a lot of PHP programming. Oh sweet Jeebus. It burns. I hate PHP with a passion. Lisp is wonderfully consistent. The parentheses do get on my nerves, but there's never any doubt what your syntax should look like at a given time. It's always parentheses. PHP on the other hand is both very verbose, and very inconsistent, which is the worst possible combination. It's never predictable or discoverable. It's impossible to program PHP without PHP.net open in another window to constantly look crap up.

Any time I'm forced to use a traditional for() loop now, I start freaking out. That is something Ruby and Lisp both thankfully avoid. Ruby's iterators are my best friend. Looping through an array copying things to temp variables seems so barbaric now that I've seen alternatives. It's just one of the little things that gets to you; in general in PHP I feel like I'm trying to build a scaffold with pieces that can't be cut to the right size. Ruby gives me pieces that bend easily, and Lisp gives me really tiny pieces that fit together in all kinds of interesting ways.

IE-b0rker returns

Mon, 05 Jun 2006 22:27:55 -0700

Now complete with large annoying red border. Here's a screenshot of Opera pretending to be IE.

I'm using this code to detect IE browsers:

<?php if(strstr($_SERVER['HTTP_USER_AGENT'],"MSIE")) { ?>

If anyone knows a better way, I'd love to hear it.