briancarper.net (λ) (Tag: Wordpress)

Lame comment spam management that works

Sun, 06 Dec 2009 02:34:08 -0800

It's been nine months since I ditched Wordpress and moved to a blog system I wrote from scratch (in Clojure). This was a great move in so many ways. One of those ways is comment spam. My site is as popular now (or maybe slightly more popular now) as it was when I was running Wordpress, so I think comparing before and after is valid.

With Wordpress, every morning I'd do the ritual of deleting overnight spambot droppings. Typically I got between 1 and 5 every night. I had a default Wordpress install and all I used for spam filtering was Askimet. Askimet did a surprisingly good job, catching literally if not thousands of spams every week which otherwise would've been ruining my site. But inevitably some would still get through. And what's worse, there were a lot more false positives than I could tolerate.

Since I started counting with my new system, which is around 6 months, to the best of my knowledge I've gotten zero spambot-produced comments that made it through my filters. This is pleasant, to say the least.

The system I'm using is stupid. None of it is stuff I thought of myself, I got ideas from other lots of other blogs or articles I read, but the implementation is mine and it's not sophisticated. It would take a bot author a few seconds to work around it. But no one has bothered. Why bother writing a bot for my one-man blog, when you can write a bot for Wordpress and have it work on tens of thousands of blogs? And I can change my system to defeat the bots with a few lines of code just as easily as they can work around it.

So here's why I think it's working.

1. It's not Wordpress

Just by using something slightly different from Wordpress, I'm think I'm already ahead. For example if you have a blog where a form posts comments to /wp-comments-post.php, a bot doesn't even need to look at your site to spam you. They can blast your server with POST data at that URL in a format they already know Wordpress will accept. My site is all custom code, so everything is different enough that default bot attempts fail immediately.

I think this is the reason that only 1853 spam comments have even been POSTed at me in the last six months. That's an improvement of one or two orders of magnitude already.

2. Honeypot text field

So what about the comments that are actually POSTed? They are presumably the result of bots that parse sites' HTML looking for comment forms and try to POST data that satisfies the form.

So in my comment form I have a field called referer. A "How did you find my site?" kind of thing. In fact I don't care how you found this site, this field is a honeypot. The div containing this field is hidden via CSS.

div#referer-row {
    display: none;
}

So you shouldn't ever see it if you're a human using a browser. But bots parsing the HTML see this field, unless they also bother to parse my CSS and see that it's hidden, which would be expensive and apparently they don't do it.

If you put anything in this referer field, your comment will be rejected as spam. Simple enough.

Many blogs require you to fill in every field or else the comment is rejected, so it seems reasonable to expect most bots to fill in all of your form fields. (My blog actually requires you to fill in nothing but the comment text; author will be set to Anonymous Cow if you don't fill it in.)

In fact this seems to be the case; of 1853 spam comments since March, 1810 put something into this field. Most of the time it's a random string of letters. Not even a URL. Sometimes it's a couple words like "insurance quotes" or something about drugs or casinos.

The downside of this is if you're a human using a browser that doesn't understand CSS, you will see this field. Then if you type something into it and try to comment, it'll end up as spam. So Lynx users and time travelers from 1987 trying to leave me comments might be confused at first.

However as far as I can tell, no intelligible data has ever been entered into this field by a human, so I don't think it's a concern. Six times, the word "None" was entered, but I don't think this is a human because that's nonsense answer to "How did you find this site?". But you never know.

3. Lame static CAPTCHA

That leaves 43 spam comments that made it this far. My other anti-spam measure is a word you have to type. But it's always the same word, and the word is COWS. This CAPTCHA caught the remaining 43. It looks like this:

There's a normal text field with a default value of <= Type this word specified right in the HTML.

<input type="text" value="<= Type this word" name="test" id="test"/>

There are no other instructions besides "Type this word". I'm assuming either that commenters are familiar enough with CAPTCHAs to know what I want, or can figure it out using common sense. Given that my target audience is computer geeks and programmers, this should be a safe assumption. In fact I've had less than a dozen false positives in the past six months via people failing this; see below for details.

To post a comment, the value of this field must contain the word "COWS" somewhere in it, case-insensitively. Otherwise it's spam. Easy enough to implement.

If you have Javacsript enabled, clicking on this field will clear out the default value. If you unfocus the field without typing anything, Javascript will put the default value back in. This is only for convenience. If you don't have Javascript enabled, you have to highlight and backspace over the default text. I don't think this is a huge burden.

Of 1853 spam comments, here's the breakdown for what values end up in this field.

1131: "&lt;= Type this word"
691:  "<= Type this word"
21:   Random letters and numbers
6:    empty
2:    A bunch of URLs
2:    Human beings making typos, e.g. "COW" or "COS".

So most bots are too stupid even to remove the default value from the field, and none of them entered the correct value. 691 times the bot was somehow smart enough to un-escape < into <, which is interesting, but didn't help it defeat the filter. A lot of the random words look like they were made by Markov chains, e.g. 'fridwolfur' and 'lyndonvolk' and 'calbertdom'. If I need to write a childerns' poem I'll know where to look for ideas. One time a bot or spammer managed to type "vows" somehow, but this might or might not be coincidence.

I think this is better than a normal CAPTCHA because:

It's always the word "COWS" in a normal font, so it requires no thought or eye strain to figure out.
It's black and white, so hopefully people with minor vision problems and color blindness can see it.
It fits thematically with my blog layout (it's COWS and it has cow spots).
It's kind of silly, so hopefully people chuckle rather than become ticked off.

It's worse than a real CAPTCHA because it requires no effort to break. So it wouldn't work for Wordpress or VBulletin or something with a million users. But I wonder if every Wordpress had a single static word as a CAPTCHA, but a different word for every blog (generated at install-time maybe), would it work better or worse than the random mangled multi-color CAPTCHAs no one can read? Real randomly-generated CAPTCHAs don't work anyways; bots can already beat them via OCR or other means. A simple word would be less annoying for a human, to be sure.

The other downside is that this is not very accessible to the blind or other people using screen readers, or browsers without image support. This is unfortunate and I'm still trying to figure out how to get around this. Right now the ALT text for the CAPTCHA image is This says 'COWS'; I don't know if this is enough help for people in those situations.

Of course I'll never know how many people see my CAPTCHA and storm away in a rage without even trying to post a comment. But I've never heard a complaint. If this level of CAPTCHA ticks you off personally, please swallow your anger and leave me a comment here saying so, if you feel obliged; I'd love to hear it.

False positives

As best I can tell, there are no false positives from people filling in the honeypot field. But even as simple as it is, some people don't succeed at the CAPTCHA image. Either they typo it or they ignore it entirely.

I just checked and I counted around 6 comments by real humans where the CAPTCHA was ignored and the default <= Type this word ended up in the spam DB. 4 of those people re-posted their comment successfully immediately afterwards by filling in the CAPTCHA. I'm not sure I'm ever going to get much better than that.

Spam that makes it through

I have still gotten spam. Maybe a dozen or so in the past six months. It's all been in the form of a human typing a normal-looking and relevant comment, about open source software or BASH for example, but with a spammy URL buried in it, e.g. a link to a really dodgy-looking blog trying to sell something, or some scummy SEO site. It's either a human or a very sophisticated (or lucky) bot; the comment text in these is indistinguishable from a real comment other than the spam URLs. I have to delete these by hand.

But I was getting these with Wordpress too. No automted anti-spam system is going to defeat a human being, so I don't worry about it.

That's it

The moral of this story is that it doesn't take much to protect yourself from comment spam if you write the code yourself. As long as it's unique, you'll probably be fine.

The other moral is that you don't have to annoy the hell out of your users to filter spam effectively. I'm making the assumption here that my COWS method is not that annoying; tell me if I'm wrong.

I don't know how well this scales. Probably not so well. My blog isn't that highly trafficked. If my site were more popular it might be worse for me. But the improvement over Wordpress is unquestionable.

I've seen all kinds of complicated measures suggested elsewhere, like trying to predict if it's a bot by how many milliseconds it takes between page load and comment posting, or measuring keypress speed, or escaping the HTML of your forms and un-escaping it at loadtime it via Javascript, or setting and retrieving cookies and such. But a lot of this stuff seems fragile and if your browser doesn't suppoort Javascript or cookies (or your users block them), you're screwed. I block these things myself, so I expect visitors to do the same.

If everyone wrote their own blog engines, the world would be a slightly less spammy place. Or else we'd have much smarter bots.

Clojure 1, PHP 0

Mon, 16 Mar 2009 18:50:00 -0700

Goodbye Wordpress

As I mentioned many times, I've been working on replacing Wordpress for my blogging needs. Wordpress has been pretty good for the past three years, but it's time to move on, for a bunch of reasons.

Primarily, the way Wordpress automatically mangles my text is annoying. For example, it turns newlines into paragraphs inconsistently (especially when it comes to pre/code blocks). This blog is mostly about programming, which means being able to post code without having my quotes turned into "smart" quotes and my --flags turned into long-dashes is kind of important. HTML is sometimes automatically escaped, and sometimes not. I can't count how many comments I've gotten where someone posted some code, then posted again to inform me that Wordpress ate the code for dinner. There are plugins to fix some of this, which break every time Wordpress releases a new version, and have never really worked that well for me.

Writing a theme for Wordpress means a mix of PHP and HTML and CSS, which is painful to read and even more painful to write. Aside from the considerable ugliness of PHP itself, there's a lot of weird magic involved with themes, based on naming conventions for files, weird fall-through behavior when certain theme files aren't present and so on. The Wordpress API is enormous and not fun to work with if you want to do something other than the standard Wordpressy kind of blog structure. Static pages aren't too much fun to work with in Wordpress either.

Lately I think I was getting hammered with spam partly because Wordpress is such an easy target. Askimet is nice but it wasn't catching enough lately; maybe 10-15 spams per week were slipping through. And there was always the chance that some widely-known exploit in Wordpress was going to leave my site susceptible to some roving bot.

And so on.

Hello Clojure

Why Clojure? Because it's awesome and fun and powerful and I wanted to learn it better.

Compojure is a web framework for Clojure that made a lot of this very easy. Coming here from a Ruby on Rails background, Compojure has a lot going for it in comparison. Compojure is lightweight and more low-level than Rails. For example Compojure doesn't enforce MVC on you, doesn't force a unit testing framework on you, and doesn't care how you access your data. Compojure just lets you route HTTP requests to Clojure functions based on the URL and request method (RESTfully: POST/GET/DELETE/PUT), and it gives you easy access to the request information, session, GET/POST parameters and cookies.

Under the hood it's all servlets and Jetty, both of which are solid, stable, well-tested, well-documented technologies. However, thankfully, all of that Java stuff is under the hood, and well under it. I didn't have to write a single line of Java or interact with single servlet directly. Everything (session, params, headers) is a Clojure hash-map from the perspective of my code.

Compojure also comes with a domain-specific language for writing HTML, which is similar to CL-WHO and myriad other Common Lisp HTML DSL's. All of which are awesome. I can't say enough how much nicer it is to write (or generate) structured s-exps than to write HTML by hand. More on that below.

Compojure doesn't come with any way to interact with a database, so I had to write one. clojure.contrib has an SQL lib which easily lets you interact with a MySQL database. (Clojure can talk to MySQL via MySQL's JDBC connector, of course.) I used clojure.contrib.sql to write a small (192 lines) library which slurps up a bunch of database tables into Clojure refs, and provides a few functions for basic CRUD operations so that any updates to the ref data is also transparently reflected in the database. The database is essentially only for keeping an on-disk cache of the data in case I need to restart the server. The average number of DB queries per page is zero; everything except posting/editing/deleting data just reads out of a Clojure ref.

With possibly multiple users posting data at once, it's nice to have Clojure's built-in concurrency support. Updating the data refs with new data is always safe from multiple threads simply by throwing a (dosync) around all of the write accesses. This was completely painless to write.

I decided I wanted to use Markdown for posting comments and authoring new pages. This was also very simple to do; I outlined how to get Markdown working in Java and Clojure, in a previous post. The real-time previews for comments are largely inspired by / ripped-off from Stack Overflow, implemented mostly using open-source Javascript libraries like Showdown, JQuery, TypeWatch and TextAreaResizer.

A Brief Comparison: Clojure vs. Wordpress

All of my code including the CRUD library, all of the HTML for the templates and layout, admin controls, and all the glue to put it together is 1,253 lines of code. Wordpress is somewhere over 78,000 lines of PHP depending what you count (doesn't include any themes or layout, but does include Wordpress features I didn't need and didn't implement). It's still a pretty nice reduction in code overall, any way you look at it.

As an example, in my old Wordpress site I had a plugin catcloud to generate a "tag cloud". This plugin itself is 226 lines of PHP, not bad. However, here's the Clojure code to generate a similar tag cloud (which you can see here currently):

(defn tag-cloud []
  (let [tags (sort-by #(.toLowerCase (:name (first %))) (all-tags-with-counts))
        counts (map second tags)
        max-count (apply max counts)
        min-count (apply min counts)
        min-size 90.0
        max-size 200.0
        color-fn (fn [val]
                   (let [b (min (- 255 (Math/round (* val 255))) 200)]
                     (str "rgb(" b "," b "," b ")")))
        tag-fn (fn [[tag c]]
                 (let [weight (/ (- (Math/log c) (Math/log min-count))
                                 (- (Math/log max-count) (Math/log min-count)))
                       size (+ min-size (Math/round (* weight
                                                       (- max-size min-size))))
                       color (color-fn (* weight 1.0))]
                   [:a {:href (:url tag)
                        :style (str "font-size: " size "%;" "color:" color)}
                    (:name tag)]))]
    (block nil
           [:h2 "Tags"]
           [:div.tag-cloud
            (apply html (interleave (map tag-fn tags)
                                    (repeat " ")))])))

This is 10 times less code, which is a good reduction in my opinion. Most of the code is the math to generate a weight logarithmically for each tag so they scale nicely. (all-tags-with-counts) fetches a seq of two-item pairs: the tags themselves (which are hash-maps) and a count of posts for each tag. There are two locally-defined functions in the let which generate the text color and the font size and HTML for each tag.

The vectors that look like [:h2 "Tags"] are input for Compojure's HTML-generating DSL; this would be transformed for example into <h2>Tags</h2>. (block ...) is a macro which wraps its content in HTML for the rounded borders of my layout. (Math/log ...) and friends are calls to standard Java math functions.

This whole function is less code than just the horrible boilerplate array declarations at the top of the Wordpress plugin:

$catcloud_field_data = array(
  array('name' => 'Minimum Font Size', 'option' => 'catcloud_min_font_size', 'size' => '4', 'maxlength' => '3',
       'default' => '9', 'note' => 'Used for the least frequent categories', 'validation' => '/^\d{1,3}(\.\d{1,3})?$/'),
  array('name' => 'Maximum Font Size', 'option' => 'catcloud_max_font_size', 'size' => '4', 'maxlength' => '3',
       'default' => '18', 'note' => 'Used for the most frequent categories', 'validation' => '/^\d{1,3}(\.\d{1,3})?$/'),
  array('name' => 'Font Face', 'option' => 'catcloud_font_face', 'size' => '15', 'maxlength' => '254',
       'default' => '', 'note' => 'Set an optional list of font faces', 'validation' => '/.*/'),
  array('name' => 'Font Units', 'option' => 'catcloud_font_units', 'size' => '3', 'maxlength' => '2',
       'default' => 'pt', 'note' => 'Choose one of em, pt, px or %', 'validation' => '/^(%|em|pt|px)$/'),
  array('name' => 'Color Start', 'option' => 'catcloud_color_start', 'size' => '7', 'maxlength' => '6',
       'default' => '0066CC', 'note' => 'For the least frequent categories. Use a hexadecimal RGB triplet. ie. 0066CC',
       'validation' => '/^[\dA-F]{6}$/i'),
  array('name' => 'Color End', 'option' => 'catcloud_color_end', 'size' => '7', 'maxlength' => '6',
       'default' => 'CC6600', 'note' => 'For the most frequent categories. Use a hexadecimal RGB triplet. ie. CC6600',
       'validation' => '/^[\dA-F]{6}$/i'),
  array('name' => 'Before Category', 'option' => 'catcloud_before', 'size' => '3', 'maxlength' => '20',
       'default' => '[', 'note' => 'Set the character(s) to display before category names', 'validation' => '/.*/'),
  array('name' => 'After Category', 'option' => 'catcloud_after', 'size' => '3', 'maxlength' => '20',
       'default' => ']', 'note' => 'Set the character(s) to display after category names', 'validation' => '/.*/'),
  array('name' => 'Show Top N Categories', 'option' => 'catcloud_top_n_cats', 'size' => '5', 'maxlength' => '3',
       'default' => '', 'note' => 'Show only the top N categories (where N is a number like 10 or 25 or whatever. Set to 0 or empty for no limit.',
       'validation' => '/^\d*$/'),
  array('name' => 'Excluded Categories', 'option' => 'catcloud_excluded_cats', 'size' => '15', 'maxlength' => '254',
       'default' => '', 'note' => 'A comma-separated list of category ids.',
       'validation' => '/^[\d, ]*$/'),
)

Ugh. As another example, here's the code that handles a POST request to add a new blog page:

(defn do-new-post []
  (check-login
   (let [post (add-post *params*)]
     (sync-tags post (:all-tags *params*))
     (redirect-to "/"))))

It does exactly what it says: Check to make sure the user is logged in, add the post based on the POST params, sync up the tags for that post and redirect to the front page. Lisp lets you say what you want very concisely, with a bare minimum of boilerplate.

How about speed? My Clojure code is actually generating HTML in the most brute-force and wasteful way possible. The HTML for each page is regenerated from scratch, via a cascade of a couple dozen function and macro calls, every time you load a page. But it's still pretty fast, a couple hundred milliseconds for most page requests. This is slightly faster than the Wordpress version of my site. If I ever have performance issues I can switch to another Clojure HTML library, like clj-html which uses the same vector-style syntax but pre-compiles the HTML.

How hard was it to set up on the server? Wordpress is pretty famous for being dirt-easy to deploy anywhere. My Clojure app by comparison was slightly more difficult, as you might expect, but it wasn't brain surgery. My server runs Debian. First I installed the JVM via apt, then I rsynced a bunch of jar's and clj files to the server, then I installed emacs and screen also via apt. Then I put two lines into an Apache config file to proxy-forward traffic to a local port where jetty would be listening. I started Emacs, did (require 'bcc.blog.server), did (bcc.blog.server/go) to start everything, and that's about it. Took about 15 minutes to set up from scratch. When I find a bug, I SSH in, re-attach to screen, fix it in Emacs, hit C-c C-c to recompile just the functions I need to update, and then detach from screen again.

I'm pretty pleased with this so far. It was fun to write and has all the features I used from Wordpress, plus more, and the building blocks are there to extend things if I imagine up a new feature I like.

Looks like my blog is still running today in spite of my predictions. Still waiting for the JVM to crash though, I know it's coming. I plan to post the source code for some of this once I'm sure it works.

New Blog... I think...

Mon, 16 Mar 2009 00:35:04 -0700

OK, here's the new blog. Apologies to anyone who may be following my RSS feed, because the whole feed is probably going to be reset by switching blog engines.

If you can call this an "engine". This is my Clojure rewrite. I'll have much more to write about this tomorrow when I'm awake. In the meantime, bug reports are welcome.

Here are my estimates:

52% chance the blog is crashed and down by the time I wake up tomorrow.
27% chance my feeble anti-spam measures are easily defeated, and hundreds of spam comments are waiting for me in the morning.
14% chance the JVM brings down the whole server.
7% chance everything works swimmingly.

I had to take down my origami gallery site just to get this to run. Fun times ahead.

When I came up with this blog layout I thought it was great, but three weeks of looking at it and now I'm starting to hate it. I can work on making it all pretty later though.

Ah well, more tomorrow. Keeping my fingers crossed.

Migrating away from Wordpress: permalinks

Fri, 27 Feb 2009 01:03:52 -0800

Work is progressing on my blog-rewrite in Clojure. It's been lots of fun and I keep adding features. Hopefully not too many features; the whole point of ditching Wordpress is that it's far too bloated. But my new blog already has categories and tags and pages with parent pages and so on and so forth. One of these days I'll actually start using it publicly, maybe.

One issue with migrating away from Wordpress is not to break all existing links that point to my Wordpress blog. Most people with Wordpress blogs (including myself) seem to use some date-based permalink structure, which I'd like to avoid.

I thought I'd have to set up some horrid mod_rewrite thing after the switch, to avoid breaking links, but actually Compojure's routes are powerful enough. Any request that looks like a Wordpress permalink, I pass to a redirect function which spits out some redirect headers to the new location. Simple enough, just a few lines of code of this sort:

(defn old-blog-redirect [name]
  (redirect-to (str "/blog/" name)))
(defservlet blog-servlet
  (GET "/2009/:m/:d/:name/" (old-blog-redirect (route :name))))

This redirects e.g. /2009/01/31/foo to /blog/foo.

Focusing so much on dates is kind of silly. A lot of blogs have a sidebar with a little calendar, or have a list of links to archives of all their posts by month. How useful is this really? Never once have I read someone's blog and thought "Wow, nice post. I wonder what this guy said in November of 2007. Good thing there's a link right there on the front page!"

Does anyone really care enough about the date something was posted, that the date needs to be encoded in the URL? The way I see it, the only thing people are going to use a date in a URL for is to say "That's too old, so I'm not reading that". How many people are going to be persuaded to read your blog by noticing (based on the URL) that a post is brand new, where otherwise they wouldn't have clicked the link? I have to think not many.

Blog replacement fun

Thu, 15 Jan 2009 21:55:11 -0800

So I'm still thinking how I want to replace this blog. I still plan to write something from scratch, for fun's sake.

One thing I'm sure of is that I don't want to write HTML by hand, at all, under any circumstances. HTML and XML are not human-writable or human-readable languages. They rely too much on things human beings suck at, namely consistency and repetition. Forget a closing tag? Typo a tag name? Now your document is malformed. Undefined behavior, at best. It's too verbose, it has too much needless punctuation.

It's also too hard to manipulate it or do anything with it after you write it. There's XPath, which is itself a mess to work with, manipulating huge strings of crap via slightly smaller strings with its own funky syntax quirks. I've never found an XML-parsing library with an interface that I liked, and I've had to use them extensively in Perl, Ruby and Python.

So first thing, I'm going to convert all posts and comments into Markdown and use that for future posting and commenting. I like Markdown. It's hard to get wrong typing it by hand and it doesn't get in your way. It also doesn't tie you to one implementation; you can turn Markdown into HTML client-side via Javascript or easily parse it server-side. Or you can display it as-is and it's still readable. It's a very nice idea.

Second thing, I plan to use my programming language to write the HTML for the skeleton of my site for me. Opening, closing, and properly nesting tags is something a machine should do for me. Making sure my tags belong to a well-defined list of allowed valid HTML tags is something a machine should check for me.

More than likely I'm going to write this in Clojure, because s-expressions (and better yet, a combination of Clojure literal lists, maps and arrays) makes writing HTML very easy and foolproof. I've also written an HTML-producing DSL in Ruby in the past though; it's not hard to do in any language.

Another thing I'm sure of is that I need a good anti-spam system but that I have no idea what that system should be. Askimet in Wordpress has caught 50,000(!) spam comments since I started my blog. Some spam still sneaks through on me now and then. I've never used a CAPTCHA and don't plan to; they just don't work. I'm probably going to come up with some funky custom anti-spam measures (which are invisible to users) and rely on the fact that no one is going to take the time to break it. My site isn't a huge or popular target, so here's hoping.

A third complication I'm dreading is how to do this without breaking every link anyone ever made to my site. Wordpress's permalink system is OK, but I'd like to change it. Problem is I can't change it; every link to my site from another site is a dependency. So I might have to mod_rewrite redirect the old URLs to new ones, or use two permalink schemes simultaneously. I don't know.

Fun times ahead. How to design a blog is a problem lots of people have solved but no one has really solved perfectly, or else there wouldn't be so many frameworks and packages to do it. The good thing about writing your own from scratch is that it'll work exactly how you want. Wordpress is close but not close enough.

Wordpress is no good for programmers

Fri, 05 Dec 2008 17:01:35 -0800

Wordpress is a good least common denominator when it comes to blogging. It's good for someone to throw some PHP scripts on a random server and have a basic blog running in a few minutes with little configuration.

It's no good for programmers. For me in particular anyways. Reasons:

PHP, yuck. Blogging is something I do purely for fun. PHP is not fun. PHP is a horrendous, sometimes-necessary evil to be avoided if at all possible.
Text mangling. Quotes are turned to ~~smart~~ dumb-quotes. Newlines are mangled into paragraph blocks. wp-includes/formatting.php has 1200 lines of mostly regular expression replacements. Getting text to show up literally is sometimes hard. People post comments and sometimes Wordpress eats them for dinner. I get complaints all the time.

One of the most important things for a blog like mine is the ability to post plaintext and not have it altered in any way. This is possible if you hack Wordpress enough, or take your chances with various plugins. I've never had success with those kinds of plugins or with plugins of my own. Wordpress changes too often, the plugins break, or there are one or two hooks that are overlooked, or one or two regex-replacements I forget to undo. * Bloat. The only features I use are the ability to post text, categorize it via tags, let people comment on it, and let people browse archives of posts. Spam filtering is nice too. These are not difficult tasks. Wordpress adds all kinds of of baggage on top of that, most of which I never use. Pings? Trackbacks? Blogrolls? User registration? Draft posts? Private posts? File uploading? Crappy WYSIWYG in-browser text editors? Plugin systems? I don't need that stuff. * Permalinks are handled by Apache mod_rewrite rules. Yuck again. There are better ways.

So I'm thinking of migrating away from Wordpress. Which is probably going to be a bit painful if I want to retain all of my current posts, and avoid breaking every link anyone ever made to my site. But it's doable.

What to migrate to? I'll probably write something myself. It's usually easier to write something yourself than to hack up someone else's stuff to get it to work right.

White text? Black text? Cow text?

Mon, 18 Feb 2008 16:13:06 -0800

I took a screenshot of my blog and went into Gimp and did Colors => Invert and thus a new blog layout was born. I also brought back the purple/green one. You can change it via the little skin-selector drop-down thing that's hopefully showing up and working properly for everyone. Skin selection is courtesy of a WP plugin; that site is not in English, but the instructions in the download are, if you want to use it yourself.

Black text on white vs. white text on black... the age-old question. My Vim theme has forever been a black background (ps_color to be specific). Even in broad daylight I find that a black background reduces eye strain considerably. Or maybe it's all in my mind, but then again this is a subjective sort of thing, so whatever's in my mind is all that really matters isn't it?

It's notoriously difficult to use a dark-background GTK/QT theme. Too many programs are written with the assumption that your theme is going to be light backgrounded. However thanks to Kore and a few tweaks here and there I've been getting along pretty well for a few months with a dark theme in KDE. I really need to start posting desktop screenshots more often again. Note to self.

So what's up with the cows? Cows are big, dumb, silly beasts. They can represent strength, or embody vulnerability. They're so disgusting that it somehow wraps around again to awesome.

Are cows really dumb though? Does their silent cud-chewing indicate stupidity, or thoughtfulness? Are cows really silly? Or do we project our own latent silliness onto them? Cows thus embody some of the deepest philosophical questions man has ever dared to ask.

Not really. I've been told by various people that I have the kind of sense of humor where it's impossible to tell whether I'm joking or being serious. Sometimes even I can't tell whether I'm joking or not. I love walking that line. Cows are partly a joke that I never get tired of telling, but also they really do make me smile. Cows are a way to have fun with this website. I view my website almost as a sort of parody of a blog, but a parody I still take seriously in a way. I believe it was Friedrich Nietzsche who said:

If you look long enough into the cow, the cow begins to look back through you.

The internet does not have to be serious business, and I don't want my website (or my opinions) to be taken as seriously as many people seem to want to. My secret hope is that whenever someone comes on here to flame me about my opinions, they'll look up and see a cow in a fedora and say "Wait a second... what am I doing?"

Also Gentoo's mascot is a cow. I estimate that the cows on my website increase its overall performance by 14%.

New blog layout (now with more cows)

Mon, 18 Feb 2008 04:13:19 -0800

"OK, what's up with the cows?", you might ask. Actually no one ever has asked. So I'm not saying.

I'm experimenting a bit with a new blog layout. It's black text on white rather than my customary white text on black/grey. I drew the cow in the top right using Inkscape. (Inkscape is such a wonderful program. So easy to use. )

I put this layout together from scratch in exactly 4 hours (not counting cow-drawing time). Just goes to show how easy it is to make Wordpress themes I guess.

I've gone from a sort of three-column layout to a one-column layout. This is partly / largely because I post source code snippets, and I need the full width of the screen for those.

Source code should generally go into PRE tags to preserve whitespace. However that usually has the side effect of preventing text from line-wrapping. In many blogs with multi-column layouts, source code snippets overrun into the sidebar area, and either end up being hidden under the sidebar (thus unreadable), overflow over top of the sidebar (thus looking messy), or if you're lucky, the PRE element itself is side-scrollable like a mini frame (which is kind of annoying). In a one-column layout, I get to use the full width of the screen, which is much nicer in my opinion.

The other reason I like one-column is that the sidebar columns in a multi-column layout invariably have just a few links at the top, and then you have a 200-pixel wide column of blank space all the way down the page to the bottom. It's a bit of a waste. This layout on the other hand is still fairly readable even if I resize Firefox to 400 pixels wide.

Do users really USE most of the links webmasters put into their sidebars? If people want to look at my archives, they can probably go to an archives page. It seems almost absurd how much we try to cater to people's whims and impatience by loading up websites (and computer interfaces in general) with every single thing anyone might want to do on every single page. It ends up looking cluttered and most of those links are probably unused the vast majority of the time. I wish I know how many times anyone ever used the tag cloud I had displayed in my sidebar for so long.

If anyone has any suggestions about things that would make this layout easier to read or use, or if you loathe the new layout (or love it, I guess that's a slim possibility) feel free to leave me a scathing comment. I've tested it in Firefox 2 / 3, IE7, Opera and Konqueror. IE6 can die in a fire.

nofollow

Thu, 04 Oct 2007 15:51:25 -0700

Someone recently mentioned that author comment links in my blog were all given a nofollow attribute. So I wrote a plugin that removes nofollow from comment author links. If you search around it's not hard to find a million other plugins that do the same thing. Putting something like this into /wp-content/plugins (and activating the plugin) should probably do it:

<?php
/*
Plugin Name: Remove nofollow
Plugin URI: http://briancarper.net
Description: Remove 'nofollow' from comment author links
Version: 1.0
Author: Brian Carper
Author URI: http://briancarper.net
*/

function remove_nofollow($text) {
    return preg_replace('/ rel=\\'.*?\\'/', '', $text);
}

add_action('get_comment_author_link', 'remove_nofollow');

?>

There is apparently some amount of controversy about nofollow. Who woulda thought. I personally don't see much need for it. Links are links. If I didn't want links to be followed, I wouldn't post them or let other people post them in the first place.

Wordpress theme download (Cow 1.0)

Sat, 22 Sep 2007 10:34:29 -0700

I put my old Wordpress theme up for download as requested. Enjoy. I think.

Wordpress - Cow

Sat, 22 Sep 2007 10:29:17 -0700

Download Cow 1.0 Wordpress theme.

Updated cows

Thu, 13 Sep 2007 15:30:46 -0700

I've read that good website layouts evolve slowly over time, and it's better to change and improve the layout you've already got rather than redo the whole thing from scratch. Probably sage advice. I tweaked my layout a bit to take more advantage of a bunch of empty space where nothing much was happening, and also to add more cow. My girlfriend drew me a cow silhouette for my blog layout. (By the way, when your girlfriend is willing to take the time to draw you a cow graphic for your blog layout, you know it's true love. <3) I used a sort of cliche'ed "reflective" effect, thanks to the GIMP, but such is life. I couldn't think of anything better to put up there, and this may be the first time it's ever been used on a purple cow, so maybe I'm breaking new ground.

I think there is a science to certain aspects of layout design. For example should you have your links underlined or not? It's likely that many people (especially people who've been using the internet for a long time) associate underlined text in webpages with links. Having links that aren't underlined may slip people's notice; on the other hand, having text underlined that isn't a link may make people click around on non-links before they notice. It may only break someone's concentration for a split second, but sometimes that can still be annoying, especially if it happens lots of times. To that end I try to make all my links underlined, and nothing else. An exception is that category cloud thing on the right, because I figure people are probably smart enough to figure out to click there, and the underlines cluttered it so much that it was hard to read. In reality I have no idea what I'm talking about, but these kind of things probably matter.

To my amazement, someone actually expressed interest in my putting this layout up for others to download, so I plan to once I remove all the places I've hard-coded the thing to work with my website. It's not the prettiest code in the world, but PHP never is.

I've said it many times, and I'll never stop saying it: you simple can't go wrong with cow-based web page layouts.

Comment subscription

Wed, 12 Sep 2007 13:43:35 -0700

I added the option to subscribe to the comments for any post in my blog (to receive an email when new comments are posted). You don't have to post a comment yourself to subscribe. This was accomplished via this Wordpress plugin.

Web design still sucks (or maybe I do)

Thu, 28 Jun 2007 07:32:33 -0700

I decided once again that I need a new blog layout. It'll be coming in a day or two. This current one is too big and clunky. I was looking over my previous layouts from over the years, just now. So much time I've spent making those things, and then I throw them away and make a new one. I need to resurrect a few old ones and find some kind of Wordpress theme-switcher so people can use whatever they want. Or just write a few lines of Javascript myself that does it.

Web design still makes me sad. If for no other reason, than that I'm bad at it. I've read many times that programmers don't make good interface designers. It's probably true in my case. And I googled for "programmer blog", and looked at a few of the syndicated blogs I read, and (no offense to anyone reading this) they aren't top of the list in artistic flair. Though they do tend to be pretty functional, meaning not too hard to navigate and fairly easy to read, which is good. But they're on par with mine in the looks department.

I can't seem to move past doing things really really simply. My designs tend to be monotone. I pick a color and then reuse the heck out of it. Usually I end up resorting to black and grey and white. I think back to art classes where we studied color theory and complimentary colors etc., and I've tried to do that before, but I can't make it work. I've also looked read articles on typography, whitespace utilization, page flow, serif vs. sans serif fonts, and so on and so forth. But that kind of academic knowledge doesn't really help me draw a nice picture, at the end of the day. I don't think you can learn to create art by analyzing it scientifically, which is probably what I'm doing.

When I design a layout my primary concern ends up being that my HTML looks nice, and the website itself looking nice is next on the list. I pick a color, and when I need another color, I do some kind of transformation on the hexadecimal color code. I usually pick symmetric color codes or colors with easy codes like #333333 or #aabbdd, so that it's easier for me to code with them. I pick layout sizes that are even 100 or 50 pixel increments. That's the kind of thing that really screws me over when I try to make a layout, I think.

In any case, sometimes it's still fun to dabble in the horrifying world of graphic design.

Even more cow

Sun, 03 Dec 2006 22:26:35 -0800

I updated my layout to be even more cow-like. My girlfriend made me a cow-themed top banner which I think looks good enough to eat. I often wish I had any kind of graphic design skill, but I don't. Not nearly as good as her, anyways.

It's always so hard to find a good balance between leaving something readable, and making it look good and catch the eye and stand out in some way. As much of a geek as I am, even I will probably pass by a plain-looking site unless I know beforehand that there's something on it that's worth reading. Putting some effort into making your site look nice shows that you actually give a crap about the site, I think. And the more you give a crap about the site, the more likely there is going to be something worth reading on it.

Wordpress impressed me today. I remember a long time ago making a calendar (to list your post archives) was extremely painful. Now you can do it with a single get_calendar call, and it looks fine and works fine. I like relying on Wordpress for this kind of crap because chances are they already worked out what works in IE and what doesn't.

Which brings me to my next point.... dear God, IE sucks. If you're looking at this site in IE right now, and you're using a screen resolution 1024x768 or lower, you'll notice that my site is so wide that it forces a horizontal scrollbar. This isn't so in Firefox. In IE though, the existence of pre tags apparently causes everything to stretch, even when I indicate overflow:auto or overflow:scroll.

And IE interprets border: 1px transparent solid to mean border: 1px BLINDING_WHITE solid. It makes you cry, after a certain point. Also my top header image is a transparent png which looks like garbage in IE, but I'm not fixing it.

Whenever IE drives me into a murderous rage, I just pull up my site stats and suddenly everything seems right with the world:

I hate the internet.

Wed, 01 Nov 2006 00:23:25 -0800

I use Wordpress for my blog. A database is used to store the posts themselves. Wordpress uses PHP to move data into and out of the database. Obviously the blog itself is displayed as HTML.

This means there are potentially three levels of escaping and un-escaping done to my text after I type text into this lovely TEXTAREA and before it's actually displayed. It's HTML-escaped in certain cases, like turning < into < (I just typed a raw < character there, note, and it was HTML-escaped automatically); it's SQL-escaped so it doesn't break whatever INSERT command is putting it into the tables; it's PHP-escaped along the way I'm sure. All my newlines are magically turned into <br> tags or <p> tags somewhere along the line too. Etc. etc.

And yet, if I type a backslash in my post, by the time it's fed through PHP, into the SQL table, fetched back out, and displayed in my blog, it will have vanished entirely. This is a problem when I post source code in my blog, for example, where backslashes are pretty common. Where along this long line of escaping and un-escaping of text is my poor backslash lost? I don't know.

Only ecently have I discovered that putting code into <code> tags will cause Wordpress do to even more escaping of special characters than usual, so that my backslashes DO survive the round trip through the system. Little did I know that, unlike in <pre> tags, newlines embedded in <code> tags are happily translated into <br> and <p> tags just like everywhere else. This messes things up quite a bit if you set white-space: pre in your stylesheet for <code> tags, for example; you'll end up with lots of extraneous HTML tags everywhere. So it turns out I had to write a Wordpress filter as a very last step in this huge mess, to undo the replacement of newlines with HTML that occurs inside <code> tags.

What does all this mean? It means that I hate the internet. Why does this have to be so complicated? Why can't we type some text in a box, and have it appear on a web page more or less as we typed it? It's a rhetorical question, I do know why. It's because the internet is layer upon layer of hack after hack. None of this crap was designed to be compatible with any of the rest of this crap. We have all these systems speaking different languages; it's no wonder things are lost in the translation. And these problems are all just the very beginning. Then comes cross-browser compatibility. And maybe throw a templating engine or two in there, for good measure. And God help you if you mix Javascript into the whole thing. I shudder to imagine being a web designer for a living. I can think of few worse tortures. I think few people realize how much of a miracle it is that any of this works at all.

Re-categorizing Lots of Wordpress Entries

Thu, 14 Sep 2006 11:08:01 -0700

I decided to add a "Gentoo" tag to each post in my WordPress blog that has a "Linux" tag on it now. Reason being that Gentoo is the only Linux flavor I use, and Gentoo deserves some lovin'.

However I didn't want to just change the name of the "Linux" category to "Gentoo". I was afraid it would mess up my permalinks. WordPress doesn't really have "tags", they're really categories. So I figured I'll just keep both; both tags are applicable anyways. But re-tagging 50+ entries manually doesn't seem like a lot of fun, does it?

So I made a new category called Gentoo. Then I ran a query similar to

INSERT INTO wp_post2cat (post_id, category_id) 
SELECT post_id as post_id, 33 as category_id 
FROM wp_post2cat 
WHERE category_id = 5

where 33 is the ID of the Gentoo category, and 5 is the ID of the Linux category. Now, each post tagged "Linux" is also tagged "Gentoo". The other thing to do is edit the category_count field value in wp_categories, for the Gentoo category. It was 0, so I set it to the number of posts it should have.

Et voil?. (And now, I get to add a SQL tag! New tags are always exciting.)

Category cloud; rant time

Tue, 08 Aug 2006 10:09:41 -0700

I added a category cloud so that it's more possible to browse the site and find something interesting. I got the WP plugin for it here.

Let's turn this into a rant. This category cloud is very "Web 2.0"-ish, isn't it? I really don't like the whole "Web 2.0" thing. Specifically AJAX. I think that many programmers agree. By "programmer", I mean someone who has an interest in programming beyond the practical. Programming is a form of science. Given a problem, manipulate a bunch of bits to solve the problem. That's the part of it I love. It's mathematics, it's well-defined and straightforward (mostly).

Web design, on the other hand, is a hack. You have 18 versions of 5 major web browsers, and you try to support them all, or you try to support the ones that are popular enough that you care. You have AJAX, which is just barely powerful enough to do a bunch of really neat things, but in a horribly unclean and hackish way. You're taking a tool which is meant to display sequential, static pages of text and markup, and you're forcing it to pretend to display a dynamic interactive interface. Web design isn't about science; it's about dealing with practical and ugly solutions to problems, given inherently flawed and limited tools. Web design is about dealing with PEOPLE, and working with (or around) their whims and their short-sighted self-interests. It's a psychological game, to a large degree. It's what I hate.

However I don't feel bad borrowing certain things. I like the category cloud. It's simplistic and it works. I am a big fan of simplicity.

Phew.

Tue, 21 Mar 2006 22:36:23 -0800

All done with the layout change. I'm getting good at this, it only took me one evening to do it.

I'm thinking of getting back into FVWM again. Argh. It's such a time-sink, but I feel a need to exercise my geek muscles.

Layout (again)

Tue, 21 Mar 2006 09:27:09 -0800

One of the main purposes of this website is to let me have an outlet for my constant tendency to toy with web design. I've come to accept that fact. Nothing wrong with that right?

On that note I believe I'm going back to an old old layout soon. I've had some good ones in the past. I kind of like this one, but someone told me this green reminds them of a hospital. :(

One of the main reasons it's a good reason to use something like WordPress is that you can screw with the layout and the content isn't affected. Separation of presentation and content makes me happy.