263 Posts in Category 'Programming' RSS

Dilemmas continued

I previously prattled on about what the right abstraction should be for an icon theme. I said a hierarchy of subclasses would work.

I was very wrong. It didn't work at all. For example I tried to write a == method for my Icon class. With the class structure I was using before, the only thing I could compare between two Icons is the icon name. In which case, two icons of different sizes in different contexts would have to be considered equal. That's not right. The only other option was to make some kind of is_equal? method all the way at the top of the class hierarchy, and have it construct a complete picture of an icon on the fly, and use that in the comparison. That kind of defeats the whole purpose of setting up the class hierarchy that way to begin with.

A better abstraction is a single Icon class with size and context as properties. An icon needs to know these things about itself. I recently was reading Head First Design Patterns and one of the first "rules" it gives is:

Favor composition over inheritance.

It's right in this case. I was tripped up by the fact that on disk a size does contain a context, and a context does contain an icon. But there's no reason my abstraction needs to match that structure.

On a side note, it's a good book, even though it mostly deals with Java. Many of the "design patterns" it gives are workarounds for the inflexibility of Java itself, but it does give some interesting ways of looking at certain problems.

March 18, 2007 @ 6:13 AM PDT
Cateogory: Programming

New layout done

I was searching for sites on web design, and I came across a novel concept: The primary goal of a site's design should be to be easy to read. Who would've thought? This new layout is the first layout I ever made with that focus in mind. Usually my design goal is "Show off", or "Try some new HTML/CSS tricks", or... well, that's about it.

I despise web design, as a rule. But I always find typography to be somehow fascinating. I remember over a decade ago sitting in Windows 3.1 on my first computer trying to make my own font files. And of course I begged my parents for a calligraphy pen, which I had lots of fun with. Serif vs. non-serif, letter height proportions, width proportions, spacing, those kinds of things, and their relation to easiness to read. I thought this article about whitespace was really interesting, especially the part saying:

Less whitespace = cheap; more whitespace = luxury.

My old layout was very cluttered and light on the whitespace, and it DID look cheap. I wasn't going for "cheap" when I made it, more like "informal" or "fun". But same idea. This time I wanted to go for something less cheap, so I have lots more whitespace.

Regardless, I am not at all skilled at web design, or typography, or anything of the sort. But it is fun to dabble sometimes.

March 17, 2007 @ 5:37 AM PDT
Cateogory: Programming

Programming dilemmas

I scrapped the throw-away version of the KDE <=> Gnome theme converter script which I started many months ago, and started over. I'm trying to do it "right" this time.

It looks like the same index.theme files (if properly written) can be read by Gnome and KDE both, which is nice. I found this handy index.theme specification which should help. Assuming Gnome and KDE implement the spec correctly.

One thing I tend to struggle with is the "big picture". I'm good at getting the details right (or if not right, at least working), but organizing all the bits into a comprehensive, clean whole is something I have trouble getting my head wrapped around at times. Ruby is a nice Object Oriented (tm) language, and that helps a bit with organizing code, assuming a problem maps nicely onto OO-space. But it's oh so very easy to write bad OO code too. Dumping sucky code into a bunch of classes doesn't make the sucky code suck any less.

I think this program is a good exercise in properly abstracting a problem. At the top level is the Theme, which contains Contexts ("applications", "filesystem", etc.), which contain Icons. Probably I need a Size abstraction in there too, each of which contains many Contexts, because each Context can be listed under multiple sizes.

So the end target in all this is really just a bunch of filenames. Now, where should those be put in this abstraction? Do I let the Icons manage and deal with their own absolute pathnames? Or the Contexts? Or the Theme? Or a higher level? Taking a step back, it seems that a relative base directory, "ThemeName", is intrinsic to a Theme, but an absolute pathname, "/home/user/some/dir/ThemeName" isn't; rather the absolute name is part of an operation you do on a theme: "export yourself into this directory". Likewise a Context has (is) a directory name ("applications") and possibly shouldn't care which Theme it's a part of, and an Icon has a filename ("icon.png") and seemingly shouldn't care which Context it lives in.

But the more I think about it, should an Icon know more? Should an Icon know exactly what absolute path on the filesystem it maps to? If not, there needs to be some mechanism to convert Icons into absolute pathnames, somewhere along the line. Where should that go? Should I just crawl down the tree top to bottom, Theme -> Sizes -> Contexts -> Icons, and have it return the pathname that way by building it out of parts? Seems cleaner conceptually, but it's so darned expedient to give the absolute pathname to every object. I think I tend to err on the side of expediency, usually to ill effect, so I'm going to try the other, "proper" way this time and we'll see how it goes.

I also need the ability to sort and search Icons by size; that seems trivial if I have a Size abstraction. And I need the ability to search for Icons by name; seems like a good place to use hash lookups. But I see other possible choices that could work too. Then I just write a tool that takes two Theme abstractions, matches the abstractions in some way, and maps the conversion to real file operations. And some tools to read index.theme files and build Themes from them, and some tools to take a Theme and write out an index.theme file. Seems easily doable. All of these little design questions are the kind of thing that I tend to get hung up (i.e. tripped up) on. It's good exercise at least. I learn a little every time I go through this kind of thing.

March 12, 2007 @ 4:09 PM PDT
Cateogory: Programming
Tags: KDE, Ruby, Gnome

Vim regexes

One thing I very much miss in Gentoo is controlling what is compiled into my Vim. You need to enable perldo and rubydo support at compiletime. Gentoo had USE flags to do it. In Ubuntu I get perldo but no rubydo by default, which is annoying.

The reason I need perldo/rubydo is because Vim's regexes are so inconsistent. * is special when not escaped, but + is special when escaped. You can use \{x,y} (escape only opening bracket), but you have to use \( \) (escape both parens), and with [] it's special when you don't escape either. I simply can't remember these, especially when coding at full speed, and speed is one of the reasons to use Vim in the first place.

Then Vim has magic and nomagic. And you can use \v to set "very magic". Very magic is almost what I want, but you can't set it in your .vimrc and even if you could, the Vim manual tells you to leave the setting of magic alone if you know what's good for you.

PCREs are so much more consistent and easier to remember (excepting Perlisms like "dot matches newline" inconsistencies). The special characters are always special, and you escape them all to make them non-special. But perldo and rubydo in Vim can't do everything Vim regexes can do; they can't properly span lines, is the major thing. They don't highlight text like Vim does with its builtin regexes if you have hls set.

I read somewhere that Vim regexes are set up to let you match C code easily, and that's why for example {} are non-special by default. I don't remember where I read it or if it's true. Doesn't help a lot when writing non-C code though.

March 05, 2007 @ 11:43 AM PST
Cateogory: Programming
Tags: Perl, Ruby, Vim, Gentoo

Taking screenshots of a single window

I am running a game and I need to take many screenshots of the game window. There are lots of Linux tools that take screenshots: scrot, import (part of ImageMagick), gnome-screenshot, ksnapshot, and the GIMP does it too.

It turns out none of them does EXACTLY what I want, on its own. My requirements are:

  1. Take a snapshot via a single configurable keystroke.
  2. Save it in a directory of my choosing.
  3. Take a snapshot of a SINGLE window. And crop off the window borders. I need to take way too many snapshots to have time to go around editing them afterwards.
  4. Look at what files already exist in the directory I pick, and give the new file a filename that is next in ascending order after the files that already exist. It should pad the filename out to 5 digits.
  5. Do all of this non-interactively. It shouldn't ask me to confirm.

The way I finally ended up doing this is using import; I wrote a script to use it, and I assigned that script to a keystroke in my window manager. And once again, it's Ruby to the rescue:


Dir.chdir('/SOME/DIRECTORY') do
        num = sprintf '%05d', Dir.glob('*').select{
            |x| x.match(/^\d+\.png/)
        }.sort[-1].match(/^(\d+)/)[1].to_i + 1
    rescue Exception => e
        puts e
        num = '00001'
    raise "How'd that happen?" if File.exist? "#{num}.png"
    `import -window 'NameOfWindow' #{num}.png`

That this kind of thing is possible is why I love Linux. I can do so much more. I can have it save in multiple file formats. I can have it generate thumbnails as it saves new snapshots. (It's so easy to generate all the thumbnails later using convert that I'm not going to bother.) I could timestamp the filenames rather than using incrementing integers. (There is a race condition in this script that would be fixed if I did this, but I don't care enough to do it that way.)

March 03, 2007 @ 3:23 PM PST
Cateogory: Programming

Mmmm rubygems

I put together my first ruby gem today (for internal use at work). Here is a good tutorial on how to do it. Or else look at chapter 17 in the Pickaxe book. Simply make a gemspec file, define a few attributes listing and describing your files and then compile it. It was astoundingly easy and such a clean way to handle versioning and distributing Ruby code. I always had some weird impression that rubygems was clunkier than Perl's CPAN, but I think I had it wrong. Not at all hard to use.

Setting the RUBYOPT environment variable to 'rubygems' lets you get away with doing a require on gem-provided modules without doing require 'rubygems' in every program you write. I'm sure I must've set RUBYOPT myself a year ago on this machine and forgotten. I was sort of wondering one day, hey, how is Ruby finding those modules? This kind of thing is good to know when you need to set up a Ruby environment on someone else's computer in the near future, I imagine. This is one bad thing about having a computer that doesn't need to be reformatted every six months. Who knows how many other tweaks I've done on this machine and forgot about and now silently depend on.

February 21, 2007 @ 7:51 AM PST
Cateogory: Programming
Tags: Perl, Ruby

Vim brute force

In the end all code ends up as machine language. Generally the way to write effective code is to use a high-level language and let a smart compiler expand a tiny bit of high-level code it into a great deal of low-level machine language.

But another way I seem to end up doing this is using vim. I will generate mounds and mounds of temporary, throw-away, ugly code, but I will generate it extremely fast using vim. It's almost like vim is acting like an extremely high-level compiler; I take a very-high-level idea and using a little bit of vim trickery, I let vim expand it out into a large amount of high-level code.

Say I have a data file with two huge tab-separated columns; I need to insert the value from the second column into a table anywhere I find the value in the first column. There's possibly some way of doing this with one or two SQL queries to slurp the data file directly, but I don't know how to do it in MS-SQL server at work (ugh) and the brute-force vim/Perl way of doing it comes immediately to mind:

:perldo s{^([^\t]+)\t(.*)$}{UPDATE table SET field_x = '$2' WHERE field_y = '$1'}

It could also be done with a native vim regex obviously. Either way I end up with pages upon pages of SQL commands each of which does a single update. This is sloppy and slow, but it scales better than you'd think. I can do many thousands of inserts in this way in a few seconds. If I was doing millions rather than thousands I'd probably take the time to do this the right way, but I rarely if ever find myself needing to do millions of updates.

Is sloppy, ugly high-level code still bad if you didn't actually WRITE it, but had a program quickly write it for you? Machine language is sloppy-looking too, but it's sloppy for a reason (i.e. that's what computers understand). My sloppiness is for another reason: I'm lazy and this saves me time. But I feel like I may be developing bad habits. Then again there are times when I only need a hammer for a few seconds to pound in a nail or two, and I'm not trying to build a whole house. In time like those any hammer will probably do.

February 19, 2007 @ 2:19 PM PST
Cateogory: Programming
Tags: Perl, SQL, Vim

Ow, my brains. (DRY)

At work I was faced with dumping out some reports of some stuff. No big deal, Ruby to the rescue. I have a huge array of lines of data and I need to select out a few bits and output them, doing some simple manipulation to it in the process. So I ended up with something like:

File.open('some_filename', 'w') do |f|
    f.write lines.sort{|a,b| a[0] <=> b[0]}.select{|a| a[0] < 50000}.collect{|a| a[1]}

But it turns out I need to do this multiple times. Each time I do it, it will have a different filename AND a different select criteria. Changing the filename is easy; just make an array of strings and iterate over it.

But how to change the select criteria? If it was something easy, say I always wanted all the data < X for various values of X, then it would be simple to code; just pass in X as a parameter. But say I want the data greater than X in one file and the data less than Y in another file? Or maybe a third report with values == Z. Or who knows what else. Now it's not just data varying; the logic varies also.

A few years ago I'd probably have copy/pasted all the code and just changed the little bit in the select block, like a goon, and rightly suffered for it. But Ruby lets you assign blocks to variables. Then you can send the block to an iterator using &block:

[ ['one_filename', lambda{|a| a[0] < 50000}], ['another_filename', lambda{|a| a[0] > 70000}]].each do |fn, test|
    File.open(fn, 'w') do |f|
        f.write lines.sort{|a,b| a[0] <=> b[0]}.select(&test).collect{|a| a[1]}

It's funny how the code is simultaneously shorter AND better AND more powerful than it would be copy/pasting or using some other method. I remember reading in a Lisp book that this is often a sign that the code is good. It's essentially the DRY principle at work.

This kind of programming is very "emotionally satisfying", but at the same time it's often very hard to wrap my head around. It took me a while to figure out that the above is what I wanted, and to realize that Ruby was capable of it.

I remember very well when I was just learning QBASIC 10+ years ago in high school, this is exactly the kind of thing I always WANTED to do. I have variables; let me use variables to vary EVERYTHING, data and logic and code, everything. I couldn't do it then, obviously. If only I'd have picked Lisp instead of BASIC. Now I feel like I'm kind of re-learning or un-learning all the limitations of the iterative, clunky ways of thinking of programming that were so ingrained by college and years of C/C++. (I remember a professor saying that function pointers in C++ are "scary things, and probably too advanced for this class to cover".)

February 01, 2007 @ 7:11 AM PST
Cateogory: Programming
Tags: Lisp, Ruby

Vim: g + norm

My recent favorite vim trick is g + norm. Look in :h g and :h norm to see what these do. Basically g will search all lines in a file for some pattern, and then do something on those lines. norm will take a string of text as an argument and process that string as though you were pressing the keys in vim yourself, starting in Normal mode.

Combined, you can do some neat things. For example to search for all lines that start with a number, and then add a semi-colon to the end of just those lines:

:g/^\d/norm A;

Note that you need to use i, A, o, or some other way of getting into Insert mode if you want to add text.

One of my favorites (I think I read this on some other site one day) is to find every line containing some text and move those lines to the end of the text file. Sort of an ad-hoc partial way of sorting a file:

:g/some text/norm ddGp

All the vim movement commands work too, so to find every line that has a ", move to the end of the line, move 3 characters back to the left and add a ~:

:g/"/norm $hhhi~

You can move between lines or add lines to the file too, which makes things interesting. What if you start with this text file:


And you do this:

:g/\d/norm o999

Happily this does not blow up into infinite recursion, because vim always makes two passes through the file (one to mark the lines to operate on; a second to actually operate on them); you end up with

January 31, 2007 @ 2:59 AM PST
Cateogory: Programming
Tags: Vim

PHP design patterns?

As one of my professors used to say, "There's no problem that can't be solved by adding another layer of abstraction". He was joking, but there's a little bit of truth to that.

I've been working on my other website for a while. It's all PHP4, not entirely by my choice but I'm limited what can be installed on my server. PHP4 turned out to be a lot more powerful than I'd initially thought. Namely the OOP options available are mostly adequate and in a few places surprisingly OK. PHP has some nice meta-programming sorts of functions like get_func_args() and call_user_func() which let you play around with doing weird things dynamically at runtime, which were almost essential in putting all this crap together. It lacks the completeness or elegance of Ruby, and PHP's syntax is generally horrific and painful, but it'll do.

I recently read Head First Design Patterns which is quite a good book. I was able to use plenty of what I learned there in designing a sort of templating system for my other website. Basically I have tons and tons of tables of different sorts of data. CSS goes a long way to making the presentation of the tables easy to change given that they are properly marked-up, but actually marking them up properly and consistently was the problem.

So I put together a bunch of classes that generate HTML tables. One of the things from Design Patterns that really makes sense when you hear it is to lean toward composition rather than inheritance. In other words, an object that has another object as an instance variable is often easier to deal with than an object that inherits the same functionality from a parent class; the reason being that you can use polymorphism to your advantage, and have your object contain a member of a GENERIC type, and then you can give your object an object of any subtype of that generic type you want. So your class then does not depend on the type of its instance variable at all, so long as all possible objects that end up in that variable have the same interface. Loose coupling and all that.

I have used this to great advantage in adding flexibility in marking up table cells. A cell will be a different class (actually a different composition of lots of classes) depending on the kind of data it stores. A TextCell holds text, so I know to pad it and format it (differently than an ImageCell, or a cell containing an unordered list). I can composite TextCells with other more specific types; a value that represents a percentile is a PercentileCell; the class handles formatting the numeric value and sticking a % on the end. I could composite it with an ImportantCell which will give it a special background color. It's as simple as $cell = new TextCell( new ImportantCell( new PercentileCell( 56 ))); The nifty thing is that I can composite the cells arbitrarily in any order or combination, because each simply holds a generic "cell" member and calling a method on an object only causes it to call the same method on its child object, all the way down the stack. Also, if I later decide to take the % off the end of all my percentile values, I can do it by changing a single class definition. That's the kind of thing you can't handle well with HTML/CSS alone. Before, it'd take me a month and half to go mucking around in static HTML files fixing it all. (Or an hour worth of Perl regexing, but still, no fun.) Or if I want my HTML to validate as XHTML Strict, I can change the HTML for all my tables at once by screwing with the base Cell class. If I want to turn it into XML or some other markup, I could do that too, in theory.

At this point I'm essentially writing most of my pages entirely in PHP, never needing to write any raw HTML by hand at all, which is a very nice change of pace. I wish they'd have taught more of this sort of thing in college. We learned how to write searching and sorting algorithms until blue in the face. But we never really touched on how to design software at a very high level. I still think I'm not too good at it in many ways, but I'm learning, at least.

January 23, 2007 @ 10:44 AM PST
Cateogory: Programming