89 Posts Tagged 'Ruby' RSS

Vim: fun with filters

Vim lets you pipe text through an external filter. There are some obviously nice ways to use this in Linux, like

:!sort | uniq

which will sort all your lines, and then get rid of duplicate lines. But you can do things that are much more sophisticated if you write your own scripts which read from STDIN and output something back to STDOUT.

For example I wrote this Ruby script.


del = %r{#{Regexp.escape ARGV[0]}} if ARGV[0]
del ||= %r{\s+}
STDIN.each do |line|
    puts '(' + line.strip.gsub(/'/,"''").split(del,-1).collect{|x| "'#{x}'"}.join(',') + '),'

This will take a line full of delimited fields, escape all the single-quotes, split into fields on the delimiter, wrap each field in single-quotes, put commas between the fields, wrap each line in (), and put a comma at the end of the line. You can either specify a delimiter, or don't specify one and it'll default to splitting on whitespace. I use this to turn a delimited ASCII file of data into a form suitable for an INSERT command in SQL. So if I start with this:

bob|2|3|Some description
chester|1|4|Another description
sarah|99|0|Let's try an apostrophe

and run this in Vim:

:%!sql_wrap.rb '|'

I get this:

('bob','2','3','Some description'),
('chester','1','4','Another description'),
('sarah','99','0','Let''s try an apostrophe'),

Or consider another simple example. This script will HTML-escape text:


require 'cgi'

STDIN.each do |line|
    puts CGI::escapeHTML(line)

So it'll turn this:

Is 5 > 3?  Yes, & isn't that neat?

into this:

Is 5 &gt; 3?  Yes, &amp; isn't that neat?
August 22, 2006 @ 1:06 PM PDT
Cateogory: Programming
Tags: HTML, Ruby, SQL, Vim


Today I found a really neat site, RubyFacets. Reminds me a bit of Perl's List::Util and List::MoreUtils; it's a bunch of methods to extend core classes in interesting ways.

A while back I posted about a way to prevent Ruby from raising an exception when trying to access an un-initialized subarray of a multidimensional array by extending NilClass. At RubyFacets I found something arguably more interesting: auto-initializing sub-hashes of a multi-dimensional hash.

The code:

def self.auto(*args)
  leet = lambda { |hsh, key| hsh[key] = new( &leet ) }

It took me a couple minutes to figure out what this is doing. The standard new method for class Hash takes a block; if you reference an uninitialized hash element (via the [] method) that block will be called, which presumably assigns the element a default value (thought it doesn't have to).

The above method assigns a default value to any uninitialized Hash elements referenced via []. The default value is a new Hash object. The new Hash object's constructor is also passed a block which assigns new Hash objects to uninitialized Hash elements. You can see above that the "leet" anonymous function contains a reference to itself. I find that mighty clever. This lets you do crazy things like

h = Hash.auto
h['a']['lot']['of']['dimensions'] = true

and you'll get hashes the whole way down.

August 15, 2006 @ 7:45 AM PDT
Cateogory: Programming
Tags: Perl, Ruby, Lambda

Lisp, part 2

Everyone I know who does Lisp says "Learn Lisp and it will change how you look at all other programming languages!" I don't know if that's true or not, but I think I'm starting to learn a bit.

August 13, 2006 @ 5:22 PM PDT
Cateogory: Programming
Tags: PCL, Lisp, PHP, Perl, Ruby, Books

Ruby date enumeration

The Date class in ruby provides an upto method, so you can iterate over a series of dates.

Date.new(2000,1,1).upto( Date.new(2001,1,1) ) do |d|
    puts d

This counts by days, so it will print 365 values or so from 2000-01-01 to 2001-01-01.

What if you want to count by months? Being able to modify classes in Ruby makes this easy enough. Not sure this handles all situations, but it worked for what I needed.

class Date
    def +(n)
        if n == 0 then
            return self
        elsif self.month + n > 12
            return Date.new(
                            self.year + (n.to_f / 12).ceil, 
                            (self.month + n) % 12,
            return Date.new(self.year, self.month + n, self.day)

    def upto(max)
        date = self
        until date > max do
            yield date
            date = date + 1

Date.new(2000,1,1).upto( Date.new(2001,1,1) ) do |d|
    puts d
July 05, 2006 @ 6:07 AM PDT
Cateogory: Programming
Tags: Ruby

Ruby > Perl

A while back I stopped coding in Perl and starting using Ruby for mostly everything. Today I had occassion to use Perl again, because there's no good working equivalent to Perl's Spreadsheet::WriteExcel that works well in Ruby; this Ruby spreadsheet package is a bit too buggy. (It's not my choice to use Excel, but they're paying me to use it. I can't complain. Well, I can complain here I guess. And will.)

One thing I notice about Perl is that Perl sure does give your fingers a workout. I looked at the fairly simple 103-line Perl script I wrote, and it has exactly 118 dollar signs. That's a lot of Shift-$ finger reaching, if you think about it. Ruby doesn't even use curly braces around blocks; it uses do and end, which type quite nicely. Ruby does use a lot of pipes, but I can easily do a one-handed Shift-| maneuver if I lift my right hand off the home row. Think of the potential gains I will have when I'm older from the avoidance of arthritis alone.

Try to come up with a more petty gripe than this. I dare you.

June 27, 2006 @ 6:36 AM PDT
Cateogory: Programming
Tags: Perl, Ruby, Excel, Rant


Here's a reason to love Ruby. In Ruby, nil is the value an unintialized variable gets. Similar to undef in Perl. In Ruby though, everything is an object. nil is an object too, of class NilClass. This lets us do strange and wonderful things, for example when referencing multi-dimensional arrays.

June 05, 2006 @ 4:18 AM PDT
Cateogory: Programming
Tags: Ruby, nil

Theseus and the Minotaur

Here's a little Java game that I found pretty entertaining.

When I got to the sixth puzzle I decided to see if I could write a program to solve these kinds of puzzles. I did; here it is in Ruby, featuring OOP goodness and a bit of recursion, but otherwise just brute force.

It only takes about .04 seconds to solve maze 9. It doesn't find an optimal solution; it tends to have Theseus wander around like a drunkard. Maybe it could be improved with heuristics, but I couldn't think of one. "Move towards the goal" doesn't work in general, because Theseus has to backtrack a lot on purpose to strand the Minotaur behind walls. It'll save one or two moves at most. "Move away from the Minotaur" or "Move toward the Minotaur" don't work because both are necessary many times. So I don't know. I only tried it on puzzle 6-9, but it seems to work.

June 03, 2006 @ 4:03 PM PDT
Cateogory: Programming
Tags: Ruby, Java


I'm likely the last person in the world who heard of FlashGot, but better late than never. FlashGot is a Firefox plugin that lets you integrate with an external download manager program. It also lets you download every link on a page via a single menu command, which is either nice or overkill, depending on what you want to do.

Linux doesn't have many (any?) good download managers. There's D4X, but I never cared much for it. I installed GWGET but FlashGot didn't auto-recognize it, and I'm not going through any trouble to get it working.

However I still find FlashGot incredibly useful, for one reason: You can use a custom downloader executable. FlashGot will then call the executable and pass it the download URL as a command line argument. You can also pass other arguments (read about them all here) but the URL is all I really need.

The downloader I use is a simple Ruby script I wrote myself which calls wget. What's the point of this, you ask? Well, you can do some neat things like:

  • Filter your downloads into directories by filetype, filename, source website, or any criteria at all.
  • Spawn massive numbers of parallel downloads with a single click. (Probably not a good idea to hammer servers too much with this though, it's not nice.)
  • Use all the power of wget, which includes:

  • custom timeout duration* download retrying* download resuming* filename timestamping* download speed throttling* FTP suport* (perhaps my favorite) GOOD filename collision resolution, so if you download a file called 1.png and then download a file called 1.png from a different site, wget will save the second one as 1.png.1. This something I miss from Safari. Firefox by default tends to ask you if you want to overwrite the old file, which gets very annoying very quickly.

  • You could even conceivably crawl a web page or do recursive downloads.

Let's say you want every MP3 you download to go into a "music" folder, every PNG you download to go into a "Pictures" folder, and ignore all other files. You could do something extremely simple like this (which I just wrote in 5 minutes and haven't tested):


require 'fileutils'

    ARGV.each do |arg|
        dir = ''
        if arg =~ /mp3/i then
            dir = '/home/chester/music'
        elsif arg =~ /png/i then
            dir = '/home/chester/pictures'
            dir = nil
        if dir then
            FileUtils.mkdir(dir) unless File.directory?(dir)
            Dir.chdir(dir) do
                `wget #{arg}`
rescue Exception => e
    # If you want to see the output
    # when the script crashes, you 
    # could log it here.
    raise e

Point FlashGot to this script and when you "FlashGot All", all linked PNGs and MP3s on a site will be downloaded and sorted, and all other links will be ignored. This would be very useful if you want to grab a whole page of wallpapers for example.

April 17, 2006 @ 3:29 AM PDT
Cateogory: Linux

Shoot me please

I have what would have been considered a supercomputer 20 years ago sitting under my desk. 3GHz. And I can't even use the thing as a simple word processor half the time. I just watched my computer become bogged down to the point where the mouse could hardly move. I CTRL+ALT+DELed and spent 5 minutes (5 MINUTES) waiting for the task manager to appear. For some reason, while this was happening things started randomly disappearing from the taskbar; however they were still open, I could ALT+TAB to them. At the same time, some things started appearing twice in the ALT+TAB screen even though there was only one of them open. I'm still trying to explain that one.

What did I do to cause this? I had a Ruby script running that was doing some text-replacement on a plaintext file, and I made the mistake of trying to open a Firefox window while it was running. I believe they invented this thing called "multitasking" sometime a few decades ago. Maybe it'll make it into Vista?

My OS? Windows XP Pro. "Pro", I believe, stands for "Propensity to Suck". Brand new computer, barely 2 months old. I often hear how "stable" XP is compared to other versions of Windows, which is true I guess; I didn't get a blue screen, I just got a computer trying to eat its own face off for 5 minutes. Already, if I leave this computer on for, say, 3 or 4 days without rebooting, I witness massive slowdown.

When I'm running Linux at home, I have Apache/mysql/sshd going, 300kbps worth of torrent-downloading, GIMP and gvim and 8 terminal windows running, and there's no slowdown whatsoever. Sometimes I have so much crap open at once I forget about it until the taskbar on all five virtual desktops are cluttered beyond being able to read the program titles.

March 17, 2006 @ 2:30 AM PST
Cateogory: Rants