FlashGot

Mon, 17 Apr 2006 10:29:35 -0700

I'm likely the last person in the world who heard of FlashGot, but better late than never. FlashGot is a Firefox plugin that lets you integrate with an external download manager program. It also lets you download every link on a page via a single menu command, which is either nice or overkill, depending on what you want to do.

Linux doesn't have many (any?) good download managers. There's D4X, but I never cared much for it. I installed GWGET but FlashGot didn't auto-recognize it, and I'm not going through any trouble to get it working.

However I still find FlashGot incredibly useful, for one reason: You can use a custom downloader executable. FlashGot will then call the executable and pass it the download URL as a command line argument. You can also pass other arguments (read about them all here) but the URL is all I really need.

The downloader I use is a simple Ruby script I wrote myself which calls wget. What's the point of this, you ask? Well, you can do some neat things like:

Filter your downloads into directories by filetype, filename, source website, or any criteria at all.
Spawn massive numbers of parallel downloads with a single click. (Probably not a good idea to hammer servers too much with this though, it's not nice.)
Use all the power of wget, which includes:
custom timeout duration* download retrying* download resuming* filename timestamping* download speed throttling* FTP suport* (perhaps my favorite) GOOD filename collision resolution, so if you download a file called 1.png and then download a file called 1.png from a different site, wget will save the second one as 1.png.1. This something I miss from Safari. Firefox by default tends to ask you if you want to overwrite the old file, which gets very annoying very quickly.
You could even conceivably crawl a web page or do recursive downloads.

Let's say you want every MP3 you download to go into a "music" folder, every PNG you download to go into a "Pictures" folder, and ignore all other files. You could do something extremely simple like this (which I just wrote in 5 minutes and haven't tested):

#!/usr/bin/ruby

require 'fileutils'

begin
    ARGV.each do |arg|
        dir = ''
        if arg =~ /mp3/i then
            dir = '/home/chester/music'
        elsif arg =~ /png/i then
            dir = '/home/chester/pictures'
        else
            dir = nil
        end
        if dir then
            FileUtils.mkdir(dir) unless File.directory?(dir)
            Dir.chdir(dir) do
                `wget #{arg}`
            end
        end            
    end
rescue Exception => e
    # If you want to see the output
    # when the script crashes, you 
    # could log it here.
    raise e
end

Point FlashGot to this script and when you "FlashGot All", all linked PNGs and MP3s on a site will be downloaded and sorted, and all other links will be ignored. This would be very useful if you want to grab a whole page of wallpapers for example.

briancarper.net (λ) (Tag: Flashgot)

FlashGot