Tuesday, March 04, 2008

Ruby to the FTP Rescue

If there is any protocol I loath more, it is the horrible FTP. I've *never* found a good client that can deal with large and/or many file uploads. I've tried GUI clients, command-line clients, clients on Linux, and clients on Windows and every single one of them was just about enough to make me scream trying to transfer a large number of pictures to my web server.

Between cutoff transmissions, light-weight GUIs, flat-out-lieing interfaces, and other random errors, most of the clients I tried could not even resume an upload to save their life. As FTP was designed back at the dawn of computing, it just was not designed to do as large transfers as I need, and most of the clients available today are just not capable of handling a few thousand megabyte-sized images. Pretty sad.

Then along came a lightbulb in Glenn's head: use Ruby's builtin FTP functionality. After about an half hour of messing with it, I wrote a program that could upload a file to my server. Then, Ruby's welcoming API made it almost trivial to check to see if a file the program was trying to upload was already on the server. Piece of cake. Then I realized that there was a problem if a file was partially-transferred as is the case when a connection drops. If I retried using my program, it would continue to the next image, leaving a half-uploaded image in its wake. Well, with a little more reading, I found that you could trivially turn on file-resume support. Could it get better? Nope.

Check out my code and feel free to steal it for your own devious purposes:


#!/usr/bin/env ruby

require 'net/ftp'

hostname = 'ftp.hostname.com'
username = 'glenn'
password = 'topsecret'

# your filesystem source directory, whichmust end with slash
sourcedir = '/home/glenn/camera/canon/'
# destination directory on ftp server
destdir = '/canon'

Net::FTP.open(hostname) do |ftp|
    ftp.login(user=username,password=password)
    ftp.passive = true
    ftp.resume = true
    ftp.chdir(destdir)
    filelist = ftp.nlst
    Dir::open(sourcedir) do |dir|
        dir.each do |file|
            fullpath = sourcedir + file
            if FileTest::file?(fullpath) #and !filelist.member?(file) then
                print "Transfering " + file + "\n"
                ftp.putbinaryfile(fullpath)
            end
        end
    end
end

A syntax-highlighted version (Blogspot is giving me all kinds of trouble trying to paste this in.
The actual Ruby file

Saturday, March 01, 2008

Good Ol' Nietzsche

The man of knowledge must be able not only to love his enemies but also to hate his friends.
- Friedrich Nietzsche