Eternal Backup of the Spotless Drive (Part 1 of 2)

Archival of editing projects and tapeless camera originals is a hot topic these days, with opinions flying everywhere. Today’s post covers our experiences with both hard drives and the new A-Series LTO drive from Quantum.

In early 2008, we decided to switch to LTO tapes for all of our long term archiving.  We previously used external IDE drives (well, internal drives put in cases to make them external) for our backups.  They work decently for at least a while.  The problem was, it got to the point that we had to use Chronosync, a file synchronizing software, to bring back any element from a project because digital hits would appear in the video files due to bad data blocks copying over from the drives back onto our system.

Chronosync Error. The bane of my existence.

There is a “Verify Copied Data” function in Chronosync that detects any of these digital hits, and causes it to restart the copying until it got it right.  Sometimes it would take fifteen tries just to get it so that it wouldn’t have a hit in it.  It is easily one of the most frustrating things I’ve ever seen.  And you know it never happened 10% into the copy, it always cruelly waited 15 minutes for your 30 GB video to get to 92% before it decided it was going to restart.  And while the Chronosync’s verification worked in ensuring a good file, doing the verify literally slowed the copy speed down a drastic amount.

IDE Hard Drive, in a special case to make it external and loadable in a hard drive reader.

Another eventual downfall of  hard drive based archiving is the longevity of hardware itself.  Hard drives tend to only last a couple years or so.   So if you were hoping 20 years from now that the Johnson project is still on that cob web covered hard drive, you probably won’t be happy to see that your hard drive most likely won’t even turn on.   It takes a little TLC to make sure that hard drives last as long as they possibly can last.   Every month or so, you should connect each hard drive you use and “spin them” in order to keep them fresh.   If you let a hard drive sit dormant for too long, it will just give up on itself.   This can be annoying and time consuming when you have 30 hard drives.  Hard drives crave a lot of attention, just like that one hyper kid in my 8th grade math class who would run up and down the aisles waiving his arms.  But tape based archiving is great because you can just put the tapes on a shelf and ignore them for long periods of time, only using them when you want to.  They are said to last approximately 30 years.  And if you are wondering what you are going to do with the files on your tapes 30 years from now, man, you are thinking way too hard about the distant future.   We will probably have thought-controlled nano-archives implanted in our brains by then.

So, the LTO drive was a perfect upgrade.  It had the power of John Rambo, the speed of Barry Sanders, and the elegance of Jackie Onassis.

Before we go any further, perhaps we should take a moment to go over a brief history of the LTO tape, and discover its mysterious origins.  Unlike most of us humans, the LTO tape was born in a laboratory, in the late 1990s.  It was given the name Linear Tape Open, named after its grandfather Linear Tape Ostrofsky, who fought bravely in The War.  It is a magnetic tape, which was first publicly released in 2000, and was only big enough to hold a measly 100 GB, but it was adorable nonetheless.  LTO grew up fast though, and in 2007, it became capable of holding up to 800 GBs!  And the roadmap for he future looks bright as well. On average, the capacity of the LTO tapes has doubled every 2-3 years.

The tape capacity, coupled with its relatively cheap price tag ($30-$40 for the tapes), makes it a valuable asset to backing up large files, for instance, video files.  It is now one of the best selling tapes on the market.  Its most prominent use?  You guessed it, backup.

The Quantum A-Series tape drive.

But the tapes won’t do any good without a drive to put them in.  Believe me, I’ve tried everything; yelling at the tapes, bribing the tapes, waterboarding the tapes, nothing works!  Then I heard a rumor that they will do whatever you say if you put them in a tape drive.  Seems logical to me.  Now, there are several LTO drives on the market, and they do alright for their purpose I suppose.   But Quantum has created a special LTO drive, the A-Series drive.  This drive was created with the post production market in mind, particularly regarding large media backup.  One great selling point of the drive is that it has a built in server. With the build in server software there is no need for any additional software to do backups, and nothing to install.

In addition to that, the A-Series is network based, so it is highly convenient that the drive can be accessed by multiple computers, as opposed to the standard method of having to nomadically hook the drive up to each separate computer that you’d like to use it on, moving it around constantly.   This is one of (if not the only) LTO drive on the market that has this networking ability.

The Series-A server interface.

The actual process of putting files on, and taking files off of the tapes is done through the drive’s web interface.  It is highly convenient that you can archive a project and order new underwear on Amazon with the same program.  There are some problems with the interface, some of which got fixed in recent updates, others that still remain to this day.  For example, one thing that has gotten fixed since was in the right click drop down menu, the “Eject Tape” and “Erase Tape” commands were literally next to each other.  This is almost cartoon-like in its insanity, much like building a robot and putting the “Self Destruct” button half an inch away from its “Off” button.  But things like that got fixed upon request, hey, pobody’s nerfect.

A-Series Drive with tape loaded in. It's good to go!

One of the earliest problems with the drive was a simple confusion with the reality of deleting folders on a tape.  That reality is that you can’t. It’s all or nothing when it comes to tape based archiving.  It is a linear device and can’t be defragmented, much like video tape.  Though, it is possible to delete the contents of an entire tape.  So what that means is if you have a tape with 200 GB on it, and you add a 100GB video to the tape (see: WatchOut), and then you realize, “Oh wait, I already added that 100 GB video to a different tape last week!”  Unless you want to erase the previous 200 GB that were already on the tape, there is no way to delete the 100 GB of previously archived video that you just put on the new tape.  So that is lost space, and granted it is “cheap” tape (costing between $30-$40), but it still feels wasteful to lose ¼ of your tape due to this.  That has nothing to do with the hardware or the software, that simply has to do with tape in general.

Now, when you deal with the Series-A drive, you have to deal with a ton of characters.  A lot of them you will want nothing to do with.  Take Quotation Mark for example, never invite that guy to your house, he won’t shut up the whole time he’s there.  And don’t even get me started on Asterisk, you don’t want to know what he did at my cousin Tony’s wine mixer.  There is actually a blacklist of jerks that weren’t allowed into the LTO drive:

The Usual Suspects: I bet one of them is Keyser Soze.

 

If you attempted to sneak in any of these characters into your archive party, any file associated would get bounced out of the club with them (something I sadly know too well in real life…).  They just wouldn’t be included into the archive. This has to do with the fact that the software that was built into the drive is set to be very strict, and follow Windows format.   So all Mac people must conform to the tyranny of Bill Gates for at least one element of this process.

In previous versions of the software, there was little to no indication that anything was skipped at all, because the transfer summary that shows in the interface used to not exactly work (which was related to a Java problem I believe).  In addition to that, when you would put files on a tape (if I filled a tape with 345 of 400 GB), it was hard to certify that the files actually made it on to the tape, because if you mounted the tape in the Finder and did “Get Info”, the Finder would say that there was actually 100 GB less on the tape than what was in the original archived project!  That of course makes you incredibly nervous that your files aren’t going to be backed up properly.  There is apparently a discrepancy between the Finder and reality itself.   The only real way to see how much space is left on a tape is to put in in the drive, and see for yourself how much is left when it shows it in the interface.  (I keep track of all of these “space remaining” numbers in an Excel document. It’s old school, but it works for an easy reference to how much space is left on each tape)

 

The interface let's you know plain and simply how much space is really on your tape.

Another small element worth noting in the battle between tapes and hard drives is the speed it takes to access your files.  While the actual process of transferring files on and off your archive is faster on tape, the time it takes to locate the files are significantly different.  Think about it this way, if you’ve taken your date from the saturday night sock-hop to the top of Make-Out Canyon in your DeLorean, and you pop in your Run-DMC mix tape cassette, and you want to listen to track 9 (because sometimes words can’t express love the way that the song “My Adidas” can), then you have to sit there and wait for your cassette to fast forward throughout the entire tape until it gets to track 9.   Where in contrast, if you were to pop in your Run-DMC mix CD, you could instantly go to track 9 without having to fast forward over the other tracks.   The wait isn’t too bad (maybe 2 minutes on a bad day) for tape fast forwarding, so it’s only a problem if you’re horribly impatient.  So, while a drive is instant access (like any folder in the Finder), the other benefits of the tapes should outweigh its inability to instantly access files.

Run-DMC helped popularize tape based archiving in the early '90s with their chart topping smash hit, "I'm Gonna Back Up Dat Booty (Right Onto My Tape Drive)".

 

Next week I’ll post Part 2. It’s an in depth look at how the Quantum A-Series drive actually works, and how to make money with it!  

We will also explore how to cook LTO tapes into a delicious winter stew, how to turn your A-Series drive into a muffler for a Dodge Ram, and we will discuss my brief stint as a vocalist for Crosby, Stills, Nash, Young & Roberts.

**DISCLAIMER**  THE STATEMENTS MADE IN THE PREVIOUS SENTENCE MAY OR MAY NOT BE FALSE.

2 thoughts on “Eternal Backup of the Spotless Drive (Part 1 of 2)

Leave a Comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.