The Dark Side of DROBO


IMPORTANT NOTE: This post was updated on Wednesday; November 4, 2009 with new information regarding resizing partitions on the Drobo using iPartition.

As brought to my attention by reader Bradley Davidson (thanks Bradley), iPartition does not actually support the method that I mentioned, and neither does drobo directly.

In my testing, I performed the resizing operation on a newly formatted drive that didn’t have any data (since I had just lost all of my data that was on the drobo). If you try to resize a partition as outlined in this post, you WILL LOSE YOUR DATA. So don’t try it. In theory it was a great idea, but apparently this too will cause problems.

You can find more information on the iPartition website, as well as from Drobo. Like we’ve pointed out many times, we’re also learning here at SuiteTake so thanks for the feedback.

Before I start, let me just say that I am a Drobo fan. I have 2 of them (an original USB and a newer FW version) and plan to purchase more Drobo’s in the not too distant future. Overall I’ve had a great experience with the units and when I needed assistance their tech support was very helpful.

All of that being said, there is a dirty little secret that they don’t warn you about and if you’re not careful you can have your Drobo crash beyond recovery, which is what happened to me this past week. I lost nearly 4 TB of files and there was nothing I could do to get them back. If you own a Drobo, this is a must read.

Here’s what the main issue boils down to. One of the ways that the Drobo is able to expand over time is by presenting the OS with a virtualized volume of its total achievable space. So if you format your drobo drive as a 16 TB drive, (the current maximum except for on the Drobo Pro which is 32 TB) the OS thinks that it has 16TB of storage available. Thus, every program you use also sees the drive as a 16TB volume. This is true even if you have a single 250 gig drive in the drobo.

When you’re manually adding files to the drobo, this is not too big of a deal. You’re interacting with the drive and if you start to get close to filling the Drobo it will start to warn you both through the lights on the drive as well as the Drobo Dashboard software. It’s kind of hard to miss, and can actually get kind of annoying if you just try to ignore the warnings.

However, in our case we don’t often interact with the Drobo directly. It’s used as a backup drive for one of our edit suites and sits in a back room attached to a server. Every 3 hours our Retrospect server runs a script to clone all active projects to the drobo. In doing this we’re protected against any failures and at most would only lose 3 hours of work (which could still be a lot, but better than 3 months of work).

We just finished a good long stretch of having projects back to back without time to offload the complete ones, so the “active projects” folder started to grow beyond the size of the Drobo’s capacity. Because Retrospect thought there was 16TB of space, it just kept trying to pile the files onto the drive until the drobo finally crashed. It unmounted itself from the computer and no amount of restarting of the server and/or Drobo would bring it back.

In this case the Drobo volume gets so overloaded with files that it has no “work space” left to do it’s own housekeeping to keep track of used and unused space, as well as file redundancy. It needs a small amount of it’s own space to do this..

You might think that since it’s a space issue you can just replace one of the drives with a larger one and let it rebuild. Nope, that doesn’t work either. I was on the phone with Tech Support for about 20 minutes as we walked through the situation and finally he informed me that if I was sure that the drive had been filled beyond its capacity than there was nothing to do other than to start over and reformat the drives.

Wow! The drive system that’s touted as the safe way to store all of your most important files has one major flaw, and most people are not even aware of it!

While on my tech support call I asked the engineer how frequently he received calls about this particular problem. After a big sigh he admitted that it was nearly every day.

For us, losing the drive was not a huge issue. It was only a clone of one of the edit rooms and could be rebuilt overnight by just re-running the backup (with fewer files selected this time). But what if this was your primary repository for something important? What if it’s the one place you keep all of your work files, photos, music or something else that’s of great importance to you? I have a Drobo at home that I use as my primary holding tank for all digital video that I shoot of my twin boys, which is all tapeless media. If it had been that drive that went down in this same way I’m pretty sure I would not have been so easy going on the phone.

When to be Scared
How do you know if this has happened to you? To be totally sure you should go through Drobo tech support, but I’ll explain what usually happens.

You’ll find that the drive is not mounted on the desktop and doesn’t show up in the drobo dashboard. Even after a restart of the computer and the drobo it will not show up.

To verify it’s not the Drobo hardware, power down the unit, take out all of the hard drives and then plug it back in (with it connected tot he computer). If with all of the drives out it is able to be seen by the Drobo Dashboard software, then you’ve verified that the Drobo itself is working fine.

Next, with the cover off the front of the drive, unplug the drive (reinstall the drives if you have pulled them out) and plug it back in while watching the lights on the front (the lights along the bottom of the Drobo). There are two sequences of lights as the Drobo first powers up. It starts by building lights from left to right, until the row of blue dots are all lit. This is the boot up sequence of the drobo (which is actually running a version of linix, so yes it really is a mini-computer).

Next, all of the blue lights along the bottom will go out and then start to build from right to left. During this process the Drobo is evaluating the drives and determining where the extra space is on each drive and reading other information that it needs. This process normally completes and the Drobo mounts. In my case, the lights never finished building and it would hang every time, 3 lights before completing. Even after leaving it on for a few days it didn’t make any more progress. This is almost surly a bad sign.

So, what to do?

If this has already happened to you, there’s not much you can do (again, make sure you go through Drobo tech support to be 100% sure). The usual procedure is to power up while holding in the reset button in the back with a paperclip. This will reset the Drobo back to it’s “out of the box” state and allow you to format the drives again.

The good thing is that since you’re now aware of this problem there are a few ways to avoid the problem in the first place.

The Easy Answer

When the drobo dashboard tells you to replace a drive, or you see a yellow or red light flashing next to one of the drives, REPLACE THE DRIVE! Don’t mess around! I know I’ll take this warning a lot more serious in the future.

The Easy Answer, But With A Trade Off
When you first start to build your drobo, format it through the drobo dashboard but instead of just going to 16TB, set the size to the size your drive will be after formatted with the you plan to use. To figure out the total formatted size, you can use the online Drobolator.

The down side? Remember how you loved just adding drives without formatting and just expanding your storage space? You can’t do that anymore. Every time that you install a new drive you will need to reformat the drive to realize the additional space.

The Pain In The Ass Answer.

Format the drive as you would normally do using the Drobo Dashboard software, and go to the maximum of 16TB.

Once this is done, you need to open the drive in Apple’s Disk Utility program and repartition the drive to create a volume that is just slightly LESS than the available space that you see in the drobo dashboard.

For example, in my case I have (2) 2 TB drives, and (2) 1 TB drives installed. Once formatted it gives me a total available space of 3.6 TB

But in the finder this drive is showing up as having almost 16TB of space. Remember, this is what causes the problem.

Inside of Disk Utility you need to change this drive from one partition to 2 as shown here.

Set the first partition to just smaller than the maximum amount of free space that you have. In my case it said I had 3.6 TB, so I have made this volume 3.5 TB. In doing so I’ve just prevented the drive from being able to be overloaded. The Finder/OS will now know what the proper amount of space available is.

On the extra partition, just set it as “Free Space”. This will allow you to still expand your drive later as you upgrade the drives on your Drobo.

IMPORTANT NOTE: This post was updated on Wednesday; November 4, 2009 with new information regarding resizing partitions on the Drobo using iPartition.

As brought to my attention by reader Bradley Davidson (thanks Bradley), iPartition does not actually support the method that I mentioned, and neither does drobo directly.

In my testing, I performed the resizing operation on a newly formatted drive that didn’t have any data (since I had just lost all of my data that was on the drobo). If you try to resize a partition as outlined in this post, you WILL LOSE YOUR DATA. So don’t try it. In theory it was a great idea, but apparently this too will cause problems.

You can find more information on the iPartition website, as well as from Drobo. Like we’ve pointed out many times, we’re also learning here at SuiteTake so thanks for the feedback.

Later on when you’ve swapped out a smaller drive for a larger one and have more space, you can use a program called iPartitian to expand the drive to accommodate for the added space. iPartitian is a much more robust partition program than Apple’s Disk Utility and works great resizing all types of volumes without losing any data. I mentioned this software in a previous post as a way to create a multiple partition emergency boot drive.

Just for the record I have NOT used disk utility to do this step (I use iPartition) but it should work in theory. Test at your own risk!

SIDE NOTE
While the official word from Drobo is that pretty much any SATA drive will work, in my experience using Hitachi drives can cause heat issues once you get 4 drives in the case. My Drobo at home is a original USB version and I’m constantly having issues with it shutting down due to overheating, even though the ambient temperature is only upper 60’s to mid 70’s. I’ve been told by tech support that Hitachi drives are known to run a bit hotter. I guess I wouldn’t disagree with that.

The better choice would be the Western Digital Green drives. They’re a good balance of performance, energy usage and heat, and they’ve never given me a problem. The latest 2TB versions of these drives have really come down to a nice price and are well worth it.

The SuiteTake?
Even with the Drobo, it’s still a good idea to have more than one copy of your data. As a rule of thumb I try to always have 2 on-site copies and one off site “cloud” copy of anything that I can’t bear to lose. In this digital age it seems we have fewer and fewer tangible objects, and it doesn’t take much to lose a great deal.

The Drobo is a great technology and offers expandability and protection that other consumer drives just can’t match. But make sure you’re aware of the downside of not tending to the drives needs in a timely manner, or next thing you know you’ll be looking for a paperclip to push in that little reset button while holding back the tears.


About the Author

author photo

Thomas Tomchak

Thomas Tomchak
President and Chief Technology Officer of Edit Creations, Inc.

Tom has been working in the broadcast industry since 1987 and has extensive experience editing corporate, broadcast and documentary programs. Tom has worked with a long list of nationally recognized companies and agencies including, Young and Rubicam, J.Walter Thompson, United Airlines, Better Homes & Gardens, Sears, Lions Clubs International, and Warner Brothers. You've seen his work on CNBC, United Airlines in-flight programming and major market television stations across the country.

Today he continues to seek new business opportunities while working with Edit-Creations' ever-expanding client base.

See All Posts by This Author

There Are 12 Responses So Far. »

  1. Great post – we are thinking of getting a couple of the Driobo systems. It's good to know of tese kind of foibles.

  2. Very informative blog piece that will help out many Drobo owners.

    Like Robin says (http://blogs.zdnet.com/storage/?p=454).. so nice/cheap you should buy two and as the old saying goes (especially in tapeless workflows) If you don't have two copies. you don't have it..

  3. I can't think of any technical reason DROBO couldn't prevent this from happening. Since they are simulating the drive area that isn't installed, why could they just simulate that the uninstalled part is already full of simulated read-only data. Then the OS would report the actual amount of free space. When an additional drive is installed then some of the simulated data would disappear.

  4. Tom, any comment on the one know Mac issue listed in the Drobo firmware release notes?

    Macintosh Specific
    Symptom: Non-destructive repartitioning is not supported.
    Condition: In Mac OS 10.5 and later, Disk Utility allows you to repartition a volume without losing the data on it. Drobo volumes do not support this functionality and should not be repartitioned once data has been placed on them.
    Workaround: Copy your data onto another drive and use Drobo Dashboard to reformat your Drobo’s volume to a smaller size.

  5. Brad, I’ll have to look into that. You raise a very good point and no I was not aware of that tech note from Drobo. I’m going to contact them directly and have them comment on my suggested resizing and see if I can find out more information.

    Thank you for bringing this to my attention. I’ll be sure to post an update to the post as soon as I have the information.

  6. […] another really big problem in my eyes can be read at http://www.suitetake.com […]

  7. […] manages them to ensure the reliability of your data and easy expandability of the storage. However, Thomas Tomchak just pointed out one major flaw: if you overflow your Drobo with data, the entire device may give up and you’ll lose […]

  8. […] first person to have my #drobofail and they know about the problem. luckily it is only my backup. http://is.gd/516uP@dtseiler would be very grateful if you could share the files and & db backup script, many […]

  9. Thanks Tom for your great explanation. We too have a Drobo and used it 'out of the box'. Then, when we wanted to enlarge the drive, it was impossible because we didn't create a larger virtual drive first. This is not very well explained in the setup procedure, IMHO. So then you need to backup and restore 3 TB of data, which is, well, quite annoying.

  10. Excellent article – as this just happened to my Drobo. I wonder if Drobo will update the drivers/software somewhere to avoid this.

  11. This is a great read. I just plugged in my first drobo ever and before moving anything onto it I took one look and said “WTF, why does it show more capacity than it has?” I wanted to know what would happen if I went over, and this article answered that in complete detail. Drobo should probably just say on the cover of the manual in big red letters “DON'T EXCEED YOUR CAPACITY CALCULATOR CAPACITY.” Everything else is self explanatory.

    Thanks!

  12. Possibly, one could create a full 16TB Drobo volume, and implement user disk quotas so that the physical size of the Drobo drive could not be exceeded. Then when you added more drives to the Drobo, you could just increase the disk quota. Disclaimer: I am not a disk quota expert.

Post a Response

You must be logged in to post a comment.