Redundant Backup and Offsite Replication with FreeNAS and Crashplan
Admittedly this article is a bit of a divergence from the usual round of website related content that we post here; however, we recently completed an overhaul of our backup solution and felt that it would be a worthwhile discussion topic. These days, everyone has digital data that they value and need to preserve, but given the plethora of backup options out there the choices can get a bit overwhelming, particularly to those who are not information technology professionals. A big part of the confusion stems from terminology: backup, replication, RAID, uptime, redundancy, offsite storage, etc. With all of those buzzwords it's easy to lose sight of what's important and what will provide you with the best level of security for your needs.
Tinfoil Hat, Seat Belt, or Underground Bunker?
Before dumping money into one solution or another, you need to decide what level of protection is the best fit for you and your data. A good question to start with is "what degree of catastrophe can I withstand before I no longer care about the state of my data?" As an example, if your house burns down, taking with it your computer which houses all of your family photos, will you be concerned about that data or will you be too distraught by the burned-down house to concern yourself with it? If the loss of data in that situation would be a problem for you, you need offsite replication. If not, you can get by with a different solution. What about a simple hard drive failure? Hard drives (excluding solid-state drives) have several moving parts and as such, can fail without much notice. If the data on that drive is important to you, you need redundancy or, more practically, RAID, which stands for "Redundant Array of Inexpensive Disks."
Once you've figured out what level of security you're willing to live with it's time to plan out how to handle those problems. In the case of our main data server, we wanted to make sure that work could continue if a hard drive failed and we could be protected in the case of total system failure from a fire or other major disaster. To that end, we combined RAID in the form of a FreeNAS server which in turn is backed up over the Internet using Crashplan software. Here's how we did it.
Building a FreeNAS System
FreeNAS is a NAS-focused variant of the FreeBSD operating system, so if you're familiar with Windows, Mac, or Linux, this will be a bit of a change. It allows you to set up a robust, reliable, and redundant network attached storage (NAS) system using a wide variety of hardware. People have built FreeNAS using nothing more than leftover PC parts and hard drives and have also integrated it into large scale multi-drive rack mounted systems, so you have a good deal of choice when it comes to your hardware. The real power of FreeNAS is in the ZFS file system, which is beyond the scope of this article -- suffice it to say, it works well in a NAS environment.
We built our server using a Supermicro 2U rack mount chassis which had six hard drive bays and it so happened that we had three 2TB and three 1TB drives available that we had salvaged from other storage systems over the years. With this configuration we opted for a RAIDZ2 setup which uses two of the six drives as parity -- meaning two of the drives could crash and we would not lose data or availability. The tradeoff is that the total storage pool would be reduced by the size of those two drives. Plus, in any kind of RAID setup the storage pool is limited to the smallest drive in the array -- 1TB in our case. Which meant that we had 4TB available to us (6 x 1TB - 2 drives = 4TB.) The good news is that we could replace the three 1TB drives for 2TB ones and we would immediately double our available space to 8TB.
The ins and outs of setting up FreeNAS is outside the scope of this article but if you're interested, a good place to start would be this great article on Engadget. It's a bit outdated but the general concepts are there and should get you up and running.
Offsite Replication: Crashplan
Once we built a reliable network attached storage system that could withstand two simultaneous drive failures, we tacked the problem of an office-scale disaster; i.e., a fire, flood, or other catastrophe affecting the entire building, particularly the server room. There are several cloud-based replication systems such as Mozy, Carbonite, and LogMeIn Backup, and all have their pros and cons. For home users we would generally recommend going with one of these services as they're simple to setup and reliable -- but they don't allow you to backup any network drives, which meant they wouldn't work in our case. Plus, backing up 4TB (or more) of data to an online data center would eat up our entire internet bandwidth for weeks.
We decided on Crashplan for two reasons: there is a FreeNAS plugin available (more on that later) and Crashplan allows you to back up to additional computers, friend's computers, or the cloud, depending on your preferences. With that in mind we first installed the Crashplan plugin on our FreeNAS machine. This is a bit of an involved process, the end result of which is that Crashplan runs in a protected sphere of functionality (a jail, in FreeNAS parlance) which gives it restricted access to the files on the NAS and separates it from the core operating system should anything go wrong. Here's a great article on how to set up Crashplan. The reason it's so tricky is that you'll need to administer Crashplan from another computer since the plugin jail doesn't have a graphical interface -- but the aforementioned article walks you through the process.
Now we needed a destination. We found an old small form factor PC, installed the latest version of Ubuntu Linux on it, plugged in a 4TB USB hard drive, installed the Crashplan application, and pointed incoming backups to the external hard drive. Once connected to our free Crashplan account, the new destination PC became available to our FreeNAS Crashplan jail as a target for backups. At this point we could have brought the small PC and external hard drive to our remote location, plugged it in, and started backing up; however, we would then run into the same bandwidth problem that we saw with the other backup solutions. Fortunately, Crashplan allows you to "seed" your destination prior to starting the backup schedule. We connected the PC and USB drive to our local network and backed our data up that way first, then moved the units offsite. This saved a great deal of time and kept us from overwhelming our internet bandwidth.
Is it Bulletproof?
Technically, no. In order for us to lose data we would have to experience either a) three simultaneous hard drive failures in our FreeNAS server and a failure of the external 4TB hard drive stored offsite or; b) a fire, flood, or other disaster at our office along with the aforementioned failure of the offsite drive. While not 100% foolproof, only extreme coincidental bad luck could affect our data. If we wanted to really strap on the tinfoil hat, Crashplan allows for multiple offsite backups so that we could replicate the data as many times as we desired. However, at this point we feel that the setup that we've built will be extremely reliable.