My recipe for not losing important stuff on my home desktop system:
1. Redundant Disks
I set up a pair of disks in a RAID 1 configuration to be the “home” folder on my system. RAID arrays can be accomplished numerous ways. I do this in software under Linux; under Windows you are more likely to end up using a semi-hardware solution. However, disk redundancy only protects you from a single disk failure. If you nuke a file on your RAID pair, it’s still gone in both places. That brings me to:
2. External drive with incremental backups
rsnapshot is “Time Machine for the rest of us”. I’ve got it set up to push hourly incremental backups out to my external USB drive. On Windows, I had great success with Acronis True Image Home in differential mode, but Norton Ghost is an old standby.
3. Next step: Off site backups
Doing what I do protects me against most data-loss woes, but it won’t protect me against the meteorite that slams into my home office. For that, there’s the concrete bunker I’m working on in my spare time and off-site backups. While there are solutions that use cross-network syncing for the ultimate in worry-free backups (unless you worry about data security), I’d rather put my faith in “sneaker net” and a second external hard drive. Swapping external hard drives on a weekly basis and carrying the “off drive” to the office would mean never losing more than a week’s worth of important stuff…
What do you do to protect your important data?
Fully redundant servers — two at home [each mirroring the other, for example using rsync over ssh with compression], two at work [each mirroring each other, and providing 'warm' sync for the home machines], and most recently two at my alternate work-location [which is a bit overkill, but geographically convenient]. A cheap Dell Inspiron refurb costs $190. A cheaper off-lease HP D530 runs $89 or $99.
I used to play the software RAID-1 game (on Linux and Windows, see my May 2005 LiveJournal post for the Windows commands), but my success rate barely exceeded 60/40 — several heat-failure and/or corruption conditions will (and did) cause the mirror to break, as you’ve noted, necessitating re-image from scratch, or, worse still, rendering *both* drives unusable.
After pricing the situation out any number of ways (purchase dedicated SDLT drive, purchase small standalone NAS, etc.), I was fairly satisfied with the $99 HP D530. I have since added a cheap 160-gig USB external drive for ease of portability and rebuild; this external drive might itself be sufficient for most scenarios, with the exception of flooding or temperature extremes or what not.
– Sven
Oh, I also do a nightly tar-zip export to sidehack, ayup and aculei for the *really* important stuff (encrypted tax returns, select family pictures, monthly pay-the-bills info for my spouse if/as needed, etc.), again observing the triply-redundant design principle.
This data subset is no greater than 100 megabytes, however; I try to be considerate of others’ bandwidth and disk-space.
– Sven
Mozy for off-site backups. It’s free for 2GB, and $35 a year for an entire computer. Worth the cost, and easy restores
Wow. I knew I wasn’t being super-paranoid, but now I see how much higher the bar really is
Although I have a lot of writing and other things I probably should have considered backing up at some point, the only time that really made me nervous was when the drive with all my digital photos started acting weird.
I realized all the photos over the past few years (mostly since Sawyer was born) would have been lost forever.
I’m nowhere in the realm of sven concerning backups, but I do backups via an external drive regularly of a certain set of data. In case of a disaster in the home, I signed up for a Mozy account. It backs up my files off site, and gives me a bit more piece of mind. It’s by no means quick (if you have a lot of data, say like picture files), but at least they are backed up (and encrypted) for me regularly.
I have toyed with mirroring servers, configuring a raid array, setting up off site backups to my parents place, etc. Though I believe, and hope, that the regular backups made at home, and the one off site will be enough. That, and until I can get FiOS out here, anything more off site then Mozy will just cause my bandwidth to go poof, or take forever to backup.
Having entered the realm of IT recently, I really am amazed at the number of horror stories I’ve been hearing recently about complete loss of data, and the amount of virus/malware infections that are out there. It’s enough to make me super paranoid about my data and machines at home.
Does prayer count? =)
Any opinions on ZFS? I’m using TimeMachine and SuperDuper on top of RAID 0+1. I’d love to pick up some space by moving to ZFS, but it’s not stable on Mac OS X Leopard yet.
@Sven
As a note… ayup and sidey are both backed up by their admins… making Sven’s triply-redundant design principle “quintuply redundant”. Plus ayup is RAID-1. Sextuple! Sex is always better.
Anyway, what do I do?
1) ANYTHING I care about is on RAID-1.
2) Almost everything is boxbackup’ed to a local disk with the intent that it get moved to a friend’s house… if I ever get my act together.
3) Important stuff (pics mostly) gets backed up to another server in the house AND one outside the house.
@dibble
“That, and until I can get FiOS out here, anything more off site then Mozy will just cause my bandwidth to go poof, or take forever to backup.”
I bet you would be pleasantly surprised at the speed and bandwidth efficiency of an incremental rsync… Damn that is some amazing software.
@dibble
The point about baby pictures is very poignant. As is the paranoia. =)
This subject hit critical mass for me a year or two back, when I was reading Audrey Niffenegger’s /Time Traveler’s Wife/, and Henry (the protagonist) had a letter in his desk entitled “A Letter to be Opened in the Event of My Death.” I was flying for business nearly all the time back then, and that kept nagging at me.
I didn’t get too morbid in my contemplation, but I started asking myself what would Ginger need to know, what bills would she need to pay, what financial information could be summed up and presented on a single page for future reference, which of my (digital/physical) possessions would she *really* want to keep, etc.
I figured that the family pictures were of highest value, that the monthly bill structure and savings accounts and investment vehicles were a very close second priority, that certain (unclassified) work documents might be requested from my employer if they lacked redundant copies, and that various of my scribblings (e.g., LiveJournal), while not “critical” in any sense, might convey a residual sense to my child(ren) of what their father had been like. At least the top two or three of those items *had* to be accessible if our house burned down, or, say, if an AWACS jet crashed into Hanscom Air Force Base.
That started the first phase of my backup infrastructure. (Since then, the aforementioned LiveJournal has doubled as a sort of “baby book” noting accomplishments and personality and milestones, which ratchets it up slightly in value.) Along the way, I fried a couple of disks on my media servers (Babylon 5, Deep Space Nine, Lost, etc.), which though non-critical proved to be an awful pain in the @$$ to rebuild. I took my scheme one step further and supplanted “portable data” with “on-site redundancy”. My rebuild process now looks like let-a-command-run-overnight as opposed to go-home-and-get-a-disk or start-a-96-hr-remote-copy.
– Sven
@pbp
I don’t have direct experience with ZFS. Honestly, though, gambling with filesystems seems a little risky. If nothing else, a venerable system like FAT32,, NTFS, or EXT2/3 has got a decent amount of community-based recovery knowledge. FWIW, though, ZFS is apparently on the roadmap for OS X Server 10.6, so if you can wait, you’ll be able to get it with Apple backing you up.