rsync Time Machine

I’ve long had the idle idea that I wanted to write my own backup program. Time Machine is pretty great, but also pretty limited. Most other Mac “backup” programs, these days, are really just disk cloners. The old king of backup, Retrospect, has been fairly thoroughly disgraced during the OS X years. Really, there aren’t that many options for Mac users wanting incremental backups–it’s pretty mind-boggling to me, as someone who has lost a bunch of hard drives.

I last tried rolling my own backup program in mid-2009, cobbling together something in Ruby that theoretically worked. (It even tapped into FSEvents to know which folders to examine for changes–not the most trivial thing for a Ruby script!) In reality it was completely broken by the fact that it wanted to store everything (everything) in a database. (I’m probably talking out of my ass here, but I don’t recall the various NoSQL databases–which might have a chance in heck of being able to store files in the database–being very prominent back then.) I knew it was busted when I wrote it, and never actually used it to store anything.

Fast forward to this last weekend, when I discovered Incremental Backup. This AppleScript file held the core to what I wanted to do years ago–multiple custom incremental backups–and that all boiled down to an rsync flag that I didn’t know existed: --link-dest. That miracle flag tells rsync to look at the specified folder for an existing copy of the file it’s examining–and if the file is unchanged, creates a hard link to the existing file instead of copying it anew. Smart use of that flag allows rsync to effectively mimic Time Machine’s behavior (albeit somewhat slower, as rsync doesn’t use Apple’s FSEvents shortcut)–but with all the flexibility that rsync provides.

Yes, using that script requires you to compile your own version of rsync, at least for now. Yes, it’s a bit clunky (it keeps a log of backup folder names as a quick way to determine the most-recent backup, rather than just examining what folders exist, and you have to be careful to escape special characters, like spaces, in your pathnames), and a tiny bit buggy (if you have a backup source directory whose name contains spaces, it’ll get backed up to a folder named whatever text follows the final space). It’s still the best-looking customizable backup script/program/what-have-you that I’ve found.

I think I’ve managed to hack my own version to sidestep the issues I’ve found… my final hurdle is figuring out how to schedule the script to run regularly. Recent versions of OS X rely on launchd instead of cron. Thing is there only seems to be one launchd editor out there, Lingon, and I’ve had absolutely no luck getting that program to work.

Oh, and the general rsync --link-dest theory should work on all Unix variants, not just Mac OS X. (Looking at the date on that article, I can’t help but wonder why I never thought to Google “rsync time machine” back in the day. I would have saved myself a whole lot of time.)

 

2 Responses to rsync Time Machine

 
  1. […] rsync Time Machine Like this:LikeBe the first to like this. Posted in: rsync, Setup, Synology, Time Machine ← Setting up Time Machine Support on Synology Be the first to start a conversation […]

  2. Elmar says:

    Thanks for the great post! I am also intrigued by rsync and prefer it over Time Machine. Though, I have taken a slightly different approach than outlined here. My backup solution is now mainly based on rsync (without using hard links). I do daily scheduled backups of my Macs on my NAS and the NAS, in turn, is synced weekly over the internet to an offsite NAS.

    I wrote a script to facilitate everything and called it Space Machine ;-).

    I have two posts on my blog about my backup solution. You can also get my script there:

    http://www.relevantcircuits.org/2013/02/a-space-machine-to-safe-immaterial.html
    http://www.relevantcircuits.org/2013/03/space-machine-made-public.html

    Hope this is interesting for people here!

 

Leave a Reply

Your email address will not be published. Required fields are marked *

powered by wordpress