Be Careful With rsync –delete

I’ve been using an rsync-based script to backup my iPhoto library to another computer, just in case my Powerbook blows up or something, so I don’t lose every photo I’ve taken for the last 4 years. It’s worked flawlessly, run every now and then from home, to squirt changes in my Pictures directory to my desktop at work, where I further back it up on an external drive using another rsync script.

But, now that I’ve added Aperture to my work desktop, the script I had been using became suddenly quite dangerous. As in, destructively dangerous. And I hadn’t thought to check out the script in awhile.

So, I ran it last night. It faithfully chugged through my ‘book’s ~/Pictures directory, so I walked away as it started working.

Then, late last night I did a quick scroll through the rsync log, and to my horror found a whole bunch of stuff like “Deleting: ~/Pictures/Aperture Library.aplibrary/...”

Holy. Crap.

I’d accidentally told my computer to nuke my Aperture library on the work desktop.

So, I cracked open the “rsyncpics” script, and lo and behold, I’d left in the “–delete” flag. Which wasn’t a problem – it was actually desired – when I was only using iPhoto. But since Aperture isn’t on the Powerbook, the “–delete” flag told rsync to nuke anything on the desktop that isn’t on the powerbook. Like the Aperture library. Doh.

Thankfully, I also periodically backup my work desktop’s home directory to an external drive, so had a slightly out of date version of the Aperture library which I could just copy back into place. But it was missing everything after Ken Ryba’s session from a week ago. I’d copied the best of the “Campus Tour” photos to my home iPhoto library, so it’s not fatal, but a good lesson learned.

The moral of the story is: be careful with rsync, especially when using --delete. I’ve learned my lesson, and have resurrected most of the lost photos. I’ve also added an Aperture Vault on the external drive. They offer these backup tools, so why not use them?

Update: Between the various locations I’d copied files, and a healthy application of the awesome Flickr Backup utility, all photos worth saving have been restored. Whew.

I’ve been using an rsync-based script to backup my iPhoto library to another computer, just in case my Powerbook blows up or something, so I don’t lose every photo I’ve taken for the last 4 years. It’s worked flawlessly, run every now and then from home, to squirt changes in my Pictures directory to my desktop at work, where I further back it up on an external drive using another rsync script.

But, now that I’ve added Aperture to my work desktop, the script I had been using became suddenly quite dangerous. As in, destructively dangerous. And I hadn’t thought to check out the script in awhile.

So, I ran it last night. It faithfully chugged through my ‘book’s ~/Pictures directory, so I walked away as it started working.

Then, late last night I did a quick scroll through the rsync log, and to my horror found a whole bunch of stuff like “Deleting: ~/Pictures/Aperture Library.aplibrary/...”

Holy. Crap.

I’d accidentally told my computer to nuke my Aperture library on the work desktop.

So, I cracked open the “rsyncpics” script, and lo and behold, I’d left in the “–delete” flag. Which wasn’t a problem – it was actually desired – when I was only using iPhoto. But since Aperture isn’t on the Powerbook, the “–delete” flag told rsync to nuke anything on the desktop that isn’t on the powerbook. Like the Aperture library. Doh.

Thankfully, I also periodically backup my work desktop’s home directory to an external drive, so had a slightly out of date version of the Aperture library which I could just copy back into place. But it was missing everything after Ken Ryba’s session from a week ago. I’d copied the best of the “Campus Tour” photos to my home iPhoto library, so it’s not fatal, but a good lesson learned.

The moral of the story is: be careful with rsync, especially when using --delete. I’ve learned my lesson, and have resurrected most of the lost photos. I’ve also added an Aperture Vault on the external drive. They offer these backup tools, so why not use them?

Update: Between the various locations I’d copied files, and a healthy application of the awesome Flickr Backup utility, all photos worth saving have been restored. Whew.

8 thoughts on “Be Careful With rsync –delete”

  1. I know it’s an old thread but I’ve been replacing rsync with Unison for those situations where I might be updating the destination filesystem as well as the source. Supports full two-way syncing, merging the changes from both directions, and can even support third-party diff tools to let you manually reconcile situations where a (text) file has been updated in both locations since the last sync run. Obviously you should try to avoid this anyway, my procedure is:

    sit down at machine x
    sync with y
    have a work session
    sync with y

    sit down at machine y
    sync with x
    work
    sync with x

    where I’m working on both x and y at the same time, I make sure to work only on certain projects/applications/filesets on a given machine, if I happen to be using say Firefox (portable) on both machines, I make a mental note as to which one is the “master” for that work session, and just hold down the appropriate arrow key in the Unison GUI before pressing “Go” for the sync job.

    Also great for quick-and-dirty backups to/from USB drives

  2. On my storage server I have a folder backup/ that contains subfolders that correspond to each machine I would like to backup.

    Then I use something like this when invoking rsync:
    rsync -e ssh -avz –delete Folder/To/Backup storageserver:/backup/folder/hostfolder

    this way, /backup/folder/hostfolder will always only have the files for the computer I’m backing up, and won’t delete any files from other machines 🙂

Comments are closed.