A few times on Twitter, I’ve mentioned how “easy” it is to move stuff between servers using the rsync shell command. It’s actually an extremely powerful program for synchronizing two directories – even if they’re not on the same volume, or even on the same computer. To do this, you’ll need to login to one of the servers via SSH. Once there, invoke the geeky incantation:
rsync -rtlzv --ignore-errors -e ssh . username@hostname:/path/to/directoryWhat that basically says is, “run rsync, and tell it to recursively copy all directories, preserving file creation and modification times, maintaining proper symlinks (for aliases and stuff like that), compress the files as they’re being copied in order to save bandwidth, and provide verbose updates as you’re doing it. Use SSH as the protocol, to securely transfer stuff from the current directory to the server ‘hostname‘ using the username ‘username‘. On that destination server, stuff the files and directories into ‘/path/to/directory‘”
What you’ll need to change:
.– if you want to specify a full path to the source directory, put it in place of the.(which means “here” in shellspeak)username– unless your username on the destination server is “username” – you’ll be prompted for the password for that account after hitting return and starting the program working.hostname– the IP address or domain name of the server you want to send the files to./path/to/directory– I’m always a bit fuzzy on this. Can it be a relative path? from where? So I just specify the full path to where I want the files to go. Something like/home/dnorman/
Because it compresses files, it’s actually pretty efficient at moving a metric boatload of stuff between servers. I’ve used this technique to easily migrate from Dreamhost to CanadianWebHosting.com and I use it regularly to move files around on campus. I use a variation of this technique to regularly backup servers as well – the beauty of rsync is that it only copies files that have been added or modified, so backing up a few gigs worth of server really only involves transferring a few megabytes of files, and can be done routinely in a matter of minutes or seconds.
Sensei, you rock. Thanks.
Can you use rsynch to backup online from a folder on a local hard drive?
@scott – no problem. I’d been meaning to write this up for awhile, and got tired of saying how easy it is 🙂
@ian – I use it all the time to back up my servers to my desktop Mac, which then backs it up again via Time Machine. You need to have the visible IP address of your desktop, and have SSH running on it, but it works great.
Or, drop the -e ssh part and just specify local paths to copy or backup local directories.
rsync is scary powerful, and can do all kinds of great and funky things, as described in the man page – it can also nuke files pretty quickly if you’re not careful…
Scott’s right, you do indeed rock. 🙂 Everywhere else I looked didn’t give me the one beautiful line I wanted along with an explanation. I’ll go back and study the man page later, but right now I just don’t have the time.
Thanks 🙂
Well the title says ‘how i MOVE stuff’ so here is how you MOVE stuff (and not COPY) : add the –remove-source-files option and then rsync will delete the source files after copying them
yeah. I had that in an earlier version of the post, but got nervous about people copying/pasting and messing up without an undo… you’re totally right, though – the command as listed just clones stuff.