[ale] Versioning backup solution

[Stephen Cristol]

On Jul 28, 2005, at 12:16 AM, Jim Popovitch wrote:
> I presently use rsync over ssh to remotely backup a few systems.   I am
> looking to switch to a secure remote versioning backup system, the
> perfect one would be a combination of cvs and rsync.  Any ideas or
> suggestions?

I currently use rsync to remotely backup some userspace data. I've been 
experimenting with using additional rsync features (requiring rsync 
2.6.1 or later) to preserve previous backups. In a nut shell, after 
doing a first backup, that is named with a timestamp. I then create a 
symbolic link named ".previous" to the that data set. Subsequent 
backups with rsync have the following additional options:

     --delete --link-dest=$path/.previous

where $path is the directory where this set of backups are stored. If 
the backup is successful, ".previous" is repointed, logs are updated, 
etc.

These options cause rsync to create a new backup that is an increment 
relative to ".previous". Files in the previous backup that have not 
changed are created in the new backup as links (real links, not 
symbolic links) to the files in the previous backup (--link-dest). 
Files that no longer exist in the source are not copied or linked in 
the new backup (--delete).

The end result is a collection of directories in $path named with 
timestamps indicating when that backup occurred. I've got a script that 
mostly does this, but it still has some rough edges.

Upsides to this approach:

- all the security of rsync
- the efficiency of rsync in determining what to transfer
- efficient storage of native file formats

Downsides to this approach:

- this can create an enormous number of links and I know of no tools to 
manage them easily (e.g., if you want to prune the data set)
- managing disk space becomes more challenging ("du" still gives 
correct answers; if you ask the right questions, they can be useful 
answers, too)

S

-- 
Stephen Cristol
cristol at emory.edu