how to take least risk on rsync dir

David Bolen db3l at fitlinxx.com
Tue Apr 16 10:44:02 EST 2002


Patrick Hsieh [pahud at pahud.net] writes:

> When rsync dir_A to dir_B, I hope I wont make any change to the original
> dir_B unless the rsync procedure end withour errors, therefore, I hope
> there's somethig like
>
> rsync -av dir_A dir_B_tmp && \
> mv dir_B dir_B.bkup &&
> mv dir_B_tmp dir_B
> 
> This small script can ensure the minimal change time between 2 versions
> of archive. Is this built in the native rsync function? Do I have to
> write scripts myself?

rsync's default behavior ensures this sort of minimal change time, but
only at a per file level.  That is, each file is actually built as a
temporary copy and then only renamed on top of the original file as a
final step.  Of course, that's largely a requirement so rsync can use
the original file as a source for the new file, but it also serves to
preclude interruption of the original file as long as possible.

But if you want the same sort of assurances at something larger than a
file level (e.g., a directory as above), then yes, you need to impose
that on your own.  For example, when backing up databases (where I
need to keep the database backup and transaction logs in sync), I copy
them into a temporary directory and only overlay the primary
destination files when fully done.

The simplest way to do it is close to what you have, but there are a
few things you need to be aware of.

First, you'll want to use the rsync --compare-dest directory so that
it can still find the original files on the destination system for its
algorithm - otherwise it'll send over the entire contents of the
source files and not use what it can from the original files.

Second, you need to realize that by default rsync will only copy files
that have changed (by default based on size/timestamp unless you add
the -c (checksum) option).  So if you do what you have above you'll
end up losing files that hadn't changed since they won't exist in
dir_B_tmp.  You can override this with the -I option at the expense of
a small amount of extra data transferred for the unchanged files.

So you could do something like:

    rsync -av -I --compare-dest=B_tmp_to_B_path dir_A dir_B_tmp

Note that the --compare-dest argument is a relative path to get from
the destination directory (dir_B_tmp in this case) back to the
original source directory.  rsync won't touch the source directory,
but it will use the files within it as masters for the new copy.

This will result in dir_B_tmp being a completely copy of dir_A, using
the original dir_B as a master whenever possible.

This all assumes that you're doing remote copies where the rsync
protocol makes sense (you don't show a remote system in your example).
If you're just making local copies, it would be better to use -W
instead, but you'd still need -I if you wanted files matching those in
the current dir_B to be transferred.  Then again, for a local setup
where you want to update the whole directory, a simple copy may be as
effective as rsync, since you're not benefitting from the selection of
a subset of files.

-- David

/-----------------------------------------------------------------------\
 \               David Bolen            \   E-mail: db3l at fitlinxx.com  /
  |             FitLinxx, Inc.            \  Phone: (203) 708-5192    |
 /  860 Canal Street, Stamford, CT  06902   \  Fax: (203) 316-5150     \
\-----------------------------------------------------------------------/




More information about the rsync mailing list