Fix for batch mode (was Re: batch mode maintainability)
josb at cncdsl.com
Sat Feb 2 06:23:33 EST 2002
Thanks Dave for your comments, I have incorporated your feedback in the patch
below. Please review.
On Fri, Feb 01, 2002 at 09:49:07AM -0600, Dave Dykstra wrote:
> Hmm, I wonder if it would be easy to use rsync's compression library to
> compress the whole flist, csum, and delta files on the fly. That would
> certainly be more convenient.
Yes it would. Any ideas on how to go about this? I'm still (re)reading the
rsync papers to ge a better idea of the program's code structure.
RCS file: /cvsroot/rsync/rsync.1,v
retrieving revision 1.108
diff -u -r1.108 rsync.1
--- rsync.1 29 Jan 2002 21:52:56 -0000 1.108
+++ rsync.1 1 Feb 2002 19:21:20 -0000
@@ -306,8 +306,8 @@
--log-format=FORMAT log file transfers using specified format
--password-file=FILE get password from FILE
--bwlimit=KBPS limit I/O bandwidth, KBytes per second
- --read-batch=FILE read batch file
- --write-batch write batch file
+ --write-batch=PREFIX write batch fileset starting with PREFIX
+ --read-batch=PREFIX read batch fileset starting with PREFIX
-h, --help show this help screen
@@ -801,12 +801,14 @@
result is an average transfer rate equalling the specified limit\&. A value
of zero specifies no limit\&.
-Apply a previously generated change batch\&.
-Generate a set of files that can be transferred
-as a batch update\&.
+Generate a set of files that can be transferred as a batch update\&. Each
+filename in the set starts with \fBPREFIX\fP. See the "BATCH MODE" section
+Apply a previously generated change batch, using the fileset whose filenames
+start with \fBPREFIX\fP\&. See the "BATCH MODE" section for details.
.SH "EXCLUDE PATTERNS"
@@ -913,29 +915,80 @@
\fBNote:\fP Batch mode should be considered experimental in this version
of rsync\&. The interface or behaviour may change before it stabilizes\&.
-The following call generates 4 files that encapsulate the information
-for synchronizing the contents of \fBtarget_dir\fP with the updates found in
+Batch mode can be used to apply the same set of updates to many
+identical systems\&. Suppose one has a tree which is replicated on a
+number of hosts\&. Now suppose some changes have been made to this
+source tree and those changes need to be propagated to the other
+hosts\&. In order to do this using batch mode, rsync is run with the
+write-batch option to apply the changes made to the source tree to one
+of the destination trees\&. The write-batch option causes the rsync
+client to store the information needed to repeat this operation against
+other destination trees in a batch update fileset (see below)\&. The
+filename of each file in the fileset starts with a prefix specified by
+the user as an argument to the write-batch option\&. This fileset is
+then copied to each remote host, where rsync is run with the read-batch
+option, again specifying the same prefix, and the destination tree\&.
+Rsync updates the destination tree using the information stored in the
+batch update fileset\&.
+The fileset consists of 4 files:
+<prefix>\fB.rsync_argvs\fP command-line arguments
+<prefix>\fB.rsync_flist\fP rsync internal file metadata
+<prefix>\fB.rsync_csums\fP rsync checksums
+<prefix>\fB.rsync_delta\fP data blocks for file update & change
+The .rsync_argvs file contains a command-line suitable for updating a
+source tree using that batch update fileset\&. It can be executed using
+a Bourne(-like) shell, optionally passing in an alternate source tree
+pathname\& which is then used instead of the original path\&. This is
+useful when the source tree path differs from the original source tree
+Generating the batch update fileset once saves having to perform the
+file status, checksum and data block generation more than once when
+updating multiple source trees\&. Multicast transport protocols can be
+used to transfer the batch update files in parallel to many hosts at
+once, instead of sending the same data to every host individually\&.
-$ rsync --write-batch [other rsync options here] \e
+$ rsync --write-batch=some_update [other rsync options] \e
+$ rsync --read-batch=some_update [other rsync options] \e
+The read-batch option expects the destination tree it is meant to update
+to be identical to the destination tree that was used to create the
+batch update fileset\&. When a difference between the destination trees
+is encountered the update will fail at that point, leaving the
+destination tree in a partially updated state\&. In that case, rsync can
+be used in its regular (non-batch) mode of operation to fix up the
+The rsync version used on all destinations should be identical to to the
+one used on the original destination\&.
+The -z/--compress option does not work in batch mode and yields a usage
+error\&. A separate compression tool can be used instead to reduce the
+size of the batch update files for transport to the destination\&.
+The -n/--dryrun option does not work in batch mode and yields a runtime
-The generated files are labeled with a common timestamp:
-\fBrsync_argvs\&.<timestamp>\fP command-line arguments
-\fBrsync_flist\&.<timestamp>\fP rsync internal file metadata
-\fBrsync_csums\&.<timestamp>\fP rsync checksums
-\fBrsync_delta\&.<timestamp>\fP data blocks for file update & change
-See \fBhttp://www\&.ils\&.unc\&.edu/i2dsi/unc_rsync+\&.html\fP for papers and technical
+See \fBhttp://www\&.ils\&.unc\&.edu/i2dsi/unc_rsync+\&.html\fP for papers and
.SH "SYMBOLIC LINKS"
Jos Backus _/ _/_/_/ Santa Clara, CA
_/ _/ _/
_/ _/ _/ _/
josb at cncdsl.com _/_/ _/_/_/ use Std::Disclaimer;
More information about the rsync