[PATCH] Batch-mode rewrite

[Chris Shoemaker]

On Sun, Jul 18, 2004 at 05:25:18PM -0700, Wayne Davison wrote:
> Seems like the two choices we have are:
> 
> (1)  Force the excludes into the batch file and read them in the local-
> to-local batch-reading transfer.
> 
> (2)  Require the user to re-specify the excludes if they want the same
> update (allowing them to skip them as they see fit).  This route would
> cause the deletes to remove more files than the original transfer if the
> user failed to re-specify the excludes and they used --delete.  Also, if
> there were excluded symlinks, directories, and devices, the generator
> would not know to skip them in the batch-reading run unless the same
> excludes were specified.
> 
> So, perhaps we should go ahead and save off the exclude list in the
> batch file and force read_batch mode to read them?  It should be as
> simple as an extra call to "send_exclude_list(batch_fd);" and the
> addition of a special recv_exclude_list() call for read_batch mode (and
> the removal of the --include/--exclude options out of the argv file).

I'm leaning in this direction.

> One other thing that I noticed is that the synchronization between the
> generator and the receiver is no longer present, so a batch-reading run
> can possibly do some things in the receiver too soon (for instance, if
> the generator hasn't gotten around to creating the required parent dirs
> for the receiver).  There are two solutions to this:

I see.

> 
> (1)  Don't /dev/null the data from the generator, but instead monitor it
> and only let the receiver process a file when its number has been
> requested from the generator.
> 
> (2)  Use another way to convey the same information, like the idea below.
> 
> There is a diff in the patches dir called g2r-basis-filename.diff
> because it sends the name that the generator found for the basis file to
> the receiver via an extra pipe that gets created before the two fork
> (the idea is to avoid duplicating the same basis-file search in the
> receiver and risking having it find a different file than what was used
> to generate the checksums -- something that is particularly useful for
> both the multiple-compare-dest diff and the fuzzy-name matching diff).
> I modified this g2r patch to also convey to the receiver what file-list
> index the name refers to (but only in batch mode) so that the receiver
> can notice when thing aren't quite in sync (i.e. if the batch data
> doesn't have exactly the same items that the generator wants to update).
> 
> One question was prompted by my work on this patch:
> 
> The sending of the extra file-list index value is only enabled in
> batch-reading mode (and  indeed, the extra basis-name pipe is not
> normally turned on unless an option such as --compare-dest or
> --read-batch is specified).  It might be advantageous to always convey
> this extra index-number information from the generator to the receiver
> since it would guard against a receiver that is sending an update that
> the generator didn't request, but I can't think of a reason to do this.
> 
> Any thoughts on any of this?

Could a simplified version of this index notification take place over
the existing error-pipe pair?
-chris

> 
> ..wayne..