[PATCH] Batch-mode rewrite

Wayne Davison wayned at samba.org
Fri Jul 23 01:36:27 GMT 2004


On Wed, Jul 21, 2004 at 03:54:11PM -0400, Chris Shoemaker wrote:
> What data exactly?  I thought:
> 	1) all recv-to-gen communications went through the error_pipe[] fds.

Yes, that became true when I got rid of the extra pipe that used to
separate the redo values from the error messages.

> 	2) the only meaningful communications were redo requests and
> 	"I'm done".

It depends on which side is the client.  Since we made the read-batch
side always the client then the errors/warnings get output to the tty
directly (for a client sender, which is the default for non-batch local
transfers, the pipe also contains any error messages from the receiver).
There is also a patch in the patches dir that sends a "success" message
for every correctly transferred file (delete-sent-files.diff).

> I thought we could skip the redos and fake the "I'm done".

Yes, it's possible to turn off the communication, which is what I was
talking about -- I was mentioning what things would need to be
special-cased.

> Ah, I see what you mean about the directory tweaks.

And don't forget the hard-link post-processing -- it would also need to
happen after the receiver finished its job.

> Just to clarify, I don't have anything against the index notification
> style gen/recv syncronization.  If you think that's better, then let's
> go that way.

In the future there may be a need for data to be communicated back from
the generator to the receiver (i.e. if the --fuzzy patch get improved
enough to make it worthwhile), so I think in the long run that having
the two things running in parallel (like normal) will be the easiest
thing to maintain.  We can certainly revisit the issue later, but for
now I have the code in the generator sending the normal index bytes down
the output pipe (it just avoids sending the checksum data), and some
special-case read-batch code in the receiver that reads the from-the-
generator pipe and only proceeds with an update if the generator said
it was time.

One interesting thing that fell out of this was the ability to skip any
part of a batched update that had already been done.  For instance, if
you run the ./BATCH.sh file and terminate it (for some reason), you can
run it again and it will skip all the already-performed updates.  (One
exception: if --partial had been specified the interrupted file will
probably not update correctly, but rsync should tell you when that
happens.)

..wayne..


More information about the rsync mailing list