Rsync Performance

[Jason Haar]

On Thu, Jul 22, 2004 at 07:33:21PM -0700, Wayne Davison wrote:
> On Fri, Jul 23, 2004 at 02:15:04PM +1200, Jason Haar wrote:
> > is there any intention of a "new improved" "--partial" option whereby
> > any failed uploads are kept as temp files
> 
> I had been contemplating whether we need a new option for this or not.
> One idea would be to change the behavior when --partial was combined
> with --temp-dir in that the partial file would be renamed to the normal
> name in the temp dir on abnormal termination and the directory would be
> preferentially checked for the basis file (kinda like --compare-dest).

Strange you should say that - I for one assumed that was what would happen
if you invoked "--partial --temp-dir"! I don't know why - but I did.

> However, it might be better to create a new option named --partial-dir
> that would indicate that this new behavior was desired.  (For instance,
> it might surprise someone that the --temp-dir could receive a non-temp
> format name in it.)

Yes - that sounds good. It should pretty pretty clean: if you call it as
"rsync --partial-dir=/path/.rsync-partial/" then it could:

1. create a new subdir, 
2. rsync to that, 
3. keep files on failure, 
4. carry on when called again
5. when finished, move to "real" location

Sanity checks could include:

a. deleting temp file if it's date is different than src file (means
   something weird has happened, so assume it's not there and start 
   from scratch)
b. --partial-dir should have to be part of the dst tree - otherwise you
   potentially have a security risk. e.g. "--partial-dir /tmp" would mean
   someone could create /tmp/etc/passwd and you could end up overwriting your
   /etc/passwd file (if that's what that particular rsync job was
   replicating of course!). So maybe "--partial-path" cannot start with ^/?
   or "^../"?.
c. on any error, delete and start from scratch again?

Keen idea :-)

-- 
Cheers

Jason Haar
Information Security Manager, Trimble Navigation Ltd.
Phone: +64 3 9635 377 Fax: +64 3 9635 417
PGP Fingerprint: 7A2E 0407 C9A6 CAF6 2B9F 8422 C063 5EBB FE1D 66D1