Future RSYNC enhancement/improvement suggestions

[David Bolen]

Martin Pool [mbp at samba.org] writes:

> TCP connections don't timeout anyhow.  Possibly a dial-on-demand line
> or a firewall might drop the connection, but there should be enough
> traffic that this is not a problem.

Unless you have quite large files, in which case there can be a
lengthy period (particularly if the file is being accessed across a
local network) while checksums are computed where there is no traffic
at all.  For a while (when we had slow drives and a 10BaseT network)
we could take 20-30 minutes for checksum computation on a 500-600MB
database file with 4K blocks.  And our long distance dialup call was
completely idle during that period.

At the time, I had planned on experimenting with the sort of changes
that Stefan's recent response to this thread suggested - transmitting
the checksum information as it was computed rather than building it up
before sending anything.

As it turns out, we upgraded to a faster RAID setup, and bumped the
needed machines to 100BaseT, an the time went down to somewhere
between 5-10 minutes typically, so the priority of making the changes
dropped.  But I do still think it would be a useful adjustment to the
data flow within rsync at some point.  I can't remember just how major
the surgery looked to get the transmission to occur at the point of
computation though.

-- David

/-----------------------------------------------------------------------\
 \               David Bolen            \   E-mail: db3l at fitlinxx.com  /
  |             FitLinxx, Inc.            \  Phone: (203) 708-5192    |
 /  860 Canal Street, Stamford, CT  06902   \  Fax: (203) 316-5150     \
\-----------------------------------------------------------------------/