Improving the rsync protocol (RE: Rsync dies)

[Bob Bagwill]

At 08:37 PM 5/20/02, you wrote:
>Doing so requires an upfront scan of the entire destination tree, and
>for the client to hold all this information in memory.

[I accidently sent this to Martin instead of the list...]

I wonder if it would be useful to have a partner program that would index
the rsync'ed file system (like Tripwire).  If the index file exists (or
meets other defined criteria), rsync is allowed to assume that it's current
and can download or do queries against the remote database.  I would guess
that most targets of huge rsync's "expect" the rsync, so that pre-indexing
the target file system would not be a burden.

If both sides did the indexing, the source side could rsync over the target's
database, zip through the comparison, compile the transfer list, and
copy away.

I think it's important to distinguish between the ad-hoc rsync'ing of a few
hundred little files, and the carefully planned rsync'ing of a hundred
thousand files containing gigabytes of data.


--
Bob Bagwill <mailto:bagwill at nist.gov>