Improving the rsync protocol (RE: Rsync dies)

[Dave Dykstra]

On Fri, May 17, 2002 at 01:42:31PM -0700, Wayne Davison wrote:
> On Fri, 17 May 2002, Allen, John L. wrote:
> > In my humble opinion, this problem with rsync growing a huge memory
> > footprint when large numbers of files are involved should be #1 on
> > the list of things to fix.
> 
> I have certainly been interested in working on this issue.  I think it
> might be time to implement a new algorithm, one that would let us
> correct a number of flaws that have shown up in the current approach.
> 
> Toward this end, I've been thinking about adding a 2nd process on the
> sending side and hooking things up in a different manner:

Be sure to see Tridge's writeup of his plan to solve these problems over
three years ago at
    http://lists.samba.org/pipermail/rsync/1999-January/000963.html

It also includes using a fourth process so it has many similarities to
your plan.

I have personally never taken the time to learn the gory details of how
rsync transfers files, and you have taken the time so as a result you know
a lot more about this than I do.  I agree with Tim that you're the person
who has the best chance to make it work at this point.

I do shudder when I read about Martin's plans for a complete redesign
because I have a lot of doubts about how it will affect performance.  The
only reason that Rsync is as popular as it is today is because of its
performance, and if it gets significantly worse people simply won't use
it.  I think an evolution of the current code base is much more likely to
be able to keep the good performance than a complete redesign.

- Dave Dykstra