Rsync Performance

[Wayne Davison]

On Fri, Jul 16, 2004 at 12:36:45PM -0700, Joe Eckstrom wrote:
> we would definitely want the ability to resume an incomplete download.

That's what the --partial option indicates, though it does move the
partial file into place awaiting the next transfer (it does not auto-
resume).

> The central rsync server could run Linux if there was a performance
> benefit.

There has been a large performance hit running rsync under Windows in
older versions due to the large number of small I/O calls that rsync
used to make.  This has been largely alleviated in modern versions due
to an I/O buffering patch, but I'm not sure how things stack up these
days.  I'd imagine that Linux would be a little faster, but that's just
a guess.

> Does rsync running as a daemon take advantage of multiple processors?

Sure -- each connection uses a new process, so it will divide the load
up between them.

> What kind of overhead does compression add to a session?

I haven't measured this, but it obviously trades CPU for net bandwidth.
In your tests if the CPU is maxing out before the bandwidth, turn
compression off.  If the opposite is true, turn it on.

> Does wrapping rsync in ssh cause as much of a performance hit as
> compression?  

I think it takes more CPU than rsync doing compression on its own.

> What's the difference between --bwlimit and --bwlimit-mod?

The --bwlimit-mod option is a suggested patch that debian applies to
their packaged version.  They have it documented in the latest release,
but it was not documented earlier.  I recall that it is a better
limiting algorithm in most circumstances compared to the old --bwlimit
option (which was pretty crude), however the CVS version of rsync has an
improved --bwlimit algorithm that works even better than both of them,
so --bwlimit-mod should go away in debian's package of the next rsync
release (whenever that will be).

..wayne..