FW: rsync performance

[jw schultz]

On Sat, Sep 13, 2003 at 07:46:05PM -0700, Ben Escoto wrote:
> On Fri, 12 Sep 2003 17:05:09 -0700 jw schultz <jw at pegasys.ws> wrote:
> > Rsync is not an efficient local copy utility.  It can be
> > used for local copying but local and high-bandwidth network
> > speed is sacrificed for low-bandwidth performance and for
> > data integrity.
> 
> I've been surprised at how fast rsync can copy locally.  For instance,
> 
> http://mail.nongnu.org/archive/html/rdiff-backup-users/2002-12/msg00066.html
> 
> finds 'rsync -aH --delete' faster than 'cp -a'.  Perhaps though this
> was some idiosyncracy of my test (the main purpose wasn't to benchmark
> rsync anyway).

The referenced mail message describes the benchmark as:
| The directory backed up or restored had 10000 1-byte files

That isn't a very good benchmark.  10,000 files is not that
many and being 1 byte means that all that is measured is the
filesystem meta-data, node-creation time and overhead.

That the test used the --delete option indicates that some
percentage of the files would have not been touched by
rsync.  My guess is that the unmodified files account for
cp -a being slower; rsync processed fewer files than cp.
The fact that the benchmark description does not indicate
the actual rate of change (a determining factor for rsync
and, i assume, rdiff-backup) makes it decidedly dubious.

The test may well be invalidated by caches.

I believe the speed complaint had to do with files that have
significant amounts of data in them.


-- 
________________________________________________________________
	J.W. Schultz            Pegasystems Technologies
	email address:		jw at pegasys.ws

		Remember Cernan and Schmitt