Rsync to remote host much faster than 10k/sec speeds on local rsync

[Paul Slootman]

On Thu 21 Jun 2007, Will Smith wrote:

> I run several remote rsyncs off a CentOS 4 server and also back up between
> two drives on the same server.
> 
> Performance is far better on the remote rsyncs and the local rsync only
> manages 10k/sec despite being on modern hardware.

If the two filesystems you're syncing between are on the same disks,
you're causing a lot of disk thrashing during the local sync. That's
probably the cause of the slowdown (it has to seek to and fro between
the source and the destination).

> What I have found is that if I empty the destination directory on the local
> backup disk, so rsync has to start from scratch and copy all files,
> performance is as to be expected but this isn't feasible for a regular
> backup routine as there are almost a million files.

That's because it doesn't need to check the existing files then.
BTW, don't be fooled by the bytes/sec statistic... If nothing has
changed, that will be basically 0 excluding any overhead. That doesn't
mean it's very slow...  More interesting is the wall clock time taken.

> I have also noticed that the load on the local rsync is much higher than the
> remote rsyncs, with 90%+ cpu usage. I have tried --bwlimit 100 in a vague

Is that including or excluding wait for IO time, which actually is CPU
idle time that could be used for processor-intensive tasks while the
disk is seeking.

> hope it might affect disk io but it didn't help. Rsync version is 2.6.9
> 1.el4.rf from rpm. The directory size of the destination backup directory is
> around 30gb. There is one directory where the majority of the files are and
> they are jpgs between 5k-150kb.
> 
> The command I am running is simple: /usr/bin/rsync -a --bwlimit=100 --stats
> /src/dir1 /src/dir2 /backup/dir

As you're not preserving hardlinks (-H), you may observe a significant
speedup when using version 3.0.0 (currently only available from the CVS
or as a daily snapshot, it's not been released yet). Prior versions of
rsync first gather all the file meta data from the source, then from the
destination, and then start comparing the lists to see what needs to be
transferred. 3.0.0 does the comparing as soon as the first couple of
directories have been read on both sides, and that can really speed
things up. Unfortunately that speedup doesn't work if hardlinks are to
be preserved (although I have hope that some day it might :-)



Paul Slootman