Nice little performance improvement
Matt McCutchen
matt at mattmccutchen.net
Thu Oct 15 21:26:42 MDT 2009
On Thu, 2009-10-15 at 19:07 -0700, Mike Connell wrote:
> Today I tried the following:
>
> For all subsub directories
> a) Fork a "du -s subsubdirectory" on the destination
> subsubdirectory
> b) Run rsync on the subsubdirectory
> c) repeat untill done
>
> Seems to have improved the time it takes by about 25-30%. It looks
> like the du can
> run ahead of the rsync...so that while rsync is building its file
> list, the du is warming up
> the file cache on the destination. Then when rsync looks to see what
> it needs to do
> on the destination, it can do this more efficiently.
Interesting. If you're not using incremental recursion (the default in
rsync >= 3.0.0), I can see that the "du" would help by forcing the
destination I/O to overlap the file-list building in time. But with
incremental recursion, the "du" shouldn't be necessary because rsync
actually overlaps the checking of destination files with the file-list
building on the source.
--
Matt
More information about the rsync
mailing list