long startup times for large file systems

Steve Howie showie at uoguelph.ca
Thu Dec 18 09:12:31 EST 2003


Rsync has been churning away for 45 mins, presumably bulding an in-core 
list of files to be copied to the destination. This is a very very large 
filesystem we are copying locally - approximately 4.2million files 
(WebCT). The resident process size for rsync has grown to 72Mb. - is 
this normal behaviour for a file system this size, and does rsync have 
the ability to handle such a large number of files? The filesystem size 
is relatively modest - 20gb. or so, but the millions of small files  
might explain while its taking so long.

I presume that once rsync has built it's in-memory list of files it 
proceeds to copy the whole shebang over. This is the initial copy, 
subsequent copies should be much faster.

Also, are there any options suitable for turning off that would speed up 
the whole process? We're using (as root)

           # rsync -a --progress --stats   /global/webct/     /target

Solaris 8 and rsync 2.5.7



More information about the rsync mailing list