long startup times for large file systems
Steve Howie
showie at uoguelph.ca
Thu Dec 18 09:12:31 EST 2003
Howdy,
Rsync has been churning away for 45 mins, presumably bulding an in-core
list of files to be copied to the destination. This is a very very large
filesystem we are copying locally - approximately 4.2million files
(WebCT). The resident process size for rsync has grown to 72Mb. - is
this normal behaviour for a file system this size, and does rsync have
the ability to handle such a large number of files? The filesystem size
is relatively modest - 20gb. or so, but the millions of small files
might explain while its taking so long.
I presume that once rsync has built it's in-memory list of files it
proceeds to copy the whole shebang over. This is the initial copy,
subsequent copies should be much faster.
Also, are there any options suitable for turning off that would speed up
the whole process? We're using (as root)
# rsync -a --progress --stats /global/webct/ /target
Solaris 8 and rsync 2.5.7
Thanks
Scotty
More information about the rsync
mailing list