rsync many small files
Sven.Hartrumpf at FernUni-Hagen.de
Tue Sep 23 11:07:10 GMT 2008
Tue, 23 Sep 2008 06:49:33 -0400, magawake wrote:
> At our lab we have storage with many small files. For example a
> directory can contain over 15,000 files and each file averages about
> 75k. I would like to sync this to another filesystem on a different
> server but I am not sure if there is a rsync tuning flag I can use for
> such a intensive job. I am using rsync 3.0.3. Also, I would like to
> use little memory as possible.
Synchronizing many files should be no problem with rsync versions >= 3 due to
new incremental feature; I do this regularly with around 15 million mostly small
files. But: The initial copy might take some days. If so, you might look around
for a somewhat faster unconditional copy method. I prefer cpio here, but cloning
tools might be better suited.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Size: 189 bytes
Desc: not available
Url : http://lists.samba.org/archive/rsync/attachments/20080923/0589446d/attachment.bin
More information about the rsync