rsync many small files

[Sven Hartrumpf]

Tue, 23 Sep 2008 06:49:33 -0400, magawake wrote:

> At our lab we have storage with many small files. For example a
> directory can contain over 15,000 files and each file averages about
> 75k. I would like to sync this to another filesystem on a different
> server but I am not sure if there is a rsync tuning flag I can use for
> such a intensive job. I am using rsync 3.0.3. Also, I would like to
> use little memory as possible.

Synchronizing many files should be no problem with rsync versions >= 3 due to
new incremental feature; I do this regularly with around 15 million mostly small
files. But: The initial copy might take some days. If so, you might look around
for a somewhat faster unconditional copy method. I prefer cpio here, but cloning
tools might be better suited.

Sven
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url : http://lists.samba.org/archive/rsync/attachments/20080923/0589446d/attachment.bin