how to migrate 40T data and 180M files

Ming Gao gaomingcn at gmail.com
Tue Aug 11 02:14:33 MDT 2009


hi,

I need to migrate 40T data and 180M files from one storage device to another
one, both source and destination will be NFS and mounted to a local suse
linux box.

The first question is that if there is any risk for such a big number of
files? should I divide them into groups and rsync them in parallel or in
serial? If yes, how many groups is better?

The second question is about memory. How much memory should I install to the
linux box? The rsync FAQ(http://rsync.samba.org/FAQ.html#4) says one file
will use 100 bytes to store relevant information, so 180M files will use
about 18G memory. How much memory should be installed totally?

And any other thing I could do to reduce the risk?

Thanks in advance.

Gao, Ming
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.samba.org/pipermail/rsync/attachments/20090811/a43c53b3/attachment.html>


More information about the rsync mailing list