how to migrate 40T data and 180M files

[Jan-Benedict Glaw]

On Tue, 2009-08-11 16:14:33 +0800, Ming Gao <gaomingcn at gmail.com> wrote:
> I need to migrate 40T data and 180M files from one storage device to another
> one, both source and destination will be NFS and mounted to a local suse
> linux box.
> 
> The first question is that if there is any risk for such a big number of
> files? should I divide them into groups and rsync them in parallel or in
> serial? If yes, how many groups is better?
> 
> The second question is about memory. How much memory should I install to the
> linux box? The rsync FAQ(http://rsync.samba.org/FAQ.html#4) says one file
> will use 100 bytes to store relevant information, so 180M files will use
> about 18G memory. How much memory should be installed totally?
> 
> And any other thing I could do to reduce the risk?

There are no specific risks these days I think, but it sounds like
this is a task like "copy over anything", so it's merely mount both
filesystems and use two `tar' instances with a pipe in between...

MfG, JBG

-- 
      Jan-Benedict Glaw      jbglaw at lug-owl.de              +49-172-7608481
 Signature of:                      http://perl.plover.com/Questions.html
 the second  :
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: Digital signature
URL: <http://lists.samba.org/pipermail/rsync/attachments/20090811/b026dd4e/attachment.pgp>