how to migrate 40T data and 180M files

Michal Suchanek hramrach at
Tue Aug 11 05:31:32 MDT 2009

2009/8/11 Brett Worth <brett at>:
> Ming Gao wrote:

>> And any other thing I could do to reduce the risk?
> Once you get all the file copied over by whatever means then a final rsync would be good
> to get all the metadata lined up.  Based on your file count I'd strongly recommend you
> break up  the filesystem into smaller problems.
Speaking of metadata tar should have an option to preserve these, and
unlike rsync it tends to preserve file names as well. When I tried to
copy a directory with rsync last time there was some file name
transcoding feature that would clobber the file names in interesting
ways. I did not manage to turn it off completely so it would always
clobber file names which had invalid code sequences in the encoding in
use. I would not assume that 180M files are all correctly named in the
same encoding.

This is one more problem to consider.



More information about the rsync mailing list