how to migrate 40T data and 180M files

[Michal Suchanek]

2009/8/11 Brett Worth <brett at worth.id.au>:
> Ming Gao wrote:

>> And any other thing I could do to reduce the risk?
>
> Once you get all the file copied over by whatever means then a final rsync would be good
> to get all the metadata lined up.  Based on your file count I'd strongly recommend you
> break up  the filesystem into smaller problems.
>
Speaking of metadata tar should have an option to preserve these, and
unlike rsync it tends to preserve file names as well. When I tried to
copy a directory with rsync last time there was some file name
transcoding feature that would clobber the file names in interesting
ways. I did not manage to turn it off completely so it would always
clobber file names which had invalid code sequences in the encoding in
use. I would not assume that 180M files are all correctly named in the
same encoding.

This is one more problem to consider.

HTH

Michal