36 Hours of building file list

[Matt McCutchen]

On Fri, 2008-02-08 at 12:41 +0100, Sylvain Gargasson wrote:
> Sorry to post again for my 220 Millions Files in 4TeraBytes.
> 
> I have make a first rsync with online applications.
> 
> Now I want make a over rsync with offline applications before changing
> Disk.
> 
> The problem is: I can't make a offline of more than 6Hours and my
> building file list take 36 Hours.
> 
> I know the only 2 month old files can change between the 2 rsync.
> 
> Have you a special argument for rsync to only build file list with
> file younger than 2 month???
> 
> For my part, I'm going to try with Bash command but I think I can have
> some hard limitation by MAX_ARG!!!
> 
> Please help me again... I'm very sorry but my client have very special
> production.

Paul's suggestion to use --files-from is good.  Use something like "find
SRC -mtime -60" to make the file list.

However you accomplish this copy, I suggest that you take this
opportunity to move the files to an LVM volume.  That way, in the
future, you can take block-level snapshots and move the partition to
another disk as necessary with no downtime.

Matt