36 Hours of building file list

Matt McCutchen matt at mattmccutchen.net
Fri Feb 8 16:35:06 GMT 2008

On Fri, 2008-02-08 at 12:41 +0100, Sylvain Gargasson wrote:
> Sorry to post again for my 220 Millions Files in 4TeraBytes.
> I have make a first rsync with online applications.
> Now I want make a over rsync with offline applications before changing
> Disk.
> The problem is: I can't make a offline of more than 6Hours and my
> building file list take 36 Hours.
> I know the only 2 month old files can change between the 2 rsync.
> Have you a special argument for rsync to only build file list with
> file younger than 2 month???
> For my part, I'm going to try with Bash command but I think I can have
> some hard limitation by MAX_ARG!!!
> Please help me again... I'm very sorry but my client have very special
> production.

Paul's suggestion to use --files-from is good.  Use something like "find
SRC -mtime -60" to make the file list.

However you accomplish this copy, I suggest that you take this
opportunity to move the files to an LVM volume.  That way, in the
future, you can take block-level snapshots and move the partition to
another disk as necessary with no downtime.


More information about the rsync mailing list