4TB and "150 000 000" files out of memory error

Matt McCutchen matt at mattmccutchen.net
Tue Jan 15 02:16:29 GMT 2008

On Mon, 2008-01-14 at 17:05 +0100, Sylvain Gargasson wrote:
> I want use you product to migrate 4TB of data (in 150 millions files)
> When rsync make the list of files it full my RAM and after my SWAP and my server crash.

Use a development version of rsync 3.0.0, which has an incremental
recursion mode that avoids keeping a list of all the source files in
memory for the entire run.  The source code for the latest pre-release
of rsync 3.0.0 is at:



More information about the rsync mailing list