issue with large number of files

[jw schultz]

On Wed, Oct 22, 2003 at 03:22:32PM +0200, Sven Geisler wrote:
> Hi,
> 
> I use rsync to syncronize two systems.
> There are 2 million + files.
> 
> My problem is
> 1. rsync is very slow (it needs more than 48 hours)
> 2. rsync needs too much RAM (ca. 750 MByte on each server)

You must have long path names but that sounds about right.

> 
> The backup software needs 14 hours including file transfer to the backup
> server for full backup.
> 
> My configuration:
> 1. Server Dual-XEON with 2 GB RAM.
> 2. Server Dual-PIII with 2 GB RAM.
> 
> The partition has 160 GB in use with approx.2 million files and a very
> large number of directories.
> 
> I saw in the TO-DO file that this is problem has been recognised.
> Is there a solution?
> Is the solution for this issue already scheduled for the next version?

The solution is a new program that is not compatible with
the current rsync.  We hope to have that done some time in
the next decade.

> Or what can I do to get it done in the next version?

Fund or write the next generation of rsync.

In the meantime, break the job up.  There are some scripts
floating around and someone working on one posted here
recently to find out how to limit the depth of an rsync.


-- 
________________________________________________________________
	J.W. Schultz            Pegasystems Technologies
	email address:		jw at pegasys.ws

		Remember Cernan and Schmitt