Rsyncing a very large directory tree (over 50,000 files)

[Andrew Hodgson]

-----Original Message-----
From: Jamie Lokier [mailto:jamie at shareable.org] 
Sent: 14 June 2006 19:43
To: Matt McCutchen
Cc: Andrew Hodgson; rsync at lists.samba.org
Subject: Re: Rsyncing a very large directory tree (over 50,000 files)

>Matt McCutchen wrote:
>> Yes, rsync will send the complete file list each time it runs.  It
seems
>> odd to me that building the file list would take 15 minutes; when I
back
>> up the system partition of my computer (300,000 files) rsync takes
>> perhaps 5 minutes to build the file list.

>That surely depends on the computer and your disk.

>My laptop disk has about 2,000,000 files, and it takes longer than 15
>minutes to build the list when doing a backup with rsync.  Also I have
>to do two rsync runs, as it can run out of memory if I do the whole
>tree with one.  (192MB RAM).

Hardware shouldn't be an issue, but I was a bit worried about it
reaching some limit somewhere and not doing something correctly.

I will try again when at work tomorrow.

Andrew.