Rsyncing a very large directory tree (over 50,000 files)
andrew at hodgsonfamily.org
Wed Jun 14 22:35:39 GMT 2006
From: Jamie Lokier [mailto:jamie at shareable.org]
Sent: 14 June 2006 19:43
To: Matt McCutchen
Cc: Andrew Hodgson; rsync at lists.samba.org
Subject: Re: Rsyncing a very large directory tree (over 50,000 files)
>Matt McCutchen wrote:
>> Yes, rsync will send the complete file list each time it runs. It
>> odd to me that building the file list would take 15 minutes; when I
>> up the system partition of my computer (300,000 files) rsync takes
>> perhaps 5 minutes to build the file list.
>That surely depends on the computer and your disk.
>My laptop disk has about 2,000,000 files, and it takes longer than 15
>minutes to build the list when doing a backup with rsync. Also I have
>to do two rsync runs, as it can run out of memory if I do the whole
>tree with one. (192MB RAM).
Hardware shouldn't be an issue, but I was a bit worried about it
reaching some limit somewhere and not doing something correctly.
I will try again when at work tomorrow.
More information about the rsync