Syncing large amounts of data

Steve Bonds knnf6cy7w001 at
Thu Feb 13 20:27:32 EST 2003

A good solution for the "lots of little files" problem is syncing below
the filesystem layer (usually either at the logical volume or disk
level.)  This allows you to avoid the stat() system call on loads of files
while looking for the ones to send.

As was already suggested, if you can avoid the search by having your
application keep track of what has changed and what has not, then you can
save yourself a lot of system impact.

  -- Steve

On Wed, 12 Feb 2003, Adam Herbert |Rsync List| wrote:

> I need some suggestions. Here's my setup:
> 	800GB of Data
> 	14,000,000+ Files
>       No changes just additions
>       Files range in size from 30k - 190k

More information about the rsync mailing list