Nice little performance improvement

Darryl Dixon - Winterhouse Consulting darryl.dixon at
Thu Oct 15 21:13:56 MDT 2009

> Hi,
> In my situation I'm using rsync to backup a server with (currently) about
> 570,000 files.
> These are all little files and maybe .1% of them change or new ones are
> added in
> any 15 minute period.

Hi Mike,

We have three filesystems that between them have approx 22 million files,
and around 10-20,000 new or changed files every business day.

In order to expeditiously move these new files offsite, we use a modified
version of pyinotify to log all added/altered files across the entire
filesystem(s) and then every five minutes feed the list to rsync with the
--files-from option. This works very effectively and quickly.

Darryl Dixon
Winterhouse Consulting Ltd

More information about the rsync mailing list