Can Rsync handle largs exclude lists without slowdown?

Matt McCutchen hashproduct+rsync at gmail.com
Sun Jul 29 14:55:57 GMT 2007


On 7/29/07, Matthias Schniedermeyer <ms at citd.de> wrote:
> Let's say i wanted to exclude 100.000 files by naming them one by one in
> a file to be used by --exclude-from.
>
> Can rsync cope with that without bigger problems?

Rsync would work correctly, but file-list building would probably be
slow because rsync would check each file against all 100,000 exclude
patterns.  Instead, you might consider (1) using per-directory filter
files or (2) preparing a list of files you do want backed up and
feeding it to rsync with --files-from.  To prepare the list for #2,
you could use "find" to list all the files on the system and "comm" to
remove excluded files from that list.

Matt


More information about the rsync mailing list