Rsyncing a very large directory tree (over 50,000 files)

Andrew Hodgson andrew at
Wed Jun 14 18:07:39 GMT 2006



I need to rsync a very large directory tree (over 50,000 files).  This
is not a regular job, after the initial rsync is done, I can do a
nightly rsync and only a few files will change.


Is there anything I need to be aware of before doing this?  I started
the script this morning, but it was still building the file list after
around 15 minutes.  Is it better to do it using several points, then
when I have the structure on the other machine I can then do the whole
tree in one go?  Will the complete file list need to be sent across each
time I run the program?


Also, I wish to exclude some directories, as they contain DBF files
which are bing manipulated on a regular basis (!).  However, I wish to
exclude some pathnames but not others under subdirectories.


For example,

- /bin/

+ /binbackup/bin/

- /binbackup/**


Is that the correct way of doing this?  I want to exclude the bin
directory and anything in the binbackup directory apart from the bin
directory within.


Any suggestions?



-------------- next part --------------
HTML attachment scrubbed and removed

More information about the rsync mailing list