hundreds of thousands files
jw at pegasys.ws
Wed Oct 30 22:02:00 EST 2002
On Wed, Oct 30, 2002 at 01:40:10PM -0700, Ryan wrote:
> I am rsyncing several hunderds of thousands of files in several
> directories. The way I got rsync to work for me was I wrote a script
> which NFS mounts the directory I am rsyncing first before starting, and
> then it goes through the directories and rsyncs them in bite sizeable
> chunks by going a few directories deep and starting there..
> Is there any way to have this as an option inside rsync directly? It
> would only have to be a change in the client. Basicly when it realizes
> there are thousands of files, instead of trying to do it in one big sweep
> and bombing out, make it so it can do a directory at a time.
> Like first rsync all the directories OMITTING individual files, and then
> go through the directory tree and go all the way deep and rsync them
> directory by directory instead of the whole tree at once.
> Any one have any ideas on this?????
Directory-at-a-time traversal and other ideas have been
discussed serveral times. Probably the best recent
discussions have had subject lines refering to
J.W. Schultz Pegasystems Technologies
email address: jw at pegasys.ws
Remember Cernan and Schmitt
More information about the rsync