hundreds of thousands files

[jw schultz]

On Wed, Oct 30, 2002 at 01:40:10PM -0700, Ryan wrote:
> I am rsyncing several hunderds of thousands of files in several
> directories.  The way I got rsync to work for me was I wrote a script
> which NFS mounts the directory I am rsyncing first before starting, and
> then it goes through the directories and rsyncs them in bite sizeable
> chunks by going a few directories deep and starting there..
> 
> Is there any way to have this as an option inside rsync directly?  It
> would only have to be a change in the client.  Basicly when it realizes
> there are thousands of files, instead of trying to do it in one big sweep
> and bombing out, make it so it can do a directory at a time.
> 
> Like first rsync all the directories OMITTING individual files, and then
> go through the directory tree and go all the way deep and rsync them
> directory by directory instead of the whole tree at once.
> 
> Any one have any ideas on this?????

Directory-at-a-time traversal and other ideas have been
discussed serveral times.  Probably the best recent
discussions have had subject lines refering to
"superlifter".

-- 
________________________________________________________________
	J.W. Schultz            Pegasystems Technologies
	email address:		jw at pegasys.ws

		Remember Cernan and Schmitt