Local copying with minimal disk activity.

[jw schultz]

On Mon, Mar 03, 2003 at 10:56:15AM -0500, Tom Limoncelli wrote:
> How do I minimize the amount of disk activity?
> 
> I'm copying a large number of files:
> 
> 	rsync -av -W /from/here/. /to/here/.
> 
> I'm using rsync because I need the incremental updates and other features. 
> Otherwise, I'd use tar piped to tar or some similar technique.
> 
> Note: Both the source and destination are "local".  Well, not really.  The
> source is a network-mounted partition (SAMBA), and the destination is a
> real disk.

I rsync thinks it is running local->local it will use
whole-file anyway unless you specify --no-whole-file.

> I'd like to minimize the amount of network activity since that seems to be
> my bottleneck.  For example, without the "-W" option, every file is read
> twice (once to checksum, once to copy).

The double-read only applies to the destination, not the source.

> Are there other options that I could enable to speed up the copy?

Install rsync on the source and eliminate smb.

> Is using -W dangerous?  If I'm doing an incremental update (a file or two
> changed on the source, so I need to recopy the data) will -W catch as much
> as checksuming if the file timestamps change?

-W is not dangerous.  Since rsync does a checksum as part of
the transfer it is less dangerous than other means.

With --whole-file the sender disk load is about as small as
it could be.  (Speaking only for the sender) We walk the tree
and stat(2) each file, then we read each file that the
stat(2) info indicates as changed.  This compares favorably
with tar and other methods which must do at least that much.


-- 
________________________________________________________________
	J.W. Schultz            Pegasystems Technologies
	email address:		jw at pegasys.ws

		Remember Cernan and Schmitt