TODO hardlink performance optimizations

[jw schultz]

Minor corrections...

On Sun, Jan 04, 2004 at 06:35:03AM -0600, John Van Essen wrote:
> Lester,
> 
> You articulated your situation clear enough for me.  Thanks.
> 
> I'll address your issue about when rsync is running locally for /vol/N
> to /vol/N_mirror syncing, it exhausts all of the RAM and swap.
> 
> If you haven't read jw schultz's "How Rsync Works" page, here the link:
> 
>   http://www.pegasys.ws/how-rsync-works.html
> 
> The sender, receiver, and generator each have a full copy of the file
> list (each file's entry uses 100 bytes on average).

The reciever and generator start out shareing the file list
but dirverge as the receiver modifies directory entries
forcing copy-on-write.  This COW doesn't just copy the
file_struct but also the strings because they are
interspersed.  Wayne has a patch in CVS to prevent
the COW.

> Additonally, the --hard-links option creates yet *another* full copy of
> the file list in the receiver, so that's even more memory consumed.

That is a partial copy.  Only the actual file_struct is
copied at 72bytes (32bit linux) per file, the strings are
not copied.  My patch (and John's i assume) reduces that to
just copying the file_list index which is 4bytes (32bit) per
file.


-- 
________________________________________________________________
	J.W. Schultz            Pegasystems Technologies
	email address:		jw at pegasys.ws

		Remember Cernan and Schmitt