A checksum question

[jw schultz]

On Wed, Mar 26, 2003 at 11:00:38AM +0000, Terry Raggett wrote:
> I'm sorry if you mis-understood me on this issue. I'm not trying to 
> denigrate Rsync, as we are convinced that it is a highly capable tool 
> that we can adapt for our needs. It is just that in certain situations 
> the checksum processing absorbs a lot of CPU time. I have a user that 
> has been using the 2.4.6 version for some considerable time and is very 
> happy with it. However, when I moved him to the 2.5.5 version he found 
> that his transfers took considerably longer. This appears to be due to 
> the introduction of the MD4 checksum, I believe was introduced at 
> release 2.5.
> 
> We have a large and disparate computer centre, with virtually every 
> flavour of unix, where we find that Rcp is less than reliable for moving 
> data between them. This unreliability has prompted us to look at other 
> solutions, with Rsync at the top of the list, as it implements many of 
> the features that we have identified for our needs, checksumming (for 
> data integrity) being one of them.
> 
> Regards,
> 
> Terry
> -----

The whole-file MD4 sums have been in rsync since protocol
version 14 in rsync 1.6.4 back in 1997.  The rolling MD4
sums are integral to rsync and always been there.  I'd
suggest that something else is responsible for the increased
CPU consuption.

> 
> 
> jw schultz wrote:
> >On Tue, Mar 25, 2003 at 04:31:00PM +0000, Terry Raggett wrote:
> >
> >>I'd like to know a little about the internals of RSYNC. I am a little 
> >>confused as to why RSYNC is using both the simple 32 bit algorithm and 
> >>the MD4 checksum function on the same files. From my testing this causes 
> >>a vast overhead that is clearly not represented by RCP (fairly 
> >>obvious!). Removing checksumming from a secure whole-file LAN transfer 
> >>brings RSYNC in line with the general performance of RCP.
> >>
> >>Can someone explain to me the rationale behind the checksum algoritm and 
> >>use within the RSYNC protocol?
> >>
> >>I'm asking this question as I am finding it rather difficult to convince 
> >>some of our users that RSYNC is a viable replacement for RCP 
> >>functionality, which is necessary to resolve some of the limitations we 
> >>encounter with the standard RCP.
> >>
> >>Many thanks in anticipation,
> >>
> >>Terry Raggett
> >
> >
> >Rsync is not a replacement for rcp.  Rcp is a remote-copy
> >utility.  Rsync is a remote-update utility.  They each exist
> >for completely different purposes.
> >
> >Without knowing what limitations you are encountering with
> >rcp or to the specific purpose of use i couldn't speak to
> >the suitability of rsync.  There are many times that rsync
> >is not the best or most efficient tool for the job.  If all
> >you want is a fast light-weight copy utility, rsync isn't
> >it.  There are even times when rsync is ill suited to an
> >update operation.  The tool-box was invented for a reason.
> >
> 
> 
> -- 
> To unsubscribe or change options: 
> http://lists.samba.org/mailman/listinfo/rsync
> Before posting, read: http://www.tuxedo.org/~esr/faqs/smart-questions.html

-- 
________________________________________________________________
	J.W. Schultz            Pegasystems Technologies
	email address:		jw at pegasys.ws

		Remember Cernan and Schmitt