Rsync completed successfully, but files are not identical

[Mark, Oren]

It does not Microsoft, but Linux Sles9 to Linux Sles9 replication. 

Thanks,
Oren Mark

-----Original Message-----
From: Tony Abernethy [mailto:tony at servacorp.com] 
Sent: Sunday, August 16, 2009 12:45 PM
To: 'Matthias Schniedermeyer'; Mark, Oren
Cc: rsync at lists.samba.org
Subject: RE: Rsync completed successfully, but files are not identical

Mark, Oren wrote:
> On 16.08.2009 11:38, Mark, Oren wrote:
> > Hi All,
> > 
> > I came into a strange issue running rsync on directory with 
> ~500,000 files.
> > Some of the file, although with same time stamps and size 
> on source and destination, were different on the destination.
> > The destination is just a mirrored area, and the data 
> written to it, is just the one that comes through rsync.
> > Needless to mentioned, that when I remove files and synced 
> them again it works, or when I did the sync with checksum.
> > 
> > Due to the large number of files, running it with checksum 
> is very bad options for me.
> > 
> > I have few questions:
> > 
> > 1)      Any idea how come a replicated area, has files with 
> same time stamp and size, but file is different than the source?
> > 2)      Is there a way for rsync to verify, that each 
> transferred size is identical as the source, after the file 
> was transferred?
> 
> My first question would be if i have a program on the source 
> side that 
> "tampers" with files and then resets the atime/mtime, as long as the 
> size stays the same it's the same file for rsync.
> 
> And i faintly remember reading about an issue with mtime and mmap 
> writing files. I don't remember the details, but i guess if 
> there was an 
> issue it is remedied in recent kernels.
> 
> 
> IOW i'm quite sure the culprit isn't rsync.
> 
He doesn't say, buy most likely the files came from Microsoft Windows.

The pattern of directory entry being identical while file contents are 
changing seems to be the norm with Microsoft Windows.
The order is to make the directory entry and then fill in the contents.
This is easily seen by looking at the target directory during a copy.
This is an issue when rsyncing the unix/linux data of samba stores.
Enough of an issue that I initiate an overnight rsync at 3:33 AM
instead of running hourly or so backups.

There is a very slight gain by preallocating space to copy into,
but this does wonders for system integrity if plug is pulled during.
This is part of the modern pattern of log what I intend to do, 
then (maybe) get around to actually doing it.
Does wonders when stuff is asynchronous and some of it is outside control.

---------------------------------------------------------------------
Intel Israel (74) Limited

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.