rsync of symbolic links bug

[jw schultz]

On Thu, May 08, 2003 at 09:44:10AM -0600, Tom Schmidt wrote:
> I am seeing a problem with the way the rsync of symbolic links
> is done.  Here is a simple example to duplicate this issue:
> 
[snip]
> 
> ACK!  The symbolically linked file pointing to foo/file4 got
> updated first, causing anyone using it to see it pointing at a
> non-existant file!
> 
> This bug is essentially a race condition that can cause
> the mirrored system to see the symbolic link get updated to
> point to a non-existant file since the file it is pointing
> at has not been mirrored yet.

Not a bug, it is a race condition.  rsync is not atomic.

> We see this problem quite often when we are mirroring our
> /usr/local filesystem.  For example, if we have /usr/local/lib/libz.so
> symbolically linked to /usr/local/zlib/1.1.3/libz.so, but then
> update it to point to /usr/local/zlib/1.1.4/libz.so, then
> when the filesystem is mirrored, the /usr/local/lib/libz.so
> link gets updated long before the /usr/local/zlib/1.1.4 directory
> tree gets pushed.  This causes any application that is started
> that using libz.so to fail while the rsync is running!  This
> downtime can be quite long when pushing many updates across our
> WAN to servers on the other side of the world!
> 
> It appears that the rsync pushes the updates out alphabetically,
> which is why the symbolic link to bar/file3 above worked OK, but
> the symbolic link to foo/file4 fails with the race condition.
> 
> The fix for this would be to have rsync internally sort its filelist
> to push the updated files and directories first, then the updated
> symbolic links last.  How can this be done?  Or am I missing
> some rsync commandline issue to fix this problem?
> 
> My only workaround at this point is to rsync twice, the first time
> without pushing any symbolic links, but this workaround can
> still burn us if an update is made to the master copy between
> the two rsync runs.

If you need atomic updates use a staging area.  Have two
directories.  rsync to the unused one and then swap them.

--copy-links might also be of use.


-- 
________________________________________________________________
	J.W. Schultz            Pegasystems Technologies
	email address:		jw at pegasys.ws

		Remember Cernan and Schmitt