rsync of symbolic links bug
jw at pegasys.ws
Fri May 9 11:00:05 EST 2003
On Thu, May 08, 2003 at 09:44:10AM -0600, Tom Schmidt wrote:
> I am seeing a problem with the way the rsync of symbolic links
> is done. Here is a simple example to duplicate this issue:
> ACK! The symbolically linked file pointing to foo/file4 got
> updated first, causing anyone using it to see it pointing at a
> non-existant file!
> This bug is essentially a race condition that can cause
> the mirrored system to see the symbolic link get updated to
> point to a non-existant file since the file it is pointing
> at has not been mirrored yet.
Not a bug, it is a race condition. rsync is not atomic.
> We see this problem quite often when we are mirroring our
> /usr/local filesystem. For example, if we have /usr/local/lib/libz.so
> symbolically linked to /usr/local/zlib/1.1.3/libz.so, but then
> update it to point to /usr/local/zlib/1.1.4/libz.so, then
> when the filesystem is mirrored, the /usr/local/lib/libz.so
> link gets updated long before the /usr/local/zlib/1.1.4 directory
> tree gets pushed. This causes any application that is started
> that using libz.so to fail while the rsync is running! This
> downtime can be quite long when pushing many updates across our
> WAN to servers on the other side of the world!
> It appears that the rsync pushes the updates out alphabetically,
> which is why the symbolic link to bar/file3 above worked OK, but
> the symbolic link to foo/file4 fails with the race condition.
> The fix for this would be to have rsync internally sort its filelist
> to push the updated files and directories first, then the updated
> symbolic links last. How can this be done? Or am I missing
> some rsync commandline issue to fix this problem?
> My only workaround at this point is to rsync twice, the first time
> without pushing any symbolic links, but this workaround can
> still burn us if an update is made to the master copy between
> the two rsync runs.
If you need atomic updates use a staging area. Have two
directories. rsync to the unused one and then swap them.
--copy-links might also be of use.
J.W. Schultz Pegasystems Technologies
email address: jw at pegasys.ws
Remember Cernan and Schmitt
More information about the rsync