rsync through a server storing the changes, time delayed rsync

[Matt McCutchen]

On Sat, 2006-02-25 at 15:48 +0100, Torbjörn Nordling wrote:
> True, it is a one-way-and-then-the-other-way system, except when I 
> forget to do the sync, so for safety it would be better to have two-way.

Keep in mind that two-way systems are several times as complex as
one-way systems and thus several times as likely to do something you
don't expect.  Unison is one of the few tools that has taken on the full
task of two-way synchronization, but it requires a direct connection
between the two ends.  Maybe a one-way-and-then-the-other-way system
that warns you if you have forgotten to do the sync would be more
practical.

> The last sounds doable, but then it would again depend on mtimes and 
> other programs messing it up, wouldn't it?

True, but only to the same extent that rsync itself depends on mtimes.

>  Taking inspiration from the 
> two copies suggestion, which unfortunately is not doable (too much data 
> ~40GB): 1. Store a file list with check sums locally, which is updated 
> last in a sync.
> 2. Sync, by first comparing with local lastsync file list in order to 
> detect changes, which are to be uploaded to the server. 3. Compare 
> these changes with the changes stored on the server (file list 
> including only check sums of changed files) in order to detect possible 
> conflicts caused by same file being changed on both. 4. Download 
> changes (rsync changed parts of file) from server and merge them with 
> existing files. (I suppose that if a conflict occur then the only 
> practical possibility is to select the local version, since only 
> changed parts of file exist on server.)
> 5. Upload final changes that have been made to local files (including 
> conflicts solved manually) and the file list including only check sums 
> of changed files. 6. Update local file list of check sums.
> Would this do the trick? Maintaining the strength of rsync, sending 
> only changed parts of files, while localy only having to store check 
> sums from last sync instead of two copies of the files.

I started coding a full two-way synchronizer along those lines for you,
but I got discouraged when I realized how nasty conflict resolution was
going to be.  (Typical nasty case: add a file to a directory on one
side, delete the whole directory on the other.)  My script used
size/mtime pairs like rsync's default quick check instead of checksums
and used line-by-line diffing and patching on find-like file listings.
I attach it in case you can make some use of it.  Most of the framework
is there, but I was only beginning to code the process of translating
file listing differences to rsync files-from lists and filter lists.  It
was to be called sisync, for small-intermediary sync.
-- 
Matt McCutchen
hashproduct at verizon.net
http://hashproduct.metaesthetics.net/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: sisync
Type: application/x-shellscript
Size: 3269 bytes
Desc: not available
Url : http://lists.samba.org/archive/rsync/attachments/20060225/67707a21/sisync.bin