Why does one of there work and the other doesn't

[tim.conway at philips.com]

rsync already has a memory-hogging issue.  Imagine having it search your 
entire directory tree, checksumming all files, storing and sending them 
all, comparing both lists looking for matching date/time/checksums to 
guess where you've moved files to.  You'd be better off to use a wrapper 
the tools you move files with, keeping a replayable log, and have your 
mirrors retrieve and replay that log, before doing the rsync.

Tim Conway
tim.conway at philips.com
303.682.4917
Philips Semiconductor - Longmont TC
1880 Industrial Circle, Suite D
Longmont, CO 80501
Available via SameTime Connect within Philips, n9hmg on AIM
perl -e 'print pack(nnnnnnnnnnnn, 
19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), 
".\n" '
"There are some who call me.... Tim?"




Phil Howard <phil-rsync at ipal.net>
Sent by: rsync-admin at lists.samba.org
12/03/2001 09:04 AM

 
        To:     rsync at samba.org
        cc:     (bcc: Tim Conway/LMT/SC/PHILIPS)
        Subject:        Re: Why does one of there work and the other doesn't
        Classification: 



On Mon, Dec 03, 2001 at 12:09:16AM +1100, Martin Pool wrote:

| On 30 Nov 2001, Randy Kramer <rhkramer at fast.net> wrote:
| 
| > I am not sure which end the 100 bytes per file applies to, and I guess
| > that is the RAM memory footprint?.  Does rsync need 100 bytes for each
| > file that might be transferred during a session (all files in the
| > specified directory(ies)), or does it need only 100 bytes as it does 
one
| > file at a time?
| 
| At the moment that is 100B for all files to be transferred in the
| whole session.  This is a big limit to scalability at the moment, and
| a goal of mine is to reduce it to at most holding file information
| from a single directory in memory.

It would still be nice to have an option to gather all files at once,
but this will be of value if it also gathers all the checksums and
syncronizes files moves that have happened on the source end by
doing the syncronization of the moved file to the new location using
the old (checksum matched) file on the destination end.  Right now
if a file gets moved from one location to another (especially in a
different directory, which is often the case with a re-organization)
things get retransferred even though most every file already exists
somewhere on the destination.

-- 
-----------------------------------------------------------------
| Phil Howard - KA9WGN |   Dallas   | http://linuxhomepage.com/ |
| phil-nospam at ipal.net | Texas, USA | http://phil.ipal.org/     |
-----------------------------------------------------------------