very big rsync only worked partially what are size limitations?

[Wayne Davison]

On Sat, Dec 18, 2004 at 09:50:53PM -0500, mlaks at verizon.net wrote:
> I then tried again and started the rsync script on the directory
> again, and it ended without copying over any more stuff, as far as I
> can see  - size wise and number of subdirectories.  :(.

Did it output an error? Or end normally? If it did not output an error,
it believes that the files you asked it to transfer are up-to-date.

> So my question, what are the size limitations on this sort of stupid
> original way of doing it?

The maximum file count is mainly limited by memory.  Figure that it
probably takes 100 bytes or so per file/directory in the transfer list.
If you're not using a modern rsync (2.6.3 is the latest) this memory
requirement will be much higher, particularly on the receiving system:
the receiving side forks off an extra rsync process, which should share
a lot of the same memory as the parent process (if your fork uses copy-
on-write memory-sharing).  Depending on your OS and the age of your
rsync (modern ones do a better job of keeping the memory shared between
the processes), the receiving side's memory use could bloat to be double
what the sending side needs.  Your best bet is to just run the command
and look at memory use, keeping an eye on the total proces size and (on
the receiving side) how much memory is shared.

..wayne..