rsync memory usage, paid feature request

Andrew Burgess aab at cichlid.com
Fri Jul 8 16:56:14 GMT 2005


> The company I work for uses rsync for backups from client computers,
> the memory usage is a problem for a lot of them since they're already
> busy doing other important things (databases, web serving, etc).

>> From the FAQ:
> out of memory
> The usual reason for "out of memory" when running rsync is that you
> are transferring a _very_ large number of files.  The size of the
> files doesn't matter, only the total number of files.

One possible scheme would be to store the file info compressed, this
might be a smaller impact than rewriting the protocol. Typically gzip
gets 3x compression on ascii, no reason why this couldn't be accomplished.
It would slow things down so would need an option to enable. This would
work best with streaming compression rather than gzips block-oriented style
else every entry would consume the minimum block size.

It _is_ a hack I admit...


More information about the rsync mailing list