rsync memory usage, paid feature request
Matthew S. Hallacy
poptix at poptix.net
Wed Jul 6 06:13:23 GMT 2005
Hello,
The company I work for uses rsync for backups from client computers,
the memory usage is a problem for a lot of them since they're already
busy doing other important things (databases, web serving, etc).
>From the FAQ:
---
out of memory
The usual reason for "out of memory" when running rsync is that you
are transferring a _very_ large number of files. The size of the
files doesn't matter, only the total number of files.
As a rule of thumb you should expect rsync to consume about 100 bytes per
file in the file list. This happens because rsync builds a internal
file list structure containing all the vital details of each file.
rsync needs to hold structure in memory because it is being constantly
traversed.
I do have a plan for how to rewrite rsync so that it consumes a fixed
(small) amount of memory no matter how many files are transferred, but
I haven't yet found a spare week of coding time to implement it!
---
Unfortunately there's no indication of who needs a spare week of coding
time, or how much a week would cost.
Since it's important (to us, probably others) to have a more memory
friendly rsync, and we're thankful for the work already done, could the
person responsible for that comment please respond (publicly or privately)
with a dollar figure to make the necessary changes.
This would be for public consumption, we believe in supporting open source
software.
A CC:'d reply would help (I am not subscribed to the list), but I will
also watch the archives.
Thanks.
--
Matthew S. Hallacy
More information about the rsync
mailing list