BIG delete list makes for BIGGER RAM requirements
cbyrum at spamaps.org
Sun Feb 6 18:50:53 GMT 2005
John Van Essen wrote:
>On Mon, 31 Jan 2005, Clint Byrum <cbyrum at spamaps.org> wrote:
>>So I'm doing daily backups with rsync, and weekly, I run it with
>>--delete after archiving the whole thing (this way I don't lose any
>>deleted files). All week long this runs fine, but when I add --delete,
>>rsync runs for a few hours then aborts because the box runs out of
>When a --delete option is given, the receiver not only has in memory
>the list of files from the sender, but it also builds its own list of
>local files so it can determine what to delete. This means that the
>memory requirements are roughly doubled when you do deletions.
>However - good news!
>Wayne has recently implemented a --delete-during option that deletes
>files in directories as they are processed. This means there's no more
>monolithic filelist for the local files on the target.
>Wayne - it appears that the monolithic target file list is no longer
>built for --delete or --delete-after as well. Is that true? If so,
>that should be mentioned in the NEWS, as it will be an improvement
>over the current procedure.
>Clint - try the current version in CVS (or use a nightly snapshot).
>It might solve your out-of-memory problem. Use with --dry-run, first,
>to see how things go.
Looks good! --delete-during worked flawlessly with dry-run. Its still
running "live" right now, but seems to be working perfectly. Memory
consumed was just 82M. :-D
When is 2.6.4 going to be released?
More information about the rsync