BIG delete list makes for BIGGER RAM requirements

[John Van Essen]

On Mon, 31 Jan 2005, Clint Byrum <cbyrum at spamaps.org> wrote:
>
> So I'm doing daily backups with rsync, and weekly, I run it with
> --delete after archiving the whole thing (this way I don't lose any
> deleted files). All week long this runs fine, but when I add --delete,
> rsync runs for a few hours then aborts because the box runs out of
> memory.

When a --delete option is given, the receiver not only has in memory
the list of files from the sender, but it also builds its own list of
local files so it can determine what to delete.  This means that the
memory requirements are roughly doubled when you do deletions.

However - good news!

Wayne has recently implemented a --delete-during option that deletes
files in directories as they are processed.  This means there's no more
monolithic filelist for the local files on the target.

Wayne - it appears that the monolithic target file list is no longer
built for --delete or --delete-after as well.  Is that true?  If so,
that should be mentioned in the NEWS, as it will be an improvement
over the current procedure.

Clint - try the current version in CVS (or use a nightly snapshot).
It might solve your out-of-memory problem.  Use with --dry-run, first,
to see how things go.
-- 
        John Van Essen  Univ of MN Alumnus  <vanes002 at umn.edu>