rsync out of memory in flist_expand

[Christophe Lyon]

Hi,

I have been using rsync for years to perform my backups, but since a
few days it crashes with an "out of memory" error.
I am running Ubuntu 11.04/x86_64, which ships rsync-3.0.8, and I have
also tried with the git version.

I can see the following messages:

[generator] expand file_list pointer array to 524288 bytes, did move
[generator] expand file_list pointer array to 1048576 bytes, did move
[generator] expand file_list pointer array to 2097152 bytes, did move
[generator] expand file_list pointer array to 4194304 bytes, did move
[generator] expand file_list pointer array to 8388608 bytes, did move
[generator] expand file_list pointer array to 16777216 bytes, did move
[generator] expand file_list pointer array to 33554432 bytes, did move
[generator] expand file_list pointer array to 67108864 bytes, did move
[generator] expand file_list pointer array to 134217728 bytes, did move
[generator] expand file_list pointer array to 268435456 bytes, did move
[generator] expand file_list pointer array to 402653184 bytes, did not move
[generator] expand file_list pointer array to 536870912 bytes, did move
[generator] expand file_list pointer array to 671088640 bytes, did not move
[generator] expand file_list pointer array to 805306368 bytes, did not move
[generator] expand file_list pointer array to 939524096 bytes, did not move
[generator] expand file_list pointer array to 1073741824 bytes, did move
ERROR: out of memory in flist_expand [generator]
usr/bin/
rsync: [sender] write error: Broken pipe (32)
rsync error: error in socket IO (code 10) at io.c(823) [sender=3.1.0dev]

Using the official 3.0.8 (ie not Ubuntu's), I get similar buffer
sizes, and a slightly  more informative error message:
error allocating core memory buffers (code 22) at ../util.c(117)
[generator=3.0.8]

When using rsync-3.0.8 from Ubuntu, it seemed to be looping with
make_file("usr/bin/head", ...).

It tried to use valgrind, but it reported no error.

My /usr/bin directory contains 2653 files, which is a bit large, but
not terrible, is it?
As I said, the very same command used to work, so there are very few
files to actually update since my last successful backup.

The machine has 6GB of RAM, 1GB of swap, and the process is executed by root.

Any hints to help me understand the actual problem and possibly fix
the bug are highly appreciated!

Thanks

Christophe.