[Bug 12769] error allocating core memory buffers (code 22) depending on source file system

[samba-bugs at samba.org]

https://bugzilla.samba.org/show_bug.cgi?id=12769

--- Comment #4 from Ovidiu Stanila <ovidiu.stanila at jentla.com> ---
We hit the same issue on a CentOS 6 server (kernel 2.6.32-754.18.2.el6.x86_64),
the sync would break with the following error:

# /usr/bin/rsync --debug=HASH --stats --no-inc-recursive -aHn --delete /app/
<remote>:/app/
[sender] created hashtable 2013770 (size: 16, keys: 64-bit)
[sender] created hashtable 2015370 (size: 512, keys: 64-bit)
[sender] growing hashtable 2015370 (size: 1024, keys: 64-bit)
[sender] growing hashtable 2015370 (size: 2048, keys: 64-bit)
[sender] growing hashtable 2015370 (size: 4096, keys: 64-bit)
[sender] growing hashtable 2015370 (size: 8192, keys: 64-bit)
[sender] growing hashtable 2015370 (size: 16384, keys: 64-bit)
[sender] growing hashtable 2015370 (size: 32768, keys: 64-bit)
[sender] growing hashtable 2015370 (size: 65536, keys: 64-bit)
[sender] growing hashtable 2015370 (size: 131072, keys: 64-bit)
[sender] growing hashtable 2015370 (size: 262144, keys: 64-bit)
[sender] growing hashtable 2015370 (size: 524288, keys: 64-bit)
[sender] growing hashtable 2015370 (size: 1048576, keys: 64-bit)
[sender] growing hashtable 2015370 (size: 2097152, keys: 64-bit)
[sender] growing hashtable 2015370 (size: 4194304, keys: 64-bit)
[sender] growing hashtable 2015370 (size: 8388608, keys: 64-bit)
[sender] growing hashtable 2015370 (size: 16777216, keys: 64-bit)
[sender] growing hashtable 2015370 (size: 33554432, keys: 64-bit)
ERROR: out of memory in hashtable_node [sender]
rsync error: error allocating core memory buffers (code 22) at util2.c(106)
[sender=3.1.2]

Both the sender and receiver part have the same rsync and OS versions.

The source we try to transfer is over 3.4Tb (65 million files) with a big
number of hard links between various directories(for avoiding duplicate files).

We initially increased the memory from 16Gb to 32Gb but even with that rsync
would die with the same error. On above example, there were over 22Gb free RAM
at the point rsync stopped working.

We also tried "--inplace" instead of "--no-inc-recursive" with the same result.

Did we hit some kind of limitation inside of rsync ? Is there anything else we
should check ?

-- 
You are receiving this mail because:
You are the QA Contact for the bug.