[Bug 12769] error allocating core memory buffers (code 22) depending on source file system
samba-bugs at samba.org
samba-bugs at samba.org
Mon Oct 21 20:47:36 UTC 2019
https://bugzilla.samba.org/show_bug.cgi?id=12769
--- Comment #6 from Dave Gordon <dg32768 at zoho.eu> ---
The hash table doubles each time it reaches 75% full. A hash table for 32m
items @ 16 bytes each (8 byte key, 8 byte void *data) needs 512MB of memory. At
the next doubling (up to 64m items) it hits the array allocator limit in
utils2.c:
#define MALLOC_MAX 0x40000000
void *_new_array(unsigned long num, unsigned int size, int use_calloc)
{
if (num >= MALLOC_MAX/size)
return NULL;
return use_calloc ? calloc(num, size) : malloc(num * size);
}
void *_realloc_array(void *ptr, unsigned int size, size_t num)
{
if (num >= MALLOC_MAX/size)
return NULL;
if (!ptr)
return malloc(size * num);
return realloc(ptr, size * num);
}
No single array allocation or reallocation is allowed to exceed MALLOC_MAX
(1GB). Hence rsync can only handle up to 32m items per invocation if a hash
table is required (e.g. for tracking hardlinks when -H is specified).
HTH,
Dave
--
You are receiving this mail because:
You are the QA Contact for the bug.
More information about the rsync
mailing list