rsync buffer overflow detected

[Boris Savelev]

ulimit:
time(seconds)        unlimited
file(blocks)         unlimited
data(kbytes)         unlimited
stack(kbytes)        8192
coredump(blocks)     0
memory(kbytes)       unlimited
locked memory(kbytes) 64
process              64098
nofiles              65536
vmemory(kbytes)      unlimited
locks                unlimited


Rsync call at the end, so i = 1600

modified test script:

---8<---
import os
import subprocess
#import psutil

F = 'test'
OPENS = 1600

cmd = [
    #'gdb', '--args',
    #'strace', '-f', '-olog',
    './rsync',
    '-aviH',
    '/etc/passwd',
    '/tmp/passwd'
]

for i in xrange(OPENS):
    fd = os.open(F, os.O_WRONLY | os.O_CREAT)

os.system('ulimit -a')
#proc = psutil.Process()
#print len(proc.open_files())
subprocess.check_call(cmd)
---8<---

strace:
https://pastebin.com/Xu9Pq4R9

If system limit excided it may fall with EMFILE, I think

2017-04-16 9:43 GMT+03:00  <devzero at web.de>:
> What's the value of "i" when this happens and what are the system ulimit values for the user running that?
>
> Roland
>
>
>
>> Gesendet: Freitag, 14. April 2017 um 19:22 Uhr
>> Von: "Boris Savelev via rsync" <rsync at lists.samba.org>
>> An: rsync at lists.samba.org
>> Betreff: rsync buffer overflow detected
>>
>> Hello!
>>
>> I use rsync from python on my Debian Jessie amd64 and get this error:
>> *** buffer overflow detected ***: <snip>/rsync terminated
>> ======= Backtrace: =========
>> /lib/x86_64-linux-gnu/libc.so.6(+0x731af)[0x7ffff78971af]
>> /lib/x86_64-linux-gnu/libc.so.6(__fortify_fail+0x37)[0x7ffff791caa7]
>> /lib/x86_64-linux-gnu/libc.so.6(+0xf6cc0)[0x7ffff791acc0]
>> /lib/x86_64-linux-gnu/libc.so.6(+0xf8a17)[0x7ffff791ca17]
>> <snip>/rsync(+0x30c78)[0x555555584c78]
>> <snip>/rsync(+0x31cfe)[0x555555585cfe]
>> <snip>/rsync(+0x31ef6)[0x555555585ef6]
>> <snip>/rsync(+0x336ed)[0x5555555876ed]
>> <snip>/rsync(+0x22417)[0x555555576417]
>> <snip>/rsync(+0x2395e)[0x55555557795e]
>> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5)[0x7ffff7845b45]
>> <snip>/rsync(+0x7f89)[0x55555555bf89]
>>
>> I guess that problem is about too many open fds
>> STR for this is a small script on python:
>> import os
>> import subprocess
>>
>> F = 'test'
>> OPENS = 1600
>>
>> cmd = [
>>     #'gdb', '--args',
>>     './rsync',
>>     '-aviH',
>>     '/etc/passwd',
>>     '/tmp/passwd'
>> ]
>>
>> for i in xrange(OPENS):
>>     fd = os.open(F, os.O_WRONLY | os.O_CREAT)
>> print(cmd)
>> subprocess.check_call(cmd)
>>
>> I rebuild rsync-3.1.1 from Debian source with debug and -O1 and get bt from gdb:
>> (gdb) bt
>> #0  0x00007ffff7859067 in __GI_raise (sig=sig at entry=6) at
>> ../nptl/sysdeps/unix/sysv/linux/raise.c:56
>> #1  0x00007ffff785a448 in __GI_abort () at abort.c:89
>> #2  0x00007ffff78971b4 in __libc_message (do_abort=do_abort at entry=2,
>> fmt=fmt at entry=0x7ffff7989cb3 "*** %s ***: %s terminated\n")
>>     at ../sysdeps/posix/libc_fatal.c:175
>> #3  0x00007ffff791caa7 in __GI___fortify_fail
>> (msg=msg at entry=0x7ffff7989c4a "buffer overflow detected") at
>> fortify_fail.c:31
>> #4  0x00007ffff791acc0 in __GI___chk_fail () at chk_fail.c:28
>> #5  0x00007ffff791ca17 in __fdelt_chk (d=d at entry=1606) at fdelt_chk.c:25
>> #6  0x0000555555584c78 in safe_read (fd=fd at entry=1606,
>> buf=buf at entry=0x7fffffffa810 "\037", len=len at entry=4) at io.c:245
>> #7  0x0000555555585cfe in read_buf (f=f at entry=1606,
>> buf=buf at entry=0x7fffffffa810 "\037", len=len at entry=4) at io.c:1815
>> #8  0x0000555555585ef6 in read_int (f=f at entry=1606) at io.c:1711
>> #9  0x00005555555876ed in setup_protocol (f_out=1605, f_in=1606) at compat.c:158
>> #10 0x0000555555576417 in client_run (f_in=1606, f_out=1605,
>> pid=24793, argc=1, argv=0x5555557d5240) at main.c:1128
>> #11 0x000055555557795e in start_client (argv=0x5555557d5240, argc=1)
>> at main.c:1423
>> #12 main (argc=2, argv=0x5555557d5240) at main.c:1651
>>
>> It looks like a bug, but I'm not sure)
>>
>> --
>> Boris
>>
>> --
>> Please use reply-all for most replies to avoid omitting the mailing list.
>> To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
>> Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
>>



-- 
Boris