Lots of rsync daemons running

Fri Jan 15 09:49:26 MST 2010

Hi,

Strace is rather quiet; they all seem to be stuck in this call:

futex(0xc81140, FUTEX_WAIT, 2, NULL

That’s not very useful I suppose, but GDB’s output might be:

$ gdb /usr/bin/rsync 26359
#0  0x00e9d402 in __kernel_vsyscall ()
#1  0x00c1dde3 in __lll_lock_wait_private () from /lib/libc.so.6
#2  0x00bad70e in _L_lock_15714 () from /lib/libc.so.6
#3  0x00bacea1 in realloc () from /lib/libc.so.6
#4  0x00b9fc8a in _IO_mem_finish () from /lib/libc.so.6
#5  0x00b97bdb in fclose@@GLIBC_2.1 () from /lib/libc.so.6
#6  0x00c0d8ea in __vsyslog_chk () from /lib/libc.so.6
#7  0x00c0dd59 in __syslog_chk () from /lib/libc.so.6
#8  0x0806892c in rwrite ()
#9  0x08068d44 in rprintf ()
#10 0x08068e19 in log_exit ()
#11 0x0805bff0 in _exit_cleanup ()
#12 0x08062df2 in sigusr1_handler ()
#13 <signal handler called>
#14 0x00bd13b1 in fork () from /lib/libc.so.6
#15 0x08061dcb in do_fork ()
#16 0x080642c6 in do_recv ()
#17 0x08064c7b in start_server ()
#18 0x08081cda in start_daemon ()
#19 0x080824b4 in daemon_main ()
#20 0x080654bf in main ()

So it’s stuck at logging it has finished? Weird. Could it be a problem that
I have several clients (about 10) connecting at once? Or that they connect
from localhost from a SSH tunnel (I use SSH portforwarding)?

Regards,

Jeroen van der Vegt
System designer 
________________________________________
Technolution B.V.

>
>From: 4wayned at gmail.com [mailto:4wayned at gmail.com] On Behalf Of Wayne
Davison
>Sent: vrijdag 15 januari 2010 16:56
>To: Jeroen van der Vegt
>Cc: rsync at lists.samba.org
>Subject: Re: Lots of rsync daemons running
>
>On Fri, Jan 15, 2010 at 5:26 AM, Jeroen van der Vegt
<jeroen.van.der.vegt at technolution.nl> wrote:
>>Today I noticed several rsync processes (23 to be exact) running on the
>>server since as long back as November 2009 (according to ps, and also
>>/proc/[pid]/stat ). As the clients run for at most a day, that's not what
>>I'd expect: I have no clue why some processes do not seem to terminate.
>
>If the clients are indeed gone, then either rsync is not getting an error
>reading from the dead socket, or it is hung up doing something else that is

>hanging, such as trying to read from a file that hangs in the kernel.
>Check 
>what system call rsync is currently doing using strace (or equivalent).  If

>it is doing a series of 30-second select() calls, the sockets are not 
>erroring out, and using a "timeout = 600" setting in your rsyncd.conf file 
>will ensure that rsync will eventually die if the socket goes silent
>(though it would be nice to investigate why the socket is behaving in such
>an abnormal manner as well).  If it is some other call, you should try to 
>figure out what it is doing and why that is hanging (e.g. lsof to figure
out currently open files may give you a clue).
>
>..wayne..
>

Telephone: 
+31(0)182 59 40 00

Fax: 
+31(0)182 53 97 36

E-mail: 
Jeroen.van.der.Vegt at technolution.eu

Visit us at: 
www.technolution.eu

Mailing address: 
P.O. Box 2013 - 2800 BD Gouda - The Netherlands

Address: 
Zuidelijk Halfrond 1 - 2801 DD Gouda - The Netherlands

________________________________________
This e-mail is intended exclusively for the addressee(s), and may not be
passed on to, or made available for use by any person other than the
addressee(s). Technolution B.V. rules out any and every liability resulting
from any electronic transmission. 

________________________________________
From: 4wayned at gmail.com [mailto:4wayned at gmail.com] On Behalf Of Wayne
Davison
Sent: vrijdag 15 januari 2010 16:56
To: Jeroen van der Vegt
Cc: rsync at lists.samba.org
Subject: Re: Lots of rsync daemons running

On Fri, Jan 15, 2010 at 5:26 AM, Jeroen van der Vegt
<jeroen.van.der.vegt at technolution.nl> wrote:
Today I noticed several rsync processes (23 to be exact) running on the
server since as long back as November 2009 (according to ps, and also
/proc/[pid]/stat ). As the clients run for at most a day, that's not what
I'd expect: I have no clue why some processes do not seem to terminate.

If the clients are indeed gone, then either rsync is not getting an error
reading from the dead socket, or it is hung up doing something else that is
hanging, such as trying to read from a file that hangs in the kernel.  Check
what system call rsync is currently doing using strace (or equivalent).  If
it is doing a series of 30-second select() calls, the sockets are not
erroring out, and using a "timeout = 600" setting in your rsyncd.conf file
will ensure that rsync will eventually die if the socket goes silent (though
it would be nice to investigate why the socket is behaving in such an
abnormal manner as well).  If it is some other call, you should try to
figure out what it is doing and why that is hanging (e.g. lsof to figure out
currently open files may give you a clue).

..wayne..