smbd process block

Sam Liddicott sam at liddicott.com
Tue Apr 7 13:08:20 GMT 2009


* Goran Lowkrantz wrote, On 07/04/09 12:11:
> Hi Sam,
>
> The only thing that looks like a timeout is after the Tree Connect
> AndX Request/Response/ACK sequence (packets 10618/19/20). After that
> the echo requests start from the Vista client. The timeout is 55 sec
> (packet 10621).
10618-106120 takes less than 1/4 second.

Certainly the 55 seconds that follow to 10621 suggests that windows is
waiting for something, but it's hard to tell what request timed out. I
don't even see (among so many) any requests that give unusual responses
that might put windows in some weird state.

Try doing an lsof -p PID on the samba process in question and see if it
has run out of file handles or something...

Sam

>
> What is supposed to happen after a Tree Connect AndX? Should the
> server respond with more data? It does send an NBSS Session keep-alive
> further down (packet 10645).
>
> Any pointers where I should look is appreciated.
>
> /glz
>
> --On April 6, 2009 15:28:31 +0100 Sam Liddicott <sam at liddicott.com>
> wrote:
>
>> As far as I can tell, samba is not hanging, it's issuing responses to
>> all the requests it gets.
>>
>> I think that the windows client thinks that samba has hung, because the
>> sending of Echo Request is a sign that the client is getting fed up of
>> waiting and is checking to see if the link is still working.
>>
>> So it looks like the client is waiting for some kind of response, but I
>> haven't been able to spot a response that wasn't answered.
>>
>> You load the pcap into wireshark and see if you can spot a request that
>> doesn't get a response...
>>
>> Sam
>>
>> * Goran Lowkrantz wrote, On 16/02/09 22:42:
>>>
>>> I have few Samba servers running FreeBSD 7.1 were we have a problem
>>> with smbd process blocking for a few Vista systems that run a program
>>> that watch directories and files on the samba shares.
>>>
>>> On my test setup I have managed to get a hang in less than 30 min.
>>>
>>> Samba 3.2.7 is built with minimum functions and full debug. Options
>>> don't seems to have any impact on the problem.
>>>
>>> The PC uses Vista Business SP1 and all patches, I run a DAM program
>>> called IMatch ver 3.6.076 that watches for changes in the photo
>>> database. No other application I have tested has the same problems.
>>> For example, Adobe Lightroom 2.2 works without problems when setup
>>> with a watched folder.
>>>
>>> I have attached logfiles from samba with the following extra settings:
>>> debug pid = yes
>>> debug timestamp = no
>>> debug prefix timestamp = yes
>>> debug uid = yes
>>> log level = 10
>>>
>>> The PID of the server that hangs is 29162.
>>>
>>> The FreeBSD server is an up-to-date quad AMD server with 8GB running
>>> 7.1-STABLE. In normal operation, I see the following:
>>>
>>> # sockstat | grep 445
>>> glz      smbd       7828  23 tcp4   10.255.253.1:445
>>> 10.255.253.2:57355
>>> root     smbd       76917 19 tcp4   127.0.0.1:445         *:*
>>> root     smbd       76917 20 tcp4   10.255.253.1:445      *:*
>>>
>>> When I get the hang, it looks like this:
>>> # sockstat | grep 445
>>> root     smbd       7828  23 tcp4   10.255.253.1:445
>>> 10.255.253.2:57355
>>> root     smbd       76917 19 tcp4   127.0.0.1:445         *:*
>>> root     smbd       76917 20 tcp4   10.255.253.1:445      *:*
>>>
>>> and the GDB session:
>>> # gdb /usr/local/sbin/smbd 7828
>>> GNU gdb 6.1.1 [FreeBSD]
>>> Copyright 2004 Free Software Foundation, Inc.
>>> GDB is free software, covered by the GNU General Public License, and
>>> you are
>>> welcome to change it and/or distribute copies of it under certain
>>> conditions.
>>> Type "show copying" to see the conditions.
>>> There is absolutely no warranty for GDB.  Type "show warranty" for
>>> details.
>>> This GDB was configured as "amd64-marcel-freebsd"...
>>> Attaching to program: /usr/local/sbin/smbd, process 7828
>>> Reading symbols from /usr/local/lib/libldap-2.3.so.2...done.
>>> Loaded symbols for /usr/local/lib/libldap-2.3.so.2
>>> Reading symbols from /usr/local/lib/liblber-2.3.so.2...done.
>>> Loaded symbols for /usr/local/lib/liblber-2.3.so.2
>>> Reading symbols from /usr/local/lib/libcups.so.2...done.
>>> Loaded symbols for /usr/local/lib/libcups.so.2
>>> Reading symbols from /usr/lib/libssl.so.5...done.
>>> Loaded symbols for /usr/lib/libssl.so.5
>>> Reading symbols from /lib/libcrypto.so.5...done.
>>> Loaded symbols for /lib/libcrypto.so.5
>>> Reading symbols from /lib/libz.so.4...done.
>>> Loaded symbols for /lib/libz.so.4
>>> Reading symbols from /lib/libm.so.5...done.
>>> Loaded symbols for /lib/libm.so.5
>>> Reading symbols from /lib/libcrypt.so.4...done.
>>> Loaded symbols for /lib/libcrypt.so.4
>>> Reading symbols from /usr/lib/libpam.so.4...done.
>>> Loaded symbols for /usr/lib/libpam.so.4
>>> Reading symbols from /usr/local/lib/libexecinfo.so.1...done.
>>> Loaded symbols for /usr/local/lib/libexecinfo.so.1
>>> Reading symbols from /usr/local/lib/libiconv.so.3...done.
>>> Loaded symbols for /usr/local/lib/libiconv.so.3
>>> Reading symbols from /usr/local/lib/libdmalloc.so.1...done.
>>> Loaded symbols for /usr/local/lib/libdmalloc.so.1
>>> Reading symbols from /usr/local/lib/libpopt.so.0...done.
>>> Loaded symbols for /usr/local/lib/libpopt.so.0
>>> Reading symbols from /lib/libthr.so.3...done.
>>> [New Thread 0x800a62e00 (LWP 100076)]
>>> Loaded symbols for /lib/libthr.so.3
>>> Reading symbols from /lib/libc.so.7...done.
>>> Loaded symbols for /lib/libc.so.7
>>> Reading symbols from /usr/local/lib/libsasl2.so.2...done.
>>> Loaded symbols for /usr/local/lib/libsasl2.so.2
>>> Reading symbols from /usr/local/lib/libintl.so.8...done.
>>> Loaded symbols for /usr/local/lib/libintl.so.8
>>> Reading symbols from /usr/local/lib/nss_ldap.so.1...done.
>>> Loaded symbols for /usr/local/lib/nss_ldap.so.1
>>> Reading symbols from /usr/lib/libcom_err.so.4...done.
>>> Loaded symbols for /usr/lib/libcom_err.so.4
>>> Reading symbols from /libexec/ld-elf.so.1...done.
>>> Loaded symbols for /libexec/ld-elf.so.1
>>> [Switching to Thread 0x800a62e00 (LWP 100076)]
>>> 0x0000000801f01d6c in select () from /lib/libc.so.7
>>> (gdb) directory /usr/ports/net/samba32-devel/work/samba-3.2.7/source/
>>> Source directories searched:
>>> /usr/ports/net/samba32-devel/work/samba-3.2.7/source:$cdir:$cwd
>>> (gdb) bt
>>> # 0  0x0000000801f01d6c in select () from /lib/libc.so.7
>>> # 1  0x0000000801d0f4d4 in select () from /lib/libthr.so.3
>>> # 2  0x00000000006749fe in sys_select (maxfd=24,
>>> readfds=0x7fffffffd420, writefds=0x7fffffffd3a0, errorfds=0x0,
>>> tval=0x7fffffffd500)
>>>    at lib/select.c:93
>>> # 3  0x00000000004df64c in smbd_process () at smbd/process.c:839
>>> # 4  0x0000000000854074 in main (argc=2, argv=0x7fffffffd638) at
>>> smbd/server.c:1450
>>> (gdb) frame 2
>>> # 2  0x00000000006749fe in sys_select (maxfd=24,
>>> readfds=0x7fffffffd420, writefds=0x7fffffffd3a0, errorfds=0x0,
>>> tval=0x7fffffffd500)
>>>    at lib/select.c:93
>>> 93              ret = select(maxfd,readfds2,writefds,errorfds,tval);
>>> (gdb) print tval
>>> $1 = (struct timeval *) 0x7fffffffd500
>>> (gdb) print *tval
>>> $2 = {tv_sec = 59, tv_usec = 999977}
>>> (gdb) The program is running.  Quit anyway (and detach it)? (y or n) y
>>> Detaching from program: /usr/local/sbin/smbd, process 7828
>>>
>>> The following is a truss of the process until I have seen the switch
>>> to root as owner:
>>> # time truss -p 8307
>>> gettimeofday({1234648077.989004 },0x0)           = 0 (0x0)
>>> gettimeofday({1234648077.989081 },0x0)           = 0 (0x0)
>>> select(24,{6 23},{},0x0,{21.288167 })            = 0 (0x0)
>>> gettimeofday({1234648099.279293 },0x0)           = 0 (0x0)
>>> gettimeofday({1234648099.279370 },0x0)           = 0 (0x0)
>>> gettimeofday({1234648099.279417 },0x0)           = 0 (0x0)
>>> select(24,{6 23},{},0x0,{59.989982 })            = 1 (0x1)
>>> gettimeofday({1234648102.286493 },0x0)           = 0 (0x0)
>>> read(23,"\0\0\0r",4)                             = 4 (0x4)
>>> read(23,"\M^?SMB2\0\0\0\0\^X\a\M-H\0\0\0"...,114) = 114 (0x72)
>>> geteuid(0x3e8,0x3e8,0x2,0x800adf750,0x2,0x800adf750) = 0 (0x0)
>>> getegid(0x3e8,0x3e8,0x2,0x801eadb8c,0xffffff006cf16a50,0x7fffffffd138)
>>> = 0 (0x0)
>>> __sysctl(0x7fffffffd0a0,0x2,0x7fffffffd0bc,0x7fffffffd0b0,0x0,0x0) = 0
>>> (0x0)
>>> 0.000u 0.001s 2:36.56 0.0%    0+0k 0+0io 0pf+0w
>>>
>>> # sockstat | grep 445
>>> glz      smbd       8307  23 tcp4   10.255.253.1:445
>>> 10.255.253.2:57438
>>> root     smbd       76917 19 tcp4   127.0.0.1:445         *:*
>>> root     smbd       76917 20 tcp4   10.255.253.1:445      *:*
>>> # ps -awxl | grep 8307
>>> 1000  8307  8556   0  44  0 34672  7984 select IX    ??    0:04.57
>>> /usr/local/sbin/smbd -D -s /usr/local/etc/smb.conf
>>>    0  8556  3273   0   8  0  4600  1204 wait   I+    p0    0:00.00
>>> truss -p 8307
>>> # sockstat | grep 445
>>> root     smbd       8307  23 tcp4   10.255.253.1:445
>>> 10.255.253.2:57438
>>> root     smbd       76917 19 tcp4   127.0.0.1:445         *:*
>>> root     smbd       76917 20 tcp4   10.255.253.1:445      *:*
>>> # ps -awxl | grep 8307
>>>    0  8307  8556   0  44  0 34672  7984 select SX    ??    0:04.57
>>> /usr/local/sbin/smbd -D -s /usr/local/etc/smb.conf
>>>    0  8556  3273   0   8  0  4600  1204 wait   I+    p0    0:00.00
>>> truss -p 8307
>>>
>>>
>>> I can recreate this at any time and the condition by killing the
>>> offending smbd process and the PC reconnects just fine.
>>>
>>> Hope this helps to pin this down. As I can recreate the hang, please
>>> let me know if there is any more information I can supply.
>>>
>>> /glz
>>>
>>> ................................................... the future isMobile
>>>
>>>  Goran Lowkrantz <goran.lowkrantz at ismobile.com>
>>>  System Architect, isMobile, Aurorum 2, S-977 75 Luleå, Sweden
>>>  Phone: +46(0)920-75559
>>>  Mobile: +46(0)70-587 87 82 Fax: +46(0)70-615 87 82
>>>
>>> http://www.ismobile.com ...............................................
>
>
>
> ................................................... the future isMobile
>
>  Goran Lowkrantz <goran.lowkrantz at ismobile.com>
>  System Architect, isMobile AB
>  Sandviksgatan 81, PO Box 58, S-971 03 Luleå, Sweden
>  Mobile: +46(0)70-587 87 82
> http://www.ismobile.com ...............................................



More information about the samba-technical mailing list