smbd process block
Sam Liddicott
sam at liddicott.com
Tue Apr 7 14:13:19 GMT 2009
* Goran Lowkrantz wrote, On 07/04/09 14:57:
> Hi Sam,
>
> Each of the calls look exactly the same except the multiplex id, i.e.
> it's the same FID and path, which is a leaf directory containing 202
> files. And there is only 48 notify calls before it stops, so I can't
> see how that maps to the total of about 400 directories in the watched
> tree. just looking at leaf directories give at least 300 directories
> to watch.
My mistake, I thought you said 245 calls, but I see now that you said
245 seconds.
>
> The thing I see is that the application issuing these notify requests
> go unresponsive after about 10 to 15 min, the frame marked Not
> responding. I assume this is because it is waiting for some response
> from the network, as killing the smbd service releases the application.
Try running some other application on the same share and see if the
share is responding - even windows explorer.
>
> And these notify setup messages are the only one I can find that do
> not have an SMB response.
This is quite normal for notify requests.
Try adding another file to a watched folder from another windows machine
and see if the application comes to life when it gets the notification
response back.
Sam
>
> /glz
>
> --On April 7, 2009 14:33:54 +0100 Sam Liddicott <sam at liddicott.com>
> wrote:
>
>> * Goran Lowkrantz wrote, On 07/04/09 14:08:
>>> Hi Sam,
>>>
>>> Looking further in the dump, I find that a series of these:
>>> NT Trans Request (0xa0)
>>> NT NOTIFY Setup
>>> Completion Filter: 0x00000003
>>> FID: 0x32f6 (\Input\2006\200607\20060729)
>>>
>>> but no response from the Samba server, only IP ACK. The FID look OK
>>> and direct queries on the FID about the directory is answered earlier.
>>>
>>> These messages are sent from the client every 5s for 245s. Then they
>>> stop.
>> I guess this may be once for every folder of 245 monitored folders.
>>
>> A response is not supposed to be given until the request is cancelled or
>> until a monitored file or folder changes; so this seems quite normal.
>>
>> What behaviour or action (or lack of) on the windows client makes you
>> think that samba has hung?
>>
>> Sam
>>>
>>> /glz
>>>
>>> --On April 6, 2009 15:28:31 +0100 Sam Liddicott <sam at liddicott.com>
>>> wrote:
>>>
>>>> As far as I can tell, samba is not hanging, it's issuing responses to
>>>> all the requests it gets.
>>>>
>>>> I think that the windows client thinks that samba has hung, because
>>>> the
>>>> sending of Echo Request is a sign that the client is getting fed up of
>>>> waiting and is checking to see if the link is still working.
>>>>
>>>> So it looks like the client is waiting for some kind of response,
>>>> but I
>>>> haven't been able to spot a response that wasn't answered.
>>>>
>>>> You load the pcap into wireshark and see if you can spot a request
>>>> that
>>>> doesn't get a response...
>>>>
>>>> Sam
>>>>
>>>> * Goran Lowkrantz wrote, On 16/02/09 22:42:
>>>>>
>>>>> I have few Samba servers running FreeBSD 7.1 were we have a problem
>>>>> with smbd process blocking for a few Vista systems that run a program
>>>>> that watch directories and files on the samba shares.
>>>>>
>>>>> On my test setup I have managed to get a hang in less than 30 min.
>>>>>
>>>>> Samba 3.2.7 is built with minimum functions and full debug. Options
>>>>> don't seems to have any impact on the problem.
>>>>>
>>>>> The PC uses Vista Business SP1 and all patches, I run a DAM program
>>>>> called IMatch ver 3.6.076 that watches for changes in the photo
>>>>> database. No other application I have tested has the same problems.
>>>>> For example, Adobe Lightroom 2.2 works without problems when setup
>>>>> with a watched folder.
>>>>>
>>>>> I have attached logfiles from samba with the following extra
>>>>> settings:
>>>>> debug pid = yes
>>>>> debug timestamp = no
>>>>> debug prefix timestamp = yes
>>>>> debug uid = yes
>>>>> log level = 10
>>>>>
>>>>> The PID of the server that hangs is 29162.
>>>>>
>>>>> The FreeBSD server is an up-to-date quad AMD server with 8GB running
>>>>> 7.1-STABLE. In normal operation, I see the following:
>>>>>
>>>>> # sockstat | grep 445
>>>>> glz smbd 7828 23 tcp4 10.255.253.1:445
>>>>> 10.255.253.2:57355
>>>>> root smbd 76917 19 tcp4 127.0.0.1:445 *:*
>>>>> root smbd 76917 20 tcp4 10.255.253.1:445 *:*
>>>>>
>>>>> When I get the hang, it looks like this:
>>>>> # sockstat | grep 445
>>>>> root smbd 7828 23 tcp4 10.255.253.1:445
>>>>> 10.255.253.2:57355
>>>>> root smbd 76917 19 tcp4 127.0.0.1:445 *:*
>>>>> root smbd 76917 20 tcp4 10.255.253.1:445 *:*
>>>>>
>>>>> and the GDB session:
>>>>> # gdb /usr/local/sbin/smbd 7828
>>>>> GNU gdb 6.1.1 [FreeBSD]
>>>>> Copyright 2004 Free Software Foundation, Inc.
>>>>> GDB is free software, covered by the GNU General Public License, and
>>>>> you are
>>>>> welcome to change it and/or distribute copies of it under certain
>>>>> conditions.
>>>>> Type "show copying" to see the conditions.
>>>>> There is absolutely no warranty for GDB. Type "show warranty" for
>>>>> details.
>>>>> This GDB was configured as "amd64-marcel-freebsd"...
>>>>> Attaching to program: /usr/local/sbin/smbd, process 7828
>>>>> Reading symbols from /usr/local/lib/libldap-2.3.so.2...done.
>>>>> Loaded symbols for /usr/local/lib/libldap-2.3.so.2
>>>>> Reading symbols from /usr/local/lib/liblber-2.3.so.2...done.
>>>>> Loaded symbols for /usr/local/lib/liblber-2.3.so.2
>>>>> Reading symbols from /usr/local/lib/libcups.so.2...done.
>>>>> Loaded symbols for /usr/local/lib/libcups.so.2
>>>>> Reading symbols from /usr/lib/libssl.so.5...done.
>>>>> Loaded symbols for /usr/lib/libssl.so.5
>>>>> Reading symbols from /lib/libcrypto.so.5...done.
>>>>> Loaded symbols for /lib/libcrypto.so.5
>>>>> Reading symbols from /lib/libz.so.4...done.
>>>>> Loaded symbols for /lib/libz.so.4
>>>>> Reading symbols from /lib/libm.so.5...done.
>>>>> Loaded symbols for /lib/libm.so.5
>>>>> Reading symbols from /lib/libcrypt.so.4...done.
>>>>> Loaded symbols for /lib/libcrypt.so.4
>>>>> Reading symbols from /usr/lib/libpam.so.4...done.
>>>>> Loaded symbols for /usr/lib/libpam.so.4
>>>>> Reading symbols from /usr/local/lib/libexecinfo.so.1...done.
>>>>> Loaded symbols for /usr/local/lib/libexecinfo.so.1
>>>>> Reading symbols from /usr/local/lib/libiconv.so.3...done.
>>>>> Loaded symbols for /usr/local/lib/libiconv.so.3
>>>>> Reading symbols from /usr/local/lib/libdmalloc.so.1...done.
>>>>> Loaded symbols for /usr/local/lib/libdmalloc.so.1
>>>>> Reading symbols from /usr/local/lib/libpopt.so.0...done.
>>>>> Loaded symbols for /usr/local/lib/libpopt.so.0
>>>>> Reading symbols from /lib/libthr.so.3...done.
>>>>> [New Thread 0x800a62e00 (LWP 100076)]
>>>>> Loaded symbols for /lib/libthr.so.3
>>>>> Reading symbols from /lib/libc.so.7...done.
>>>>> Loaded symbols for /lib/libc.so.7
>>>>> Reading symbols from /usr/local/lib/libsasl2.so.2...done.
>>>>> Loaded symbols for /usr/local/lib/libsasl2.so.2
>>>>> Reading symbols from /usr/local/lib/libintl.so.8...done.
>>>>> Loaded symbols for /usr/local/lib/libintl.so.8
>>>>> Reading symbols from /usr/local/lib/nss_ldap.so.1...done.
>>>>> Loaded symbols for /usr/local/lib/nss_ldap.so.1
>>>>> Reading symbols from /usr/lib/libcom_err.so.4...done.
>>>>> Loaded symbols for /usr/lib/libcom_err.so.4
>>>>> Reading symbols from /libexec/ld-elf.so.1...done.
>>>>> Loaded symbols for /libexec/ld-elf.so.1
>>>>> [Switching to Thread 0x800a62e00 (LWP 100076)]
>>>>> 0x0000000801f01d6c in select () from /lib/libc.so.7
>>>>> (gdb) directory /usr/ports/net/samba32-devel/work/samba-3.2.7/source/
>>>>> Source directories searched:
>>>>> /usr/ports/net/samba32-devel/work/samba-3.2.7/source:$cdir:$cwd
>>>>> (gdb) bt
>>>>> # 0 0x0000000801f01d6c in select () from /lib/libc.so.7
>>>>> # 1 0x0000000801d0f4d4 in select () from /lib/libthr.so.3
>>>>> # 2 0x00000000006749fe in sys_select (maxfd=24,
>>>>> readfds=0x7fffffffd420, writefds=0x7fffffffd3a0, errorfds=0x0,
>>>>> tval=0x7fffffffd500)
>>>>> at lib/select.c:93
>>>>> # 3 0x00000000004df64c in smbd_process () at smbd/process.c:839
>>>>> # 4 0x0000000000854074 in main (argc=2, argv=0x7fffffffd638) at
>>>>> smbd/server.c:1450
>>>>> (gdb) frame 2
>>>>> # 2 0x00000000006749fe in sys_select (maxfd=24,
>>>>> readfds=0x7fffffffd420, writefds=0x7fffffffd3a0, errorfds=0x0,
>>>>> tval=0x7fffffffd500)
>>>>> at lib/select.c:93
>>>>> 93 ret = select(maxfd,readfds2,writefds,errorfds,tval);
>>>>> (gdb) print tval
>>>>> $1 = (struct timeval *) 0x7fffffffd500
>>>>> (gdb) print *tval
>>>>> $2 = {tv_sec = 59, tv_usec = 999977}
>>>>> (gdb) The program is running. Quit anyway (and detach it)? (y or
>>>>> n) y
>>>>> Detaching from program: /usr/local/sbin/smbd, process 7828
>>>>>
>>>>> The following is a truss of the process until I have seen the switch
>>>>> to root as owner:
>>>>> # time truss -p 8307
>>>>> gettimeofday({1234648077.989004 },0x0) = 0 (0x0)
>>>>> gettimeofday({1234648077.989081 },0x0) = 0 (0x0)
>>>>> select(24,{6 23},{},0x0,{21.288167 }) = 0 (0x0)
>>>>> gettimeofday({1234648099.279293 },0x0) = 0 (0x0)
>>>>> gettimeofday({1234648099.279370 },0x0) = 0 (0x0)
>>>>> gettimeofday({1234648099.279417 },0x0) = 0 (0x0)
>>>>> select(24,{6 23},{},0x0,{59.989982 }) = 1 (0x1)
>>>>> gettimeofday({1234648102.286493 },0x0) = 0 (0x0)
>>>>> read(23,"\0\0\0r",4) = 4 (0x4)
>>>>> read(23,"\M^?SMB2\0\0\0\0\^X\a\M-H\0\0\0"...,114) = 114 (0x72)
>>>>> geteuid(0x3e8,0x3e8,0x2,0x800adf750,0x2,0x800adf750) = 0 (0x0)
>>>>> getegid(0x3e8,0x3e8,0x2,0x801eadb8c,0xffffff006cf16a50,0x7fffffffd138)
>>>>>
>>>>> = 0 (0x0)
>>>>> __sysctl(0x7fffffffd0a0,0x2,0x7fffffffd0bc,0x7fffffffd0b0,0x0,0x0)
>>>>> = 0
>>>>> (0x0)
>>>>> 0.000u 0.001s 2:36.56 0.0% 0+0k 0+0io 0pf+0w
>>>>>
>>>>> # sockstat | grep 445
>>>>> glz smbd 8307 23 tcp4 10.255.253.1:445
>>>>> 10.255.253.2:57438
>>>>> root smbd 76917 19 tcp4 127.0.0.1:445 *:*
>>>>> root smbd 76917 20 tcp4 10.255.253.1:445 *:*
>>>>> # ps -awxl | grep 8307
>>>>> 1000 8307 8556 0 44 0 34672 7984 select IX ?? 0:04.57
>>>>> /usr/local/sbin/smbd -D -s /usr/local/etc/smb.conf
>>>>> 0 8556 3273 0 8 0 4600 1204 wait I+ p0 0:00.00
>>>>> truss -p 8307
>>>>> # sockstat | grep 445
>>>>> root smbd 8307 23 tcp4 10.255.253.1:445
>>>>> 10.255.253.2:57438
>>>>> root smbd 76917 19 tcp4 127.0.0.1:445 *:*
>>>>> root smbd 76917 20 tcp4 10.255.253.1:445 *:*
>>>>> # ps -awxl | grep 8307
>>>>> 0 8307 8556 0 44 0 34672 7984 select SX ?? 0:04.57
>>>>> /usr/local/sbin/smbd -D -s /usr/local/etc/smb.conf
>>>>> 0 8556 3273 0 8 0 4600 1204 wait I+ p0 0:00.00
>>>>> truss -p 8307
>>>>>
>>>>>
>>>>> I can recreate this at any time and the condition by killing the
>>>>> offending smbd process and the PC reconnects just fine.
>>>>>
>>>>> Hope this helps to pin this down. As I can recreate the hang, please
>>>>> let me know if there is any more information I can supply.
>>>>>
>>>>> /glz
>>>>>
>>>>> ................................................... the future
>>>>> isMobile
>>>>>
>>>>> Goran Lowkrantz <goran.lowkrantz at ismobile.com>
>>>>> System Architect, isMobile, Aurorum 2, S-977 75 Luleå, Sweden
>>>>> Phone: +46(0)920-75559
>>>>> Mobile: +46(0)70-587 87 82 Fax: +46(0)70-615 87 82
>>>>>
>>>>> http://www.ismobile.com
>>>>> ...............................................
>>>
>>>
>>>
>>> ................................................... the future isMobile
>>>
>>> Goran Lowkrantz <goran.lowkrantz at ismobile.com>
>>> System Architect, isMobile AB
>>> Sandviksgatan 81, PO Box 58, S-971 03 Luleå, Sweden
>>> Mobile: +46(0)70-587 87 82
>>> http://www.ismobile.com ...............................................
>
>
>
> ................................................... the future isMobile
>
> Goran Lowkrantz <goran.lowkrantz at ismobile.com>
> System Architect, isMobile AB
> Sandviksgatan 81, PO Box 58, S-971 03 Luleå, Sweden
> Mobile: +46(0)70-587 87 82
> http://www.ismobile.com ...............................................
More information about the samba-technical
mailing list