[Samba] 2500 smbd processes for 30 users. tdb_oob len beyond eof.

[Bob Crandell]

I have almost the same setup except not as many users.  That particular problem went
away when I stopped using inetd.

ard at waikato.ac.nz wrote*:
>
>
>Yesterday my Samba server stopped answering mount requests for the first
>time in four months.  There were 2500 smbd processes running, consuming
>both CPUs, a gig of RAM, and a gig of swap.  Sysstat provided interesting
>graphs.  "smbstatus" showed about 30 connected users.  Normally I have
>about ten times that.
>
>New mount requests were being refused, or timing out:
>
>        # mount -tsmbfs //server/share /mnt -ousername=etc,uid=etc
>        Password:
>        15438: session setup failed: SUCCESS - 0
>        SMB connection failed
>
>Windows users were unable to log in because this Samba box is their home H:
>drive server.
>
>I fixed it with "killall smbd" and now everything is back to normal (smbd
>is started by inetd).  I did not delete any log files or the connection
>database, as others on this list have had to do.
>
>Occasionally in the past smbstatus has reported "tdb_oob len beyond eof"
>but I have always ignored it because smbd continues working.  Only now,
>after searching the mailing list archives, I see that it has been terminal
>for some users.
>
>The system is a dual P3 Compaq, 2.4.18-ac1 (-ac1 to get proper quotas),
>Samba-2.2.3a.  The original installation was Slackware-8.0, if I remember
>correctly, but all apps are compiled from source rather than not taken from
>packages.  Load peaks around 300-400 concurrent connections per day with
>deadtime set to 30 minutes.  This peak happens between 12pm and 1pm every
>weekday.  The "crisis" happened at 12:35pm Thursday.
>
>Logging went into orbit:
>
># grep smbd daemon.log | cut -f 1 -d: | uniq -c
>    304 Jul 18 00
>    120 Jul 18 01
>    122 Jul 18 02
>     98 Jul 18 03
>    102 Jul 18 04
>    102 Jul 18 05
>    118 Jul 18 06
>    170 Jul 18 07
>   2012 Jul 18 08
>   4896 Jul 18 09
>   8262 Jul 18 10
>   8348 Jul 18 11
>1886813 Jul 18 12
> 618309 Jul 18 13
>  78478 Jul 18 14
>    323 Jul 18 15
>    342 Jul 18 16
>[...continues around 300 for the rest of the day...]
>
># grep tdb_oob daemon.log |cut -f 1 -d: |uniq -c
> 937028 Jul 18 12
> 300362 Jul 18 13
>  35386 Jul 18 14
>
># grep tdb_oob daemon.log | cut -f 1,2 -d: | uniq -c
>   9007 Jul 18 12:04
>  18203 Jul 18 12:05
>  21734 Jul 18 12:06
>[...about 20000 *every* minute until 12:30, then linear dropoff to 3000 per
>   minute at 14:10, ...]
>   3041 Jul 18 14:11
>   2134 Jul 18 14:12
>   1659 Jul 18 14:13      <---- killall smbd here
>      6 Jul 18 14:14      <---- ...taking several minutes to complete
>      6 Jul 18 14:15
>      5 Jul 18 14:16
>      6 Jul 18 14:17
>      5 Jul 18 14:18
>      2 Jul 18 14:19
>
>Next week I will upgrade to 2.2.5, but with a large user base I have to
>take some care.  In the meantime, can anybody offer any suggestions?
>
>
>_________________________________________________________________________
>Andrew Donkin                  Waikato University, Hamilton,  New Zealand
>
>
>P.S. does anybody else think that splitting log lines in two, the way Samba
>does, is madness?
>
>
--
Bob Crandell
Assured Computing
When you need to be sure.
Cell 541-914-3985
FAX  240-371-7237
bob at assuredcomp.com
www.assuredcomp.com
Eugene, Or. 97402