Stale smbd processes (was: DOS: Clients can freeze other clients smbd)

[Nicolas Williams]

On Fri, Sep 03, 1999 at 09:08:04AM +0200, Mattias Gronlund wrote:
> Nicolas Williams wrote:
> 
> > There's two ways around this problem:
> >
> >  - set the 'keepalive' parameter to a sufficiently small value
> >
> >  - run a script via 'root preexec' and 'root postexec' to check for and
> >    kill smbd process[es] that were serving the same client.
> 
> But how do you set the keepalive-parameter per connection?

With the 'keepalive' smb.conf parameter. I don't know how well it works
by itself however.

NOTE: This is NOT the SO_KEEPALIVE option. You cannot rely on
      SO_KEEPALIVE achieving what you want.

> Is the script tested with multiuser Windows NT systems?

The script runs on the Samba server. I quoted the URL for my original
e-mail about this to the list; that e-mail includes the root
preexec/postexec commands I used and the script in question (I should
post a newer version of it). Read that. Basically, the script is called
with a number of arguments (% token substitutions) about the share
connection and it creates a pid/lock file named after those arguments.
These pid/lock files are used to detect stale smbd processes; it works
because the script's arguments identify the session/share connection in
question accurately.

> Your analysis sounds right, but I think that because we think that the NT-
> client is aggressive in reconnecting (we should check the exact timeout) we
> could just say that it is just hopeless to wait for longer than that in recv.

NT clients reconnect after a 45 second timeout.

> If the PC do not alvays reconnect after a short timeout, we should either
> process oplock-requests when waiting and give up if someone needs one
> of our locks. But I think we will find a timeout and not have to do any
> more advanced oplock-handling...

It's not just oplocks though. Share modes and plain locks are involved
as well. SMB relies on the client to keep lock state across failures; it
is up to SMB clients to attempt to restablish any locks they may have
been holding prior to any [network or server] failure. I like this
approach, actually.

Samba needs a way to deal with these stale smbd processes. I'm still not
exactly clear on what goes on that causes Samba to block on a socket,
that ought to be dead, waiting for input; I've not spent enough time
tracing the packets or the smbd processes so my analysis is partly based
on guessing (I had no idea about the FIN-FIN/ACK bug when I sent my very
first e-mail about this to the list); it could even be that there's a
bug in the way the NT clients abandon the old connection (i.e., maybe
they don't explicitly close it) or maybe there's a bug in NT's TCP/IP
stack that causes TCP shutdown to not be reliable.

Whatever the case I do know this: the solution I use works.

> At the moment I have a fix only in one-place that timeouts after 5 minutes
> that seems to keep us from getting hurds of smbd:s for a user...

Yeah. Five minutes works. When I foudn this problem I determined, during
testing, that with no workarounds the system recovers withing 12-15
minutes (i.e., the stale smbd processes take that long to figure out
that they ought to exit).

> /Mattias

We've only got experience with Samba running on Solaris, so the above
might only apply to Solaris. I wonder what others' experiences on other
platforms have been.

Nico
--
Nicolas Williams	(x5220, Stamford, CT)
Stamford SysAdmin

This message contains confidential information and is intended only 
for the individual named.  If you are not the named addressee you 
should not disseminate, distribute or copy this e-mail.  Please 
notify the sender immediately by e-mail if you have received this 
e-mail by mistake and delete this e-mail from your system.

E-mail transmission cannot be guaranteed to be secure or error-free 
as information could be intercepted, corrupted, lost, destroyed, 
arrive late or incomplete, or contain viruses.  The sender therefore 
does not accept liability for any errors or omissions in the contents 
of this message which arise as a result of e-mail transmission.  If 
verification is required please request a hard-copy version.  This 
message is provided for informational purposes and should not be 
construed as a solicitation or offer to buy or sell any securities or 
related financial instruments.