Stale smbd processes (was: DOS: Clients can freeze other clients smbd)

[Nicolas Williams]

On Tue, Sep 07, 1999 at 10:34:21PM +0200, Mattias.Gronlund wrote:
> Nicolas Williams wrote:
> 
[...]
> Yesterday I debugged one client which couldn't upload files to the
> server
> and saw that the timeout was trigged. We sniffed the network and it
> looked
> like the client sent the packets, but after a while there where no more
> reply:s from the server. The client started to resedn the packet sevral
> times but no reply. When insted looking from the servers point of view
> the packet in question never reached the server. The network just ate
> the
> packet! So, the problems I see is not Sambas fault, it is just that
> Samba
> in the official release isn't that good on handling bad networks :-(...

Well, yes. And as you point out [below] smbd's handling of oplock breaks
leaves something to be desired...

[...]
> > Technically TCP ought to recover connection shutdowns from short term
> > packet loss network conditions. What's happening indicates that this is
> > not happening. This is why the other day I theorized that the
> > FIN-FIN/ACK bug may be to blame, but it could be other things. Whatever
> > it is we ought to find it and get Sun and/or Microsoft to fix it.
> 
> Or 3COM! But I do not expect them to fix buggs.

Well, yes, but some random packet loss should not result in this kind of
problem. 

[...]
> > It's easy to test folks, connect to some [non-Solaris] Samba server from
> > an NT workstation, open a file on that share with some editor, yank
> > either the client's or the server's network connection, try to save,
> > wait 1 minute, put the network back, contionue trying to save the file.
> > 
> > If after you put the network back your editor hangs for long, then you
> > have the same problem we're talking about in this thread.
> 
> This sounds odd. My understanding of Sambas behaviour is like this:
> o The editor gets an oplock that the smbd-process register.
> o The smbd-process waits for oplock-breaks or client requests.
> o You cut the network.
> o The client tries to save, but times out and closes its side of the
>   socket.
> o You reconnect the network and the client will make a new connection
> which
>   will start a new smbd-process.
> o The client will try to get the oplock again so the new smbd-process
> will
>   contact the "stale" process which should try to contact the client.
>   The client has closed the connection and should therefor respond with
>   a reset of the connection.
> o The smbd should let go of the lock report to the other smbd and die.
> o The new smbd should get the oplock.
> o This should go fast.

Well, this is what ought to happen. But remember that oplocks are not
the only kind of locks involved; I don't think there's anything like
oplock break requests for the other kinds of locks (nor would it make
sense to me for them to exist).

> But I see that there may be some open ends:

Let me add some myself:

Does the TCP shutdown proceed correctly and recover from packet losses?

> Do the client respond with a reset?
> Do the server handle the reset the right way?

Does the server's kernel indicate the reset to smbd (by returning
read(2) or what have you with an appropriate error)?

> Do smbd understand the closed connection?

If smbd is using select() or poll() is it asking for error events?

> Do smbd release the oplocks before terminating?

Yes. Smbd does release all locks prior to exiting, when terminating
cleanly (i.e., don't SIGKILL smbd!). Were this otherwise my stale smbd
stomper script wouldn't work :)

> I do not have time to test this for a while, but it looks interesting.
> 
> /Mattias

Nor do I... I am curious about other platforms though. It sounds like
there's some interplay of bugs here.

Nico
--
-DISCLAIMER: an automatically appended disclaimer may follow. By posting-
-to a public e-mail mailing list I hereby grant permission to distribute-
-and copy this message.-

This message contains confidential information and is intended only 
for the individual named.  If you are not the named addressee you 
should not disseminate, distribute or copy this e-mail.  Please 
notify the sender immediately by e-mail if you have received this 
e-mail by mistake and delete this e-mail from your system.

E-mail transmission cannot be guaranteed to be secure or error-free 
as information could be intercepted, corrupted, lost, destroyed, 
arrive late or incomplete, or contain viruses.  The sender therefore 
does not accept liability for any errors or omissions in the contents 
of this message which arise as a result of e-mail transmission.  If 
verification is required please request a hard-copy version.  This 
message is provided for informational purposes and should not be 
construed as a solicitation or offer to buy or sell any securities or 
related financial instruments.