An interesting shutdown race in messaging

[Jeremy Allison]

On Tue, Jun 13, 2017 at 09:15:33AM -0700, Jeremy Allison via samba-technical wrote:
> On Tue, Jun 13, 2017 at 05:14:01PM +1200, Andrew Bartlett wrote:
> > G'Day,
> > 
> > I have another messaging issue that has come up.
> > 
> > In my attempts to make progress on the ldb locking code, I tried moving
> > all of the servers in 'make test' to the standard process model. 
> > 
> > However, this hits an issue in the auth_log tests, which use messaging
> > to contact an external python process, the tests, which expect to be
> > sent the audit messages.
> > 
> > The issue is that in -M standard and in smbd, the connection can be
> > closed so fast that the messages that are written to the logs, and have
> > presumably been dispached to a messaging thread, never make it to the
> > other end.
> > 
> > I can work around it by holding the connection open in some of the
> > tests, but I can't easily do that for the tests that call rpcclient
> > over subprocess, or where the password fails on LDAP.
> > 
> > So, I wonder if there is a way to, before we call exit(0) in the
> > standard process model, have it try again to dispatch the messages to
> > somewhere safely on the other side of a connected or datagram fd?
> > 
> > This can be seen in the auth_log tests in 
> > http://git.catalyst.net.nz/gitweb?p=samba.git;a=shortlog;h=refs/heads/ldb-safe-locking-for-master
> > 
> > The commit that causes the issue is 
> > http://git.catalyst.net.nz/gitweb?p=samba.git;a=commitdiff;h=6cf5e9f9b9f27af56a5a42ea6d7259f06f35240c
> > 
> > (changing all "single" to "standard" in Samba4.pm).
> > 
> > Fixing this would allow folks to use the messaging transport for the
> > auth audit messages in a reliable way (instead of reading the logs),
> > which isn't currently promised (only listed as a developer feature
> > right now), but could be quite handy.  
> > 
> > Any clues most appreciated,
> > 
> > (Jeremy CC'ed because I mentioned this to him a long while ago).
> 
> So what you need is a 'drain_sent_messages' kind of function,
> that will block until all pending messages are flushed out of
> the system or the receiver disconnects, or an error send gets
> and error - yes ?

Hmmm. I think this could go inside of imessaging_cleanup()
in source4. That seems to be called in all the right places
and suggests it should do the right thing (currently it
does nothing).