An interesting shutdown race in messaging

[Jeremy Allison]

On Tue, Jun 13, 2017 at 12:45:24PM -0400, jim via samba-technical wrote:
> Perhaps with a reasonable timeout in case one of the send targets is
> unresponsive so shutdown/exit is not blocked for an excessive time.

Yep, allow a milisecond configurable timeout here.

> On 6/13/2017 12:30 PM, Jeremy Allison via samba-technical wrote:
> >On Tue, Jun 13, 2017 at 09:15:33AM -0700, Jeremy Allison via samba-technical wrote:
> >>On Tue, Jun 13, 2017 at 05:14:01PM +1200, Andrew Bartlett wrote:
> >>>G'Day,
> >>>
> >>>I have another messaging issue that has come up.
> >>>
> >>>In my attempts to make progress on the ldb locking code, I tried moving
> >>>all of the servers in 'make test' to the standard process model.
> >>>
> >>>However, this hits an issue in the auth_log tests, which use messaging
> >>>to contact an external python process, the tests, which expect to be
> >>>sent the audit messages.
> >>>
> >>>The issue is that in -M standard and in smbd, the connection can be
> >>>closed so fast that the messages that are written to the logs, and have
> >>>presumably been dispached to a messaging thread, never make it to the
> >>>other end.
> >>>
> >>>I can work around it by holding the connection open in some of the
> >>>tests, but I can't easily do that for the tests that call rpcclient
> >>>over subprocess, or where the password fails on LDAP.
> >>>
> >>>So, I wonder if there is a way to, before we call exit(0) in the
> >>>standard process model, have it try again to dispatch the messages to
> >>>somewhere safely on the other side of a connected or datagram fd?
> >>>
> >>>This can be seen in the auth_log tests in
> >>>https://urldefense.proofpoint.com/v2/url?u=http-3A__git.catalyst.net.nz_gitweb-3Fp-3Dsamba.git-3Ba-3Dshortlog-3Bh-3Drefs_heads_ldb-2Dsafe-2Dlocking-2Dfor-2Dmaster&d=DwIBAg&c=y2w-uYmhgFWijp_IQN0DhA&r=Ru1enaR19oK8d7j6CTyYvKyHRw2ZzjrSu2PJyn0fKAQ&m=dATI2CzoCWdCutLO-aBWR9gkhLDCUKpGFxwAS2Bk2-0&s=ELYWZqACBhU2KcpkDMKzhdVfbg5C8i_YjDoA3Ct-xwU&e=
> >>>
> >>>The commit that causes the issue is
> >>>https://urldefense.proofpoint.com/v2/url?u=http-3A__git.catalyst.net.nz_gitweb-3Fp-3Dsamba.git-3Ba-3Dcommitdiff-3Bh-3D6cf5e9f9b9f27af56a5a42ea6d7259f06f35240c&d=DwIBAg&c=y2w-uYmhgFWijp_IQN0DhA&r=Ru1enaR19oK8d7j6CTyYvKyHRw2ZzjrSu2PJyn0fKAQ&m=dATI2CzoCWdCutLO-aBWR9gkhLDCUKpGFxwAS2Bk2-0&s=eT09MGnbSidl-oeDf0uzHgh5pBqzPTEHT_7GLyYm3u0&e=
> >>>
> >>>(changing all "single" to "standard" in Samba4.pm).
> >>>
> >>>Fixing this would allow folks to use the messaging transport for the
> >>>auth audit messages in a reliable way (instead of reading the logs),
> >>>which isn't currently promised (only listed as a developer feature
> >>>right now), but could be quite handy.
> >>>
> >>>Any clues most appreciated,
> >>>
> >>>(Jeremy CC'ed because I mentioned this to him a long while ago).
> >>So what you need is a 'drain_sent_messages' kind of function,
> >>that will block until all pending messages are flushed out of
> >>the system or the receiver disconnects, or an error send gets
> >>and error - yes ?
> >Hmmm. I think this could go inside of imessaging_cleanup()
> >in source4. That seems to be called in all the right places
> >and suggests it should do the right thing (currently it
> >does nothing).
> >
> 
>