An interesting shutdown race in messaging

[jim]

Perhaps with a reasonable timeout in case one of the send targets is 
unresponsive so shutdown/exit is not blocked for an excessive time.

On 6/13/2017 12:30 PM, Jeremy Allison via samba-technical wrote:
> On Tue, Jun 13, 2017 at 09:15:33AM -0700, Jeremy Allison via samba-technical wrote:
>> On Tue, Jun 13, 2017 at 05:14:01PM +1200, Andrew Bartlett wrote:
>>> G'Day,
>>>
>>> I have another messaging issue that has come up.
>>>
>>> In my attempts to make progress on the ldb locking code, I tried moving
>>> all of the servers in 'make test' to the standard process model.
>>>
>>> However, this hits an issue in the auth_log tests, which use messaging
>>> to contact an external python process, the tests, which expect to be
>>> sent the audit messages.
>>>
>>> The issue is that in -M standard and in smbd, the connection can be
>>> closed so fast that the messages that are written to the logs, and have
>>> presumably been dispached to a messaging thread, never make it to the
>>> other end.
>>>
>>> I can work around it by holding the connection open in some of the
>>> tests, but I can't easily do that for the tests that call rpcclient
>>> over subprocess, or where the password fails on LDAP.
>>>
>>> So, I wonder if there is a way to, before we call exit(0) in the
>>> standard process model, have it try again to dispatch the messages to
>>> somewhere safely on the other side of a connected or datagram fd?
>>>
>>> This can be seen in the auth_log tests in
>>> https://urldefense.proofpoint.com/v2/url?u=http-3A__git.catalyst.net.nz_gitweb-3Fp-3Dsamba.git-3Ba-3Dshortlog-3Bh-3Drefs_heads_ldb-2Dsafe-2Dlocking-2Dfor-2Dmaster&d=DwIBAg&c=y2w-uYmhgFWijp_IQN0DhA&r=Ru1enaR19oK8d7j6CTyYvKyHRw2ZzjrSu2PJyn0fKAQ&m=dATI2CzoCWdCutLO-aBWR9gkhLDCUKpGFxwAS2Bk2-0&s=ELYWZqACBhU2KcpkDMKzhdVfbg5C8i_YjDoA3Ct-xwU&e=
>>>
>>> The commit that causes the issue is
>>> https://urldefense.proofpoint.com/v2/url?u=http-3A__git.catalyst.net.nz_gitweb-3Fp-3Dsamba.git-3Ba-3Dcommitdiff-3Bh-3D6cf5e9f9b9f27af56a5a42ea6d7259f06f35240c&d=DwIBAg&c=y2w-uYmhgFWijp_IQN0DhA&r=Ru1enaR19oK8d7j6CTyYvKyHRw2ZzjrSu2PJyn0fKAQ&m=dATI2CzoCWdCutLO-aBWR9gkhLDCUKpGFxwAS2Bk2-0&s=eT09MGnbSidl-oeDf0uzHgh5pBqzPTEHT_7GLyYm3u0&e=
>>>
>>> (changing all "single" to "standard" in Samba4.pm).
>>>
>>> Fixing this would allow folks to use the messaging transport for the
>>> auth audit messages in a reliable way (instead of reading the logs),
>>> which isn't currently promised (only listed as a developer feature
>>> right now), but could be quite handy.
>>>
>>> Any clues most appreciated,
>>>
>>> (Jeremy CC'ed because I mentioned this to him a long while ago).
>> So what you need is a 'drain_sent_messages' kind of function,
>> that will block until all pending messages are flushed out of
>> the system or the receiver disconnects, or an error send gets
>> and error - yes ?
> Hmmm. I think this could go inside of imessaging_cleanup()
> in source4. That seems to be called in all the right places
> and suggests it should do the right thing (currently it
> does nothing).
>