Unix domain datagram based messaging

[Jeremy Allison]

On Wed, Apr 09, 2014 at 10:49:31PM +0200, Volker Lendecke wrote:
> Hi!
> 
> Attached find a patchset that implements messaging for
> source3 based on unix domain datagram sockets. It does it a
> bit differently than the source4 system: The busy fallback
> is not done with a periodic retry but with a blocking
> thread. In my experiments this has turned out to be much
> friendlier in a complete overload situation, a thread
> blocking in sendmsg was the only way I could avoid a
> thundering herd, at least under linux.
> 
> With this, smbd leaks a few lockfiles and sockets, so
> this needs a bit more work, but the general approach seems
> to work fine for me.
> 
> This code has two advantages over the tdb-based one:
> 
> No signals
> 
> Significantly better performance (no array marshalling etc)
> 
> There's one little disadvantage: 
> 
> We need one more fd and an fcntl lock per smbd held open.
> The reason for this is that there is no atomic O_EXCL bind
> operation for unix domain sockets, so safely cleaning up for
> dead processes is not possible without an fcntl lock. The
> fcntl lock might hurt, but in normal operations we never
> contend on this, so it should be okay. And it's one per
> process, so not too much even with thousands of processes.
> We can turn this disadvantage into an advantage later on:
> serverid.tdb can go, this code saves the unique id in the
> lock file, so serverid_exists can work without the tdb file.
> 
> Comments (no formal review yet :-) would be appreciated!

Amazing work ! Thanks !!!!

Just one comment from a quick review.

It's removing most of our signal issues,
which is great ! However, at least within
smbd we still have to handle slow system
calls returning -1,EINTR if we get hit
by POSIX realtime signals from the leases
(of course due to this patchset this should
now be *extremely* rare :-)

So shouldn't:

recv() in unix_dgram_recv_handler()
connect() in unix_dgram_send_queue_init()
send() in unix_dgram_send_job()
sendmsg() in unix_dgram_send()

in this patchset be changed to handle -1,EINTR with
an immediate retry ?

Of course, if this is the case we also probably
need to fix the calls inside lib/addns/dnssock.c:

write_all() -> uses write()
dns_send_udp() -> uses sendto()
dns_receive_udp()  -> uses recv()

all without signal retry protection :-).

Let me know if I'm even vaguely correct here,
this is indeed tricky stuff :-).

Cheers,

	Jeremy.