ephemeral ports and port re-use for socket_wrapper emulated TCP bind()

[Andrew Bartlett]

On Thu, 2016-10-20 at 08:54 +0200, Andreas Schneider wrote:
> On Thursday, 20 October 2016 12:08:55 CEST Andrew Bartlett wrote:
> > 
> > I've found an interesting issue while working our Samba's RPC
> > server.
> > 
> > I wanted to change Samba from listening only on 1024 to listening
> > on
> > many ports for ncacn_ip_tcp so that I could multi-thread our
> > netlogon
> > server.  
> > 
> > The most practical way to do that is to have it listen on a
> > distinct
> > socket. 
> > 
> > However, because swrap_bind has the (new, at that point)
> > check_addr_port_in_use() function removed with an #if 0 by commit
> > 064592d9cb6349e625b881cfcfab37b19d141ebe, no check is made to see
> > if a
> > socket is available.
> 
> This is code is commented out because
> 
> a) it is incomplete
> b) it doesn't have tests
> 
> The way to fix this is to start working on a) and b). Well, implement
> b) first 
> ... :)
> 
> However this is limited to one process, the process socket_wrapper
> is 
> preloaded. It will be possible to check that as soon as
> socket_wrapper has a 
> database. This is the work being done do make it thread safe and
> support fd-
> passing.
> 
> https://git.samba.org/?p=obnox/cwrap/socket_wrapper.git;a=shortlog;h=
> refs/
> heads/fd-passing-threadsafe

Thanks for the pointer. 

> > 
> > 
> > This means we just unilaterally unlink() the socket.  
> > 
> > This broke this loop in stream_setup_socket() where we (yes,
> > really...)
> > loop from 1024 up, binding on each port trying to get a free port!
> > 
> > https://git.samba.org/?p=samba.git;a=blob;f=source4/smbd/service_st
> > ream
> > .c;h=f0a379acf6a68be6f9296b0af51ecb1507dbfb5f;hb=HEAD#l334
> > 
> > In my patched case that put each ncacn_ip_tcp services on a new
> > endpoint (binding), it meant that we had 10 difference services
> > listening on 1024, the last of which was the only one actually able
> > to
> > be contacted! :-)
> > 
> > So, given the limitations of unix domain sockets (it seems
> > impossible
> > to determine without making a connect() if the socket is currently
> > bound by someone else), it seems the options are to:
> > 
> >  1) specify the port in a config file
> >  2) change the code to use 0 for a real ephemeral port and use the
> > code
> > in swrap_auto_bind() to produce it
> >  3) add some kind of lock file or (non-portably) use an fcntl()
> > lock on
> > the socket to determine if it is bound. 
> > 
> > I'm thinking to just do option 1 (with the advantage that many of
> > our
> > users would like to be able to control these ports), but in the
> > meantime do you have any other suggestions?
> 
> Why don't you want to improve socket_wrapper?

Mostly because of the reasons you mentioned, which were clear in the
disabled code: that without a database it would just work per process. 

I hasn't considered a whole database, which is certainly an improvement
over my option 3.

Sadly I don't think I'll have enough budget to divert into major
socket_wrapper patching, but neither did I want to go ahead without
checking with you on the current status. 

Thanks!

Andrew Bartlett

-- 
Andrew Bartlett                       http://samba.org/~abartlet/
Authentication Developer, Samba Team  http://samba.org
Samba Developer, Catalyst IT          http://catalyst.net.nz/services/samba