ctdb/tcpip: duplicate connections?

[tridge at samba.org]

 > My questions:
 > - is it intended?
 > - will it change in the future?
 > - wouldn't be better to answer to a request on the same connection (like in a 
 > normal client-server scenario)?

yes, it's intended. It uses 2(N-1) sockets per node instead of (N-1),
but running out of file descriptors isn't a worry.

The reason I did it is that for the tcp backend is makes it
considerably easier. That's because it's not client-server, its
symmetric. So when node A wants to talk to node B and there isn't
currently as established connection then node A can initiate a
connection itself, and immediately get a failure (like connection
refused) if appropriate. Otherwise node A would have to wait for node
B to open a connection to node A, if we decided that we wanted the
connection to go in that direction.

We could avoid this fairly easily by having logic that used an
existing incoming connection for outgoing data is there was one, but
to do that requires a few extra things:

 1) during initial startup there will be races, which will lead to
    some nodes having 2 connections to another node, and some having 1

 2) we can't rely on the tcp peername for the connection to determine
    which node is at the other end (think about multi-home, and
    NAT). So we'd need to wait until not just a connection was
    established, but until the other end sent an identifying message

So we could cope, but by using one more socket, the code stays very
simple.

For IB, you may find it simpler to have one connection. That's totally
up to you. The backend/frontend split doesn't force it one way or
another.

Cheers, Tridge