[Samba] Clustered Samba 3.6.6 connection issues

[Adi Kriegisch]

Dear Samba Community,

(answering my own request) 
> we recently did upgrade our data server cluster from Debian Squeeze (Samba
> 3.5.6) to Debian Wheezy (Samba 3.6.6).
> The cluster is configured to act as BDC too. After the upgrade, connecting
> to the server works for a short while and then users experience disconnects
[...]
>   | This computer was not able to set up a secure session with a domain
>   | controller in domain DOMAIN due to the following:
>   | The RPC server is unavailable.
[...]
> A test cluster showed that with the very same config files, Samba 3.6.6
> works just fine in a cluster when not being a BDC (domain logons = no and
> security = domain).
We made a huge mistake here: During the upgrade process we shut down ctdb
on the two servers that were upgraded and left one running and serving the
cluster.
After we completed the upgrade, we stopped ctdb on the remaining node and
started it on the freshly upgraded nodes.
That works just fine, but the disabled node still has all 3 cluster ips
configured and sometimes wins the arp response race which leads to
connection requests going to the wrong server and leading to the behavior
we've seen here.
Whenever you plan to do the same (ie. still have a fallback configuration
at least on one node), make sure to manually remove the ip addresses.

-- Adi