[Samba] Network issue with RH ES 3.0

[Jose Martinez]

Any help would be appreciated.

I am having problems with a network card on my system. Let me explain the
environment a bit.

RTPOPSSRV1 and RTPOPSSRV2 are the system names. Both running RedHat Linux ES
3.0.  Each have their own ip address

rtpopssrv1	9.44.50.33
rtpopssrv2	9.44.50.34

rtpopssrv2 is a failover for rtpopssrv1. Therefore there is a floating IP
that goes between them. We call that RTPOPSSRV, with ip address of
9.44.50.37.

For that floaing ip to work correctly, and since rtpopssrv1 is the primary,
we also float the mac address of the NIC card that is on rtpopssrv1 for the
9.44.50.37 ip.  It is the eth2 interface.  So when rtpopssrv2 has the
floating ip on it, the eth2 lan card on rtpopssrv2 is forced to have the MAC
address of the eth2 lan card on rtpopssrv1. We do this so that the
workstations that connect to it do not have ARP problems.  We do this on 2
other "clusters" with no problems.

Currently rtpopssrv2 has the floating ip and all application services are
running on that server.   Everything is working fine.  You can ping both
rtpopssrv2 and rtpopssrv and the workstations can get to it.  In addition,
the eth2 card on that server is running on the mac address of the eth2 card
from rtpopssrv1. So that is not a problem.

This past weekend, I brought down the eth2 interface on rtpopssrv2 and tried
to bring up the eth2 interface with the same network settings on rtpopssrv1.
Didn't work. I couldn't ping rtpopssrv which is the ip for the eth2 card.
Upon some troubleshooting, I realized that the system rtpopssrv1 wasn't
letting me run both eth1 and eth2 on that system (unlike rtpopssrv2 that was
ok).  I could have eth1 (rtpopssrv1) or eth2 (rtpopssrv) pinging but not
both at the same time.  Very strange.  I looked at the routing tables on the
rtpopssrv2 server when the floating ip was over there, and it was the same
as the routing table on rtpopssrv1 when the floating ip was on that server.

It was just very strange.  If I set up both eth1 and eth2 to startup on boot
time, the eth2 settings would completely override the eth1 settings, and
eth2 would become the "primary " card with the gateway interface pointing to
that card and not eth1.  This is the correct routing table as it is now on
rtpopssrv2.

[root at rtpopssrv2 etc]# netstat -rn
Kernel IP routing table
Destination     Gateway         Genmask         Flags   MSS Window  irtt
Iface
9.44.50.0       0.0.0.0         255.255.255.128 U         0 0          0
eth1
9.44.50.0       0.0.0.0         255.255.255.128 U         0 0          0
eth2
10.0.0.0        0.0.0.0         255.255.255.0   U         0 0          0
eth0
169.254.0.0     0.0.0.0         255.255.0.0     U         0 0          0
eth1
127.0.0.0       0.0.0.0         255.0.0.0       U         0 0          0 lo
0.0.0.0         9.44.50.1       0.0.0.0         UG        0 0          0
eth1


If you notice, the gateway 9.44.50.1 is pointing to eth1 which it should
since that is the eth1 card on the box.  When I tell both cards on
rtpopssrv1 to come up on startup... eveyrthing starts to point at eth2. I
can get to the floating ip, however I can't access the rtpopssrv1 ip unless
I'm on the 9.44.50.x subnet, which is no good.