[clug] multiple network ports on one machine

[Daniel Pittman]

steve jenkin <sjenkin at canb.auug.org.au> writes:
> Adrian wrote on 31/3/09 2:00 PM:
>> More detail:
>>
>> A special purpose machine , a correlator, has 24 cards cramed packed
>> with fast Xilinx FPGAs. At the end of a 30 second period each card in
>> turn will dump the contents of its buffers onto the network. A total of
>> 23, 721, 984 Kbps  or per board (24 boards) =  988, 416 Kbps = 0.989
>> Gbps. With packetisation and other overheads this will exceed the
>> capacity of a single Gigabit port. Fortunately there are 4 Ethernet
>> ports per card, so I can use 2 at the same time. Via a network switch
>> this is feed into a computer(s) which has 4 gigabit Ethernet ports. So
>> there are two streams of data to be written to disk(s). Each port on the
>> FPGA card can address a specific destination Ethernet address on the
>> computer. The data will be sent as UDP packets. If I start 2 UDP servers
>> will each be associated with the different Ethernet address (physical
>> ports)? Is the solution to use different port numbers, i.e. bind IP
>> address x.x.x.y:aaaa and bind x.x.x.z:bbbb.
>>
>> I know that the next bottleneck will be writing to disk, but I will
>> consider that later.
>>
>> This data dumping occurs every 30 sec for 12 hours ..... lot of data.
>
>
>
> Please excuse me if this is a really dumb question/proposal...
>
> Doesn't Linux support Virtual IP's for redundancy - two machines each
> with an IP, and between the two of them, a third 'virtual IP' (and
> MAC) is constructed?

Yes.  In so far as any system with an IP stack can do that.

> Can that be adapted to provide a type of 'link bonding'?

No.

> Why couldn't that same software be run on one machine with multiple
> network interfaces??

The "virtual" IP stuff is just an efficient way to move a single IP
address between a single system.


Alternately, you might be thinking of something like anycast, which
isn't going to help in this situation unless you have routers, not
switches, between the collection and reception machines.

> Normally there's no sense in attempting redundancy like this...
>
> If the code is non-trivial to adapt, perhaps a low-footprint VM method
> (like openSolaris 'containers' or BSD 'jail') would provide enough
> scaffolding to test...
>
> The sender points its UDP packets at the Virtual IP addrs, routed via
> the real IP's.

This is indistinguishable from using multiple real IP addresses, and
will have exactly the same set of problems.  You just made it more
complicated by also adding extra layers of virtual stuff on top. :)

[...]

> The nub of the solution:
>
> Does the host O/S pick the next link (via the metric) if the primary is
> saturated?

No.

> This is UDP, not a single TCP session, so without have done it or
> tested it, my expectation is that part would work...

Nope, sorry.

> Anyone worked in these areas and know any answers for sure??

Yup.  I can't claim to know everything there is to know about
networking, IP or the Linux networking stack, but I know enough to
answer these questions quite confidently.

To tell the truth, I really hope that everyone who intends to give
answers — rather than, like you, asks questions — is in that same
position.

People asking questions when they are unsure is a great thing, and it
helps everyone learn.  People giving answers when they don't actually
know is ... exciting, but generally not helpful.

Regards,
        Daniel