[clug] multiple network ports on one machine

[Daniel Pittman]

Adrian <adrian.blake at ieee.org> writes:
> Daniel Pittman wrote:
>> Adrian <adrian.blake at ieee.org> writes:
>>
>>> A special purpose machine , a correlator, has 24 cards cramed packed
>>> with fast Xilinx FPGAs. At the end of a 30 second period each card in
>>> turn will dump the contents of its buffers onto the network. A total
>>> of 23, 721, 984 Kbps or per board (24 boards) = 988, 416 Kbps = 0.989
>>> Gbps. With packetisation and other overheads this will exceed the
>>> capacity of a single Gigabit port.

[...]

>>> If I start 2 UDP servers will each be associated with the different
>>> Ethernet address (physical ports)?
>>
>> No.  UDP services are associated with *IP* addresses, not Ethernet
>> addresses.  There is no direct relationship.
>
> ipconfig will associate a physical port to an IP address, true?

Not on Linux, but on some systems.  Isn't it special?

A brief digression into history: there are two models for IP services,
the weak end system model and the strong end system model.

Anyway, in the weak end system model, which Linux implements, an IP
address is a property of the *system*, not the physical link.

In the strong model it is a link property, but that isn't relevant right
now as we are talking about Linux.


Anyway, so, Linux assigns the address to a link, but it isn't a property
of the link.  This is mostly for administrative convenience: it allows
sensible source address selection based on outgoing interface, defines
basic local routing, and so forth.

However, Linux will happily respond to the requested IP address on *ANY*
physical port, and can be convinced to route from any source address on
any physical port with appropriate configuration.


So, no, it isn't associated in the way you mean there, even if there is
some relationship between the two.


>> IP addresses have some relationship with Ethernet addresses, if your
>> computer has Ethernet cards, but the relationship is complicated,
>> especially if they are on the same subnet, or the same physical segment.
>>
>> How are your network ports on the server configured?
>> What switch hardware are you using?
>
> The present switch is not suitable for final testing, but I will
> purchase one that has 50 or more gigabit switching capability.

Ah.  I mostly meant "which managed switch OS are you using", so that you
could be pointed in the right direction for configuring link grouping on
that specific platform.

>> Anyway, when it comes time you want to either (a) use some sort of
>> bonding on the links that presents two cables as one logical link, or
>> (b) assign a distinct address to each cable.
>>
>> For case (a) you run a single UDP listener.  (More or less. :)
>>
>> For case (b) you listen to a different IP address for each stream.
>
> In the connectionless state ( if that is the right expression) there is
> no listen. So I do ????

Well, if you want to have two listeners on two addresses you will need
to use bind(2) on the sockets to assign them appropriate local
addresses.

If you try to use connectionless configuration then, no, you can't
configure which listener receives connections on which interface.

>>> Is the solution to use different port numbers, i.e. bind IP address
>>> x.x.x.y:aaaa and bind x.x.x.z:bbbb.
>>
>> Not as such, no, if the problem you are trying to solve is "how do I
>> get both cables active at the same time."
>>
>> To achieve that, bonding or careful network setup.  We can help with
>> either, but both are complicated and vendor specific.

[...]

>> For a dump of data every 30 seconds you have a *bursty* traffic
>> stream, with peaks and troughs of load.
>>
>> Are you sure that your equipment is going to handle that peak, not
>> just the average load, without dropping data?
>>
>> Heck, are you sure that your peak data rate is only going to be
>> 1Gbit, come to that?
>
> Yes I am sure of the peak rate. That can be controlled by the timing of
> the code in the FPGAs

Good.  I am glad that has been looked at; I have met plenty of people
who don't consider the difference between the two types of load.

Regards,
        Daniel