using virtual synchrony for CTDB

tridge at tridge at
Fri Oct 6 22:56:29 GMT 2006


 > In raw throughput terms, vs can deliver approx 40k messages per second
 > and does indeed saturate a GIGe network, whereas (over a switched
 > network) point to point can handle 1600 exchanges per second between two
 > nodes.  This scales to multiples of 1600 between multiple nodes
 > depending on switch bandwidth availability.

Those numbers don't look good I'm afraid. Between two nodes I measured
31000 messages/sec on gigabit (both with TCP and UDP). With some
fancier hardware and MPI we measured around 170000 messages/sec with

Maybe you could try the test code at on your nodes? I'd
be curious how close you get to the 31000 we measured.

To use the tcp2 example, compile on two hosts and run:

on host1:  ./tcp2 host2 2000 2001
on host2:  ./tcp2 host1 2001 2002

it's very primitive code, but is useful for basic throughput on tcp
and udp. Also does unix domain sockets.

Note that I get about 17000 messages/sec on my 100 MBit switch at home
between my laptop and a server box. That uses a 1 byte payload. With a
64 byte payload I get about 14000 messages/sec on 100 MBit ethernet.

So 1600 exchanges/sec (presumably 3200 messages/sec for coomparison
with my test) is very slow for gigabit.

Cheers, Tridge

More information about the samba-technical mailing list