Rev 40: Raw impl. of ibwrapper test tool. in http://samba.org/~tridge/psomogyi/

[tridge at samba.org]

Peter,

 > Tridge, can I have a max for message size?

I think your fixed size buffer is analogous to the tcp socket
buffer. You should be able to keep it as a fixed size while still
handling an unlimited number of messages for send.

If you look at ctdb_tcp_queue_pkt() it should give you some ideas. It
works by first trying to put the packet in the socket buffer (by
writing to a non-blocking socket), and if it won't fit, then it puts
the packet into a software queue and asks the event layer to tell us
when the socket buffer is no longer full (indicated in this case by
the socket becoming writeable).

So in your ib backend, I expect you will do much the same thing. You
will have a fixed size ib/verbs level queue, but if when you are asked
to queue a packet it doesn't fit then you will link the packet into a
linked list and wait for an event to tell you that there is more room
available in the ib level queue.

I expect that actually filling the outgoing queue will be very rare,
and I suspect that a queue size of just a few packets of a few kb each
will be all that is required, but I think the backend should correctly
handle the case of lots of packets.

As for the maximum size of an individual messages, I think we might
end up making this settable when ctdb is initialised. For now, pick
some arbitrary number (maybe 64k?) and if we ever get close to that we
can make it settable.

Cheers, Tridge