ctdb with Infiniband

[ronnie sahlberg]

Hi

Yes,
--enable-infiniband   means trying to use IB Verbs as a raw transport.
If  you run IPoverIB   then this is handled by ctdb as normal TCP transport
and you dont need --enable-infiniband.

regards
ronnie sahlberg

On Fri, Jul 30, 2010 at 1:13 PM, Shuichi Ihara <ihashu at gmail.com> wrote:
> Ronnie
>
> Thanks for response.
> I understood no much need for Infiniband for CTDB to get performance,
> I don't much care about the
> performance, but we need to use Infiniband (TCP over IB) as "a network
> platform" for TCP/IP.
>
> If I am misunderstanding, please let me know. --enable-infiniband
> means for Cluster Interlink of CTDB?
> I was tying to ib0 interface one of TCP/IP network interface for
> public network of CTDB, but was getting
> PARTIALLYONLINE and couldn't become OK on 'ctdb status'.
>
> Thanks
> Ihara
>
> 2010/7/29 ronnie sahlberg <ronniesahlberg at gmail.com>:
>> Hi,
>>
>> The bad news is that the infiniband support has experienced bit-rot
>> and is not maintained. So you can not use it. You have to use TCP
>> today.
>>
>>
>> The good news is that we managed to get the ctdb protocol reasonably
>> fast and non-chatty   so there is no real need for the low-latency
>> infiniband transport for the inter-ctdb communications.
>>
>>
>>
>> Basically, the ctdb protocol is fast enough with GbE (or 10GbE) and
>> TCP that the cifs serving bottlenecks lies elsewhere.
>>
>>
>>
>> regards
>> ronnie sahlberg
>>
>>
>>
>> On Thu, Jul 29, 2010 at 12:09 PM, Shuichi Ihara <ihashu at gmail.com> wrote:
>>> Hi,
>>>
>>> I'm building ctdb (1.0.114) with --enable-infiniband option, but
>>> getting the following errors.
>>>
>>> Compiling utils/ping_pong/ping_pong.c
>>> Linking bin/ping_pong
>>> Linking bin/ctdbd
>>> ib/ibw_ctdb_init.o: In function `ctdb_flush_cn_queue':
>>> /root/ctdb/ib/ibw_ctdb_init.c:140: undefined reference to `DLIST_REMOVE'
>>> ib/ibw_ctdb_init.o: In function `ctdb_ibw_queue_pkt':
>>> /root/ctdb/ib/ibw_ctdb_init.c:174: undefined reference to `DLIST_ADD_AFTER'
>>> collect2: ld returned 1 exit status
>>> make: *** [bin/ctdbd] Error 1
>>>
>>> I've tested with OFED-1.4.x and OFED-1.5.x for Infiniband stuff, but
>>> got a same error.
>>> Any idea to fix this build issue?
>>>
>>> Thanks
>>> Ihara
>>>
>>
>