[RFC] Strategies for CTDB integration into Samba

[Amitay Isaacs]

On Sat, Oct 25, 2014 at 12:23 AM, Michael Adam <obnox at samba.org> wrote:

> Hi,
>
> Triggered by another mail thread, would like to officially
> start the discussion or clarification, where we are heading
> with the integrated CTDB.
>
>
> What we have already done:
> ==========================
>
> - CTDB code was intergrated into Samba code, because
>   it is not (yet?) a project of its own.
>   For instance, parts of the client implementation are
>   in the samba code.
>
> - CTDB's build system has been converted to waf, and
>   some of the duplicated libraries have been removed.
>
> - As of recently, the top level samba build has been
>   extended to also build ctdb.
>
> - CTDB has no version number of its own any more but
>   will be released along with Samba as a component
>   in the future, starting with 4.2.
>
>
> What is going on:
> ================
>
> - Amitay has just (re)proposed patches that remove
>   the ability to build against older ctdb versions
>   and maybe with the intention to disable the ability
>   to build against external ctdb versions at all.
>   (I think not quite achieved.)
>
> - autobuild needs to be adapted.  Will post a patch.
>
> - Selftest needs to be augmented to run the samba-ctdb stack.
>   ==> I need to revive, finish and propose my
>   clustered-samba selftest branches.
>
>
If we manage to setup cluster as part of selftest, then we will be able to
run ctdb's cluster tests.

Few more things that Martin and I are working on.

 - Improve the ctdb protocol from current "structs on wire" approach to
proper protocol definition and automatic generation of serialization
routines.

 - To avoid changing the protocol in two implementations (source3
ctdbd_conn/dbwrap and ctdb client/server), abstract the protocol and
provide ctdb api for file server code.  This resurrects the idea of
libctdb, but focuses mainly on providing marshaling/unmarshaling routines,
rather that providing a complete API for ctdb.  Thanks to Volker for making
valuable suggestions to avoid memory allocations in the marshaling code.
The wip code is available in my ctdb-libctdb branch.

 - Split some of the ctdb functionality in separate daemons.  This requires
a messaging framework.  Adopt new datagram messaging in an attempt to unify
messaging across samba and ctdb daemons.


> The question is now: Where do we want to go?
> ============================================
>
> If we logcically pursue this path, I think the next steps
> would be:
>
> - to really disable the possibility to build
>   against an external ctdb.
>
> - maybe to make sure that samba is always build with ctdb.
>
> - to disable the standalone build of ctdb.


Before we disable standalone build of ctdb, I would like to make sure that
there is a way of building samba+ctdb rpms.  This might mean updating
packaging/RHEL-CTDB and making sure it works.  The main reason for building
RPMs is to be able to build samba/ctdb clusters for automated testing.  We
have set up a nightly autobuild which builds ctdb rpms for various
branches, creates virtual samba clusters and runs ctdb cluster tests.
Currently autobuild does not run ctdb clustered tests and I don't want to
lose the ability to do run clustered tests.


> This will make things a lot easier for our development
> and make it much more clear, also to the distributors
> and other consumers that ctdb is now considered a component
> of samba that is only shipped and released as such.
>
> We may need to guide the distributors and other consumers
> how to make the change.
>
> This will make it also much easier to continue the way some
> of us envision namely to integrate ctdb's clustering
> more and more into samba, maybe splitting various
> components out of ctdb, and removing the strict distinction
> between ctdb and samba in the long run...
>

I am currently not aware of any consumers of ctdb, other than samba.  If we
really want to support ctdb as a clustering extension to tdb, so other
applications can make use of it, then in the long run we need a proper
(thread-safe?) ctdb library.  It would also indicate that we need to
maintain the ability to run ctdb daemon(s) independently.


>
>
> Any comments or strong objections?
> Otherwise, I think we should go that way.
>
>
> Cheers - Michael
>
>
Amitay.