ctdb client python bindings

[Amitay Isaacs]

Hi Andrew,

On Fri, Apr 29, 2022 at 1:04 AM Andrew Walker <awalker at ixsystems.com> wrote:
> On Thu, Apr 28, 2022 at 4:14 AM Amitay Isaacs <amitay at gmail.com> wrote:
>>
>> I appreciate the efforts to implement python bindings for ctdb client
>> interfaces.  However, I fail to understand the motivation behind this
>> work.  Is there a requirement from some applications to have a python
>> interface to CTDB?  Or do you have some other plans?
>
>
> Well, I was working on this because our own (truenas) has python-based
> middleware and I was wanting to be able to get ctdb status info without
> having to launch subprocesses. I was also planning to write python-based
> collectd plugin to gather stats from ctdb at configurable intervals.

Thanks for describing the motivation for python bindings for CTDB client API.

>> In the past, Martin and I had considered developing python bindings
>> for client interfaces.  The motivation there was to rewrite the ctdb
>> tool in python. However, we never got around to doing that.
>
>
> That's a good idea. I could go that route, which would reduce code
> duplication. Basically keep existing behaviors and arguments for ctdb tool,
> but have it be python tool. Then it will probably not increase maintenance
> load.

Before you commit to the idea of rewriting the ctdb tool in python,
there are few things that need some consideration (Martin might have
more things to add.)

For the last few years, Martin and I have been discussing the
monolithic ctdb daemon into separate daemons based on gross
functionality.  These include database, cluster, failover, event etc.
Unfortunately we have not been able to make much progress on that
front in recent years.  That does not mean we have given up on the
idea, it's still in the works.  Whenever that happens, obviously the
python bindings need to be modified accordingly.  This has
implications on the ctdb tool also.  We would like to group the ctdb
functions as per the gross functionality.  This means restructuring
the ctdb commands in the style of "ctdb event" and subcommands, rather
than top-level commands.  This breakup of ctdb tool functionality is
likely to happen sooner than splitting of the daemon code.  We are
also thinking of transforming the ctdb tool into a ctdb shell with
readline.

I am sure it will be possible to adapt all the features with python
implementation.  One thing I haven't yet figured out is how to run
tevent event loop along with the python main loop.  If we decide to
rewrite the ctdb tool in python, then it's essential to maintain the
async event handling in python. I would like to get rid of the
synchronous api layer completely.  Especially when we have multiple
daemons and their client APIs, this avoids the extra burden of
developing a synchronous API layer just for the client tool.

In the short run, we can definitely add python bindings that are
read-only (e.g. getting status or querying statistics).  It's much
easier to write unit tests for those APIs using fake ctdbd.  For any
APIs that modify the running of the daemon, I would prefer to first
figure out how to handle async (tevent) events in python.

Amitay.