[PATCH 00/14] tdb: Update pytdb API to match what is provided by libtdb

[Kirill Smelkov]

Hi Jelmer,

On Sun, Sep 19, 2010 at 10:16:18AM -0700, Jelmer Vernooij wrote:
> Hi Kirill,
> 
> On Sun, 2010-09-19 at 13:53 +0400, Kirill Smelkov wrote: 
> > Rusty, Jelmer,
> > 
> > The subject says it all. Not 100% complete, but near.
> Thanks for the patches. I've applied most of the Python ones. I'm not at
> all convinced we should match the C API in the Python API though, I
> rather think we should let the needs of our Python users drive what we
> expose. Some of the worst Python bindings I've seen were created by
> simply mapping every C function one on one to Python.
> 
> Is there any particular reason why some of these functions should be
> exposed? Why do you need low-level locking?

Thanks for applying some patches and sorry I've not described my context
initially...

In this case I myself is tdb python user - I use tdb in embedded system
for internal database to which many programms "connect" simultaneously
to read/write it.

That's why I need locking, and better, to avoid lock contention, the
chainlock_* family variants.

Also, sometimes it is not important to write data to db immediately, so
to minize latencies, apps keep to-be-written queue internally until they
know they can write to some chain, or start transaction - that's why I
need *_nonblock variants.

Same for reading - once initially read, it's not that important to get
up-to-date values immediately, that's why I'd also use
tdb_chainlock_read_nonblock().

And to make life a bit more interesting, db is stored on compact flash
-- various types, from various vendors, so with various types of flash
translation layers (FTL) -- so inevitably with bugs in FTL with respect
to sudden power failures, so I'm preparing to have corrupt tdb one day 

http://ozlabs.org/~rusty/index.cgi/tech/2009-10-20.html
http://lwn.net/Articles/349970/

That's why I'd also like to have debugging routines (dump_all,
print_freelist, etc..,), and tdb_check (not yet done, should I?), and
also tdb_fd and tdb_repack come for completness (doesn't tdb_repack
complement tdb_wipe_all() which has python bindings?).

And we don't have shutdown sequence - normal shutdown is poweroff...


Hope this clarifies my rationale about why we should expose more
functionality in pytdb.

Thanks,
Kirill


On Sun, Sep 19, 2010 at 09:29:19AM -0700, Jelmer Vernooij wrote:
> Hi Kirill,
> 
> On Sun, 2010-09-19 at 13:53 +0400, Kirill Smelkov wrote:
> > Though when doing `make check-python` the dump is intermixed into
> > testing output :(
> I'm not convinced exposing these methods in the Python API is all that
> useful. They're mainly useful for debugging tdb itself. Is there
> anything in particular you'd like to use them for?

See above about corrupt tdbs and debugging.