multi-indexed tdb

[simo]

<Flame on>

I am not sure what you think, but it really seem that you just don't
trust people that work on samba4, therefore I will not try to convince
you, I will just point out what I find not agreeable, so that others can
evaluate themselves the facts. 

On Sat, 2006-08-12 at 20:03 +0200, Volker Lendecke wrote:
> On Sat, Aug 12, 2006 at 08:53:05AM +0200, tridge at samba.org wrote:
> > ok, so why separate it out at all? ldb is already designed to be built
> > standalone (see Makefile.in in lib/ldb).
> 
> Hmmm. Just tried it. Does not work, this seems to depend on
> other parts of Samba4.

As Tridge already pointed out, ldb depends only on talloc and tdb, if
you copy out the 3 directories ldb, talloc, tdb from /source/lib and
place them somewhere else, you will have a standalone ldb build, all you
have to do is to cd in ldb and do: ./autogen.sh; ./configure; make

There is indeed one bug that I have already fixed, a double declaration
of pread and pwrite in tdb_private.h and tdb/common/io.c, but that's not
a dependency on samba4 and was very easy to spot.

> > Why would you delete all the OIDs ? They don't do any harm. You would
> > grab all of lib/ldb/ as it is.
> 
> Sorry to be negative, but this would essentially mean a fork
> of ldb. Even tdb and talloc have gone into different
> directions. The diff between the respective Samba3 and
> Samba4 versions is considerable. For example I don't expect
> Samba4 to follow my attempts to make that compile with C++
> to see additional warnings.

Why don't you expect it? Have you ever asked if anybody was interested?
I am interested, and I am more than happy to make the ldb library
compatible with C++. 

> We just have different goals.

And which are these goals? Have you asked what are the goals of the
people working on samba4 or more specifically on ldb?

> And as there is quite a bit of development being done in
> Samba4 ldb right now and this from the start is a lot more
> complex code, I would expect both to diverge at a much
> quicker pace.
> 
> The only way to keep both together would be to really
> separate the stable ldb_tdb base API and "link" them into
> one directory. But this from my point of view is a
> considerable effort.

This has been proposed and agreed before, we just need to find some time
to do it. It has not been done yet for many reasons, one of which were
that samba3 was close to a new release and most of us were too busy
fixing bugs or doing something else urgent in short term.

> Or would the Samba4 developers be willing to separate out
> the non-modular ldb part? Is that even possible?

I will tell you if you explain me what dos it mean "separate out the
non-modular ldb part", I do not understand what it really means.

> > Regarding the merging pain, the ldb API that Samba3 would need hasn't
> > changed for quite a while, so the merge would consist of running
> > cp. It would only be painful if you tried to grab only part of ldb.
> 
> Who would be responsible if the build farm breaks after such
> a merge? Looking at the Samba3 vs Samba4 build farm results
> I get the impression that building on anything but Linux is
> not a priority for Samba4. I would not like to lose the
> other Unixes.

Who is responsible when something breaks in samba3?

Actually, for dome parts of samba4, compatibility has not been a
priority, this doesn't mean that we don't care about that at all,
if/when we will separate out talloc/tdb/ldb they will have their own
runs in the build farm anyway and they will be made to never break.
This will be a requirement anyways if samba3 has to use them, so I
really don't see the point of worrying about future imaginary doomsday
scenarios.

> Need to check that when I get it to compile at all. You
> might be right. But for example with ldb_ildap I fear that I
> have to import all of gensec to get it.

ldb_ildap is NOT compiled in the standalone build, because it depends on
samba4, and yes, this is already correctly detected so you do not need
to change anything. I suppose that in the samba3 build ldb_ldap (which
is the module you use in the standalone build) will even be disabled as
what you really want to use is the ldb_tdb backend, and I actually see
no use for ldb_ldap. If/When you will get more confident with the ldb
api and you will consider it for parts of samba3 that use ldap, then we
may consider using ldb_ldap, or even backporting the stuff needed to
build ldb_ildap.

> > well, development is certainly happening, but it is happening with a
> > stable base API (the API you would use), and with a quite decent test
> > suite so we know things don't break.
> 
> I went through a *LOT* of pain understanding and fixing the
> warnings all the archane machines gave me. I'm a bit
> reluctant compiling many modules that we never use and which
> just don't meet the coding standards we have in Samba3 these
> days. I don't have specific examples, but wading through the
> Samba4 build farm can't be called fun these days.
> 
> For example I would want to remove the use of talloc_steal
> from ldb, something that Samba4 will probably never follow.
> But in Samba3 we came to the conclusion that talloc_steal
> can very easily lead to complex code.

talloc_steal() will not go away from ldb, as it is part of the way ldb
has been thought from scratch, especially since we use a completely
async interface.
talloc_steal is dangerous only if you use it in code that is not
structured to use the new talloc from scratch and does not adhere to a
coding standard thought against memory hierarchies.
Ldb is now completely adherent to this way of making the code, and this
is enforced in it's internal interfaces.
But you will not see anything about this, as it is completely concealed
behind the simple API. You are exposed to it only if you use the
advanced API (namely the ldb_request() call), as it will force you to
use a talloc context for requests which are the memory container for
everything that happens inside ldb.

<flame off>

Simo.

-- 
Simo Sorce
Samba Team GPL Compliance Officer
email: idra at samba.org
http://samba.org