ntdb in Samba?

[Andrew Bartlett]

On Wed, 2015-03-11 at 10:52 +1030, Rusty Russell wrote:
> Volker Lendecke <Volker.Lendecke at SerNet.DE> writes:
> > Hi, Rusty!
> >
> > ntdb gives quite some warnings/errors in the Coverity scan I've recently
> > started looking at again.
> >
> > Is it worthwhile fixing those, i.e. can we expect development in ntdb
> > any time soon?
> 
> I think it's dead.
> 
> Originally it was a replacement which handle tdb too, with a new API.
> But then the Samba Team consensus was that Samba already had a dbwrap
> layer which could handle both, so we should use that instead.
> 
> So I threw away that (already-completed) NTDB work, and then discovered
> that dbwrap in Samba didn't cover many uses of TDB.  So, now my job was
> to rework some of the oldest and crustiest parts of Samba.  Plus enhance
> dbwrap, since it wasn't a complete abstraction.

I think this really was a miss-step on our part, and I really wish we
had instead found a way to use TDB2 on TDB1 files, as a first step, and
then TDB2.  The issue around APIs and contracts could have been solved
with feature flags, where strictly required.

I'm thinking hard about the lessons here when we look at a similar
transition with the efforts of Petr to allow us to move to python3 -
some changes, like tdb2 or python3 are going to be large, and disruptive
in terms of code, but asking folks to boil the ocean in order to make
their contribution isn't helpful either. 

> Meanwhile, nobody actually needs > 4GB tdbs.  And the hacky garbage
> collection tdb uses is sufficient in practice.  And there's been
> excellent scalability work done in TDB itself, which hasn't been
> mirrored in ntdb.
>
> And while the ntdb API is much nicer than the tdb API, that's not
> very helpful since Samba needs to support both.  And since ntdb can
> no longer handle tdb files, it's not useful for non-Samba projects
> unless they're completely new code.
> 
> And to be honest, NTDB is technologically an incremental, not
> revolutionary enhancement.  Without the benefit of years of testing
> which TDB has.

I've wondered if you think it might help in ldb?  I've not done the
performance work to understand what (if anything) is 'slow', but have
pondered if backing on ntdb or Howard's lmdb would be worthwhile.

Andrew Bartlett

-- 
Andrew Bartlett                       http://samba.org/~abartlet/
Authentication Developer, Samba Team  http://samba.org
Samba Developer, Catalyst IT          http://catalyst.net.nz/services/samba