NTDB progress!

[Andrew Bartlett]

Rusty,

In short, I'm starting to agree with others who suggest that this just
feels *way* too late.  Perhaps if we had been using tdb2 format in
Samba4 for a while, it wouldn't be an issue, but there hasn't been major
use of the db layer (only the API) anywhere, and we are getting way to
close to the release. 

On Thu, 2012-06-14 at 11:50 +0930, Rusty Russell wrote:

> I just did more stresstesting to prove ntdb was "safe", and found
> another bug.  Gah!  So I'm going to eat humble pie and suggest we
> proceed cautiously:
> 
> (1) Finish the dbwrap conversions (still using dbwrap_tdb as the
>     backend) for beta2.  That's what I'm debugging now, though as you
>     point out, s3/messaging and several s4 databases still need
>     conversion (I generally skipped CLEAR_IF_FIRST or INTERNAL dbs).
> 
> (2) Make dbwrap select ntdb or tdb depending on the filename handed to
>     it (ie. foo.ntdb or foo.tdb).  Make any INTERNAL dbs use a dummy
>     name if necessary (good practice anyway, since that name is good for
>     log messages).
> 
> (3) Have the ntdb version automatically upgrade tdb versions:
>         if (!file_exists(ntdbname) && file_exists(tdbname)) {
>                 ... traverse and convert ...
>                 ... rename tdbname to tdbname.bak ...
>                 fail to open ntdb if something goes wrong here.
>         }
> 
> (4) We can switch the callers based on an 'old tdb format' lpcfg.  This
>     can be done at any stage, even an individual db basis if we want.
> 
> (5) ldb files will be handled as planned (that code is already written):
>     new ldb files will be created as ntdb unless 'old tdb format' is
>     set.
> 
> I'm hoping to commit the dbwrap conversions today or tomorrow, depending
> when debugging finishes.
> 
> > I might also help out in converting things to dbwrap if that
> > would be useful. What do others think? For the s4-part, I would
> > also like to hear the opinion of (among others) Andrew and Metze.
> 
> That would be nice, thanks!  Want to do s3/messaging and s4?

Would we get this in faster if we just dropped the s4 and ldb parts, at
least initially?

When I first argued against just dropping this whole thing, it was
Easter, and exceptional circumstances meant that it couldn't all be
decided then.  

If we could live with TDB as is (I may have misunderstood the
conclusion of that discussion), and the 'walk over the whole tree' work
is the dbwrap conversion (rather than the tdb_compat API), would we do 
better to punt the actual use of ntdb until after 4.0?

We do still have other major changes to land - the SMB3 stuff is still
pending, but we don't have long, and it's mostly moving around upper
layers (the work at the guts of it included gensec back in
Jan/Feb/March). 

I guess I'm just feeling that we are doing in June what (particularly)
for our now-agreed timeline we really should also have been doing in
Jan/Feb/March.

I'm sorry to lower the boom over this major project of yours, but I hope
you understand.

Thanks,

Andrew Bartlett

-- 
Andrew Bartlett                                http://samba.org/~abartlet/
Authentication Developer, Samba Team           http://samba.org