ntdb in Samba?

[Ralph Böhme]

Hi

On Thu, Mar 12, 2015 at 04:28:21PM +0100, Volker Lendecke wrote:
> On Wed, Mar 11, 2015 at 12:25:40PM +0100, Volker Lendecke wrote:
> > On Wed, Mar 11, 2015 at 10:52:55AM +1030, Rusty Russell wrote:
> > > > ntdb gives quite some warnings/errors in the Coverity scan I've recently
> > > > started looking at again.
> > > >
> > > > Is it worthwhile fixing those, i.e. can we expect development in ntdb
> > > > any time soon?
> > > 
> > > I think it's dead.
> > >
> > > Originally it was a replacement which handle tdb too, with a new API.
> > > But then the Samba Team consensus was that Samba already had a dbwrap
> > > layer which could handle both, so we should use that instead.
> > 
> > Yes, the history around ntdb was not really great. Sorry for that.
> > 
> > Question to the broader audience: Do we have any current users of ntdb? Or
> > can we just remove it for 4.3?
> 
> If we decide it can go, there is some cleanup potential:
> 
>  406 files changed, 234 insertions(+), 49030 deletions(-)
> 
> See
> 
> https://git.samba.org/?p=vl/samba.git/.git;a=shortlog;h=refs/heads/wip

+1.

Some numbers: tdb (with mutexes and volatile/dead-records) is faster
across all variants, only showing number of processes variation here.

2 processes:

$ ./bin/tbench -m -b ripple -n 5 -b ripple -r 500000 -h 65536 -d tdb
Testing with 5 processes, recs: 100000, LFR: 10000, ripple recs: 500000
5 processes rippled 500000 records in 4.251 seconds (117618 rec/s)

$ ./bin/tbench -b ripple -n 5 -b ripple -r 500000 -d ntdb
Testing with 5 processes, recs: 100000, LFR: 10000, ripple recs: 500000
5 processes rippled 500000 records in 6.175 seconds (80977 rec/s)

That's the "ripple" benchmark which preloads a database with some
number of random records (100k in this case), then deletes a certain
percentage of those records (default 10%) and finally starts the
ripple benchmark which consists of a loop adding a random record and
randomly deleting another record n times.

50 proceses:

$ ./bin/tbench -m -b ripple -n 50 -b ripple -r 500000 -h 65536 -d tdb
Testing with 50 processes, recs: 100000, LFR: 10000, ripple recs: 500000
50 processes rippled 500000 records in 5.089 seconds (98255 rec/s)

$ ./bin/tbench -b ripple -n 50 -b ripple -r 500000 -d ntdb
Testing with 50 processes, recs: 100000, LFR: 10000, ripple recs: 500000
50 processes rippled 500000 records in 9.465 seconds (52828 rec/s)

500 processes:

$ ./bin/tbench -m -b ripple -n 500 -b ripple -r 500000 -h 65536 -d tdb
Testing with 500 processes, recs: 100000, LFR: 10000, ripple recs: 500000
500 processes rippled 500000 records in 5.088 seconds (98271 rec/s)

$ ./bin/tbench -b ripple -n 500 -b ripple -r 500000 -d ntdb
Testing with 500 processes, recs: 100000, LFR: 10000, ripple recs: 500000
500 processes rippled 500000 records in 21.665 seconds (23079 rec/s)

-Ralph

tbench: <https://git.samba.org/?p=slow/samba.git;a=shortlog;h=refs/heads/tbench>

-- 
SerNet GmbH, Bahnhofsallee 1b, 37081 Göttingen
phone: +49-551-370000-0, fax: +49-551-370000-9
AG Göttingen, HRB 2816, GF: Dr. Johannes Loxen
http://www.sernet.de,mailto:kontakt@sernet.de