ntdb in Samba?

[Michael Adam]

Very cool benchmarking, Ralph!!

It is especially interesting to note rec/s rate
stay almost constand in number of procs with
mutex locking, and drop logarithmically for
fcntl locking (ntdb in that case).

Michael


On 2015-03-12 at 17:52 +0100, Ralph Böhme wrote:
> Hi
> 
> On Thu, Mar 12, 2015 at 04:28:21PM +0100, Volker Lendecke wrote:
> > On Wed, Mar 11, 2015 at 12:25:40PM +0100, Volker Lendecke wrote:
> > > On Wed, Mar 11, 2015 at 10:52:55AM +1030, Rusty Russell wrote:
> > > > > ntdb gives quite some warnings/errors in the Coverity scan I've recently
> > > > > started looking at again.
> > > > >
> > > > > Is it worthwhile fixing those, i.e. can we expect development in ntdb
> > > > > any time soon?
> > > > 
> > > > I think it's dead.
> > > >
> > > > Originally it was a replacement which handle tdb too, with a new API.
> > > > But then the Samba Team consensus was that Samba already had a dbwrap
> > > > layer which could handle both, so we should use that instead.
> > > 
> > > Yes, the history around ntdb was not really great. Sorry for that.
> > > 
> > > Question to the broader audience: Do we have any current users of ntdb? Or
> > > can we just remove it for 4.3?
> > 
> > If we decide it can go, there is some cleanup potential:
> > 
> >  406 files changed, 234 insertions(+), 49030 deletions(-)
> > 
> > See
> > 
> > https://git.samba.org/?p=vl/samba.git/.git;a=shortlog;h=refs/heads/wip
> 
> +1.
> 
> Some numbers: tdb (with mutexes and volatile/dead-records) is faster
> across all variants, only showing number of processes variation here.
> 
> 2 processes:
> 
> $ ./bin/tbench -m -b ripple -n 5 -b ripple -r 500000 -h 65536 -d tdb
> Testing with 5 processes, recs: 100000, LFR: 10000, ripple recs: 500000
> 5 processes rippled 500000 records in 4.251 seconds (117618 rec/s)
> 
> $ ./bin/tbench -b ripple -n 5 -b ripple -r 500000 -d ntdb
> Testing with 5 processes, recs: 100000, LFR: 10000, ripple recs: 500000
> 5 processes rippled 500000 records in 6.175 seconds (80977 rec/s)
> 
> That's the "ripple" benchmark which preloads a database with some
> number of random records (100k in this case), then deletes a certain
> percentage of those records (default 10%) and finally starts the
> ripple benchmark which consists of a loop adding a random record and
> randomly deleting another record n times.
> 
> 50 proceses:
> 
> $ ./bin/tbench -m -b ripple -n 50 -b ripple -r 500000 -h 65536 -d tdb
> Testing with 50 processes, recs: 100000, LFR: 10000, ripple recs: 500000
> 50 processes rippled 500000 records in 5.089 seconds (98255 rec/s)
> 
> $ ./bin/tbench -b ripple -n 50 -b ripple -r 500000 -d ntdb
> Testing with 50 processes, recs: 100000, LFR: 10000, ripple recs: 500000
> 50 processes rippled 500000 records in 9.465 seconds (52828 rec/s)
> 
> 500 processes:
> 
> $ ./bin/tbench -m -b ripple -n 500 -b ripple -r 500000 -h 65536 -d tdb
> Testing with 500 processes, recs: 100000, LFR: 10000, ripple recs: 500000
> 500 processes rippled 500000 records in 5.088 seconds (98271 rec/s)
> 
> $ ./bin/tbench -b ripple -n 500 -b ripple -r 500000 -d ntdb
> Testing with 500 processes, recs: 100000, LFR: 10000, ripple recs: 500000
> 500 processes rippled 500000 records in 21.665 seconds (23079 rec/s)
> 
> -Ralph
> 
> tbench: <https://git.samba.org/?p=slow/samba.git;a=shortlog;h=refs/heads/tbench>
> 
> -- 
> SerNet GmbH, Bahnhofsallee 1b, 37081 Göttingen
> phone: +49-551-370000-0, fax: +49-551-370000-9
> AG Göttingen, HRB 2816, GF: Dr. Johannes Loxen
> http://www.sernet.de,mailto:kontakt@sernet.de
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: not available
URL: <http://lists.samba.org/pipermail/samba-technical/attachments/20150312/495d18df/attachment.pgp>