[PATCH] lib: Fix gencache_del

[Ralph Boehme]

Moin!

On Sun, Mar 06, 2016 at 06:28:17PM +0100, Volker Lendecke wrote:
> On Sun, Mar 06, 2016 at 04:57:01PM +0100, Ralph Boehme wrote:
> > I did a quick prototype with gencache on top of your dbwrap_lmdb
> > patches and ran some random artificial torture based benchmarks.
> 
> Be aware that dbwrap_lmdb is far from finished. It might be
> possible to use it when my dbwrap_nolock lands. lmdb does
> not not have per-record locking, which we right now need.

Iirc we only need per-record locking when doing a traverse in order to
prevent deletion of the record from underneath us in the callback.

lmdb traverse handles (cursors) must be used with rw or ro transaction
which ensures the record is cow protected from deletion.

> If we get rid of fetch_locked keeping os-level locks, things might
> change.
>
> Also, Howard has included the patches to work with large
> lmdbs even for 32-bit architectures, lmdb can be compiled in
> a mode that does not always have to map in the complete db.

cool!

On Sun, Mar 06, 2016 at 07:48:49PM +0100, Volker Lendecke wrote:
> On Sun, Mar 06, 2016 at 04:57:01PM +0100, Ralph Boehme wrote:
> > maybe we should get rid of the double (or tripple with memcache) cache
> > layering and use lmdb for the gencache.
> 
> There's a reason for the double-caching: The stuff that's in
> gencache.tdb survives crashes even under writes. The
> saf_join cache might be among the more important ones, but I
> could imagine that other entries might also be valuable.

Safe, fast, simple. Choose two. :) The reason for this exercise to
determine whether moving the complexity layer from gencache to the
database engine layer preserves the safe and fast properties.

> With the current 2-db architecture we can write and get
> "eventual consistency" across crashes. I'm not sure lmdb
> without sync keeps the stuff that was last sync'ed always
> consistent, even under subsequent write traffic.

It's sync by default, so unless you specify MDB_NOSYNC when creating
the env the db will always be consistent. We should also pass
MDB_WRITEMAP and for things like gencache probably MDB_NOMETASYNC.

-Ralph

-- 
SerNet GmbH, Bahnhofsallee 1b, 37081 Göttingen
phone: +49-551-370000-0, fax: +49-551-370000-9
AG Göttingen, HRB 2816, GF: Dr. Johannes Loxen
http://www.sernet.de,mailto:kontakt@sernet.de