[PATCHES] Convert gencache to dbwrap to enable mutexes

[Michael Adam]

Hi Christof,

On 2014-11-04 at 15:43 -0700, Christof Schmitt wrote:
> On Wed, Jul 09, 2014 at 05:25:05PM +0200, Michael Adam wrote:
> > On 2014-07-08 at 15:14 -0700, Christof Schmitt wrote:
> > > On Wed, Jun 25, 2014 at 02:25:46PM +0200, Michael Adam wrote:
> > > > On 2014-06-25 at 14:19 +0200, Volker Lendecke wrote:
> > > > > On Wed, Jun 25, 2014 at 02:05:08PM +0200, Michael Adam wrote:
> > > > > > On 2014-06-25 at 13:35 +0200, Volker Lendecke wrote:
> > > > > > > On Wed, Jun 25, 2014 at 01:18:27PM +0200, Michael Adam wrote:
> > > > > > > > This would allow to use mutexes correctly
> > > > > > > > on gencache_notrans. In order to convert
> > > > > > > > it to dbwrap, we need to add a means of
> > > > > > > > using allrecord lock via dbrwap (it does
> > > > > > > > currently not offer it).
> > > > > > > 
> > > > > > > ... and I would try to avoid this. I'm not sure I want to
> > > > > > > think about implementing an allrecord lock for a ctdb
> > > > > > > database. I'd rather not go through dbwrap for
> > > > > > > gencache_notrans if that's the only reason to implement a
> > > > > > > dbwrap_allrecord_lock.
> > > > > > 
> > > > > > Right, but we could of course choose to *not* implement it
> > > > > > in the dbwrap_ctdb case...
> > > > > > 
> > > > > > Independently of the choice whether to do the allrecord
> > > > > > lock throught ctdb or not, do you think the above change
> > > > > > to gencache_stabilize (on the tdb level) would make sense?
> > > > > 
> > > > > Sure. But with your freelist patches, do we still need the
> > > > > wipe_all? This should be a nice test if freelist compaction
> > > > > actually works as designed :-)
> > > > 
> > > > Ok, sure!
> > > > But in that case I would do rather a second traverse of the
> > > > cache_notrans db and do tdb_delete there, in order not to
> > > > loose entries if the transaction on the persistent db fails.
> > > > Not as performant as using wipe_all probably, but still. :-)
> > > 
> > > Sorry, i got a bit lost in this discussion. Is the current proposal
> > > changing gencache_notrans to not use transactions, convert it to dbwrap
> > > and add the mutex check to the local dbwrap codepath?
> > > 
> > > I can take a stab at implementing that.
> > 
> > Sorry, I lost track myself.
> > I have a bunch of almost finished patches to the gencache
> > db code triggered by this discussion that I meant to post a
> > lot earlier. These remove the transaction from the notrans db.
> > These could serve as a foundation for your further work.
> > 
> > I hope to be able to post them asap, hopefully tonight or tomorrow,
> > if you still have a little bit of patience with me. :-)
> 
> I just found your wip patch to remove transactions from
> gencache_notrans:
> http://git.samba.org/?p=obnox/samba/samba-obnox.git;a=commitdiff;h=aa4fddf24f7fa8c2d4feba2adc70417c4f935ede

Ouch, thanks for digging that up again...

> Can we work towards getting that patch ready?

I just rebased and polished that branch:

https://git.samba.org/?p=obnox/samba/samba-obnox.git;a=shortlog;h=master-gencache

It builds and survives the cache tests.
Could you check if the patches look ok for you and do
a few more tests? Then we can move forward.

Note that instead of doing a tdb_wipe_all(),
I do a traverse, and delete only those records
that are not the "@LAST_STABILIZED" key and
that have a vaild timeout set.
I'm not 100% certain that this is the best way.
Maybe this can be solved more elegantly.

Cheers - Michael

> After that, we can either
> convert gencache_notrans to dbwrap, or add mutex support to tdb_wrap and
> convert gencache_notrans to tdb_wrap as suggested by Volker in
> https://lists.samba.org/archive/samba-technical/2014-September/102699.html
> 
> Christof
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature
URL: <http://lists.samba.org/pipermail/samba-technical/attachments/20141105/e4e654e4/attachment.pgp>