TDB: using the jenkins hash for non-persistent tdbs

[Rusty Russell]

On Tue, 21 Sep 2010 11:51:16 am Rusty Russell wrote:
> On Mon, 20 Sep 2010 07:11:08 pm Stefan (metze) Metzmacher wrote:
> > > The only way we could do this is as a separate flag to tdb_open, eg.
> > > TDB_INCOMPATIBLE_HASH or something.  I'm not sure that's worthwhile though?
> > 
> > I'd like to avoid that.
> 
> Me too.  But I really don't want to break some random TDB-relying program :(

OK, here are the patches I ended up with.  It combines Metze's 
use-new-hash-automatically idea with the TDB_INCOMPATIBLE_HASH idea.

So this version is compatible with everything. If you specify
TDB_INCOMPATIBLE_HASH when creating a tdb, you get the improved hash and
no earlier version of tdb can open it.  So it's the user's decision whether
to break compatibility.

http://git.samba.org/data/git/rusty/samba.git #tdb-jhash-default

commit 9c59780cf74dbde5b0027d23d28327e3c455fe93
Author: Rusty Russell <rusty at rustcorp.com.au>
Date:   Fri Sep 24 15:34:06 2010 +0930

    tdb: add Bob Jenkins lookup3 hash as helper hash.
    
    This is a better hash than the default: shipping it with tdb makes it easy
    for callers to use it as the hash by passing it to tdb_open_ex().
    
    This version taken from CCAN and modified, which took it from
    http://www.burtleburtle.net/bob/c/lookup3.c.
    
    Signed-off-by: Rusty Russell <rusty at rustcorp.com.au>

commit b324bb4db25887037600f5ddba1ec6110f2bfcbb
Author: Rusty Russell <rusty at rustcorp.com.au>
Date:   Fri Sep 24 15:39:43 2010 +0930

    tdb: automatically identify Jenkins hash tdbs
    
    If the caller to tdb_open_ex() doesn't specify a hash, and tdb_old_hash
    doesn't match, try tdb_jenkins_hash.
    
    This was Metze's idea: it makes life simpler, especially with the upcoming
    TDB_INCOMPATIBLE_HASH flag.
    
    Signed-off-by: Rusty Russell <rusty at rustcorp.com.au>

commit 88bc7bfabb1e63452c15a9739188f56de245535f
Author: Rusty Russell <rusty at rustcorp.com.au>
Date:   Fri Sep 24 15:45:11 2010 +0930

    tdb: TDB_INCOMPATIBLE_HASH, to allow safe changing of default hash.
    
    This flag to tdb_open/tdb_open_ex effects creation of a new database:
    1) Uses the Jenkins lookup3 hash instead of the old gdbm hash if none is
       specified,
    2) Places a non-zero field in header->rwlocks, so older versions of TDB will
       refuse to open it.
    
    This means that the caller (ie Samba) can set this flag to safely
    change the hash function.  Versions of TDB from this one on will either
    use the correct hash or refuse to open (if a different hash is specified).
    Older TDB versions will see the nonzero rwlocks field and refuse to open
    it under any conditions.
    
    Signed-off-by: Rusty Russell <rusty at rustcorp.com.au>