tight loop in tdb_find()

[tridge at samba.org]

Hi Volker,

 > If possible, can you remove the idmap cache from
 > gencache.tdb again if you use transactions on gencache?

You mean move it into winbindd_cache.tdb? 

 > Logging into a fresh server with a large token will
 > definitely hurt.

because it does a transaction per element in the token? Wouldn't the
better solution be to wrap the loop over the token elements in a
single transaction, so we only pay the transaction cost once?  
(** actually, lots of transactions in a row don't seem to cost much - see
the comments on transaction speed below).

 > The idmap cache was moved to gencache to avoid roundtrips from smbd
 > into winbind, but this might turn out to be the wrong decision now.

It depends on the hit ratio on the cache. If avoiding the roundtrips
saves 2ms per operation (a complete guess), and a transaction costs
10ms (see comments below) then we're better off leaving it in gencache
if we get better than 5:1 hit ratio on the cache.

We also have the same sort of corruption problem in winbindd_cache.tdb
as we currently have in gencache.tdb if "winbind offline logon" is set
to true, as we then don't use TDB_CLEAR_IF_FIRST and don't use
transactions. So with offline logons enabled we could get arbitrary
corruption in the winbind cache on shutdown. That could easily lead to
a security problem, as we could end up putting a user in an incorrect
group.

Regarding transaction cost, I previously estimated the cost of a
transaction as about 200ms. I've now added a transaction speed test to
tdbtool and found it quite a bit lower on average, more like 10ms. 

The first transaction when I start a test often costs 200ms, but
closely spaced transactions after that initial transaction cost a lot
less - around 10ms per transaction on my laptop.

I think the initial transaction cost is high because it adds a new
record, which in the case of this test grows the file. 

Cheers, Tridge