Read corruption possible during ldb_search

[Andrew Bartlett]

On Wed, 2017-04-26 at 10:03 +1200, Garming Sam wrote:
> As far as I know, all the known deadlocks and data corruption should
> be
> fixed in this patchset.
> 
> What I think Andrew was trying to say was that there was no direct
> test
> of my patch ([PATCH 07/17] ldb_tdb: Ensure we correctly decrement
> ltdb->read_lock_count) which fixes the original read consistency
> issue
> we were attempting to resolve. Instead he wrote a cmocka test to show
> that separately (prior to that we only detected it through spurious
> deadlocks during make test).

Thanks.  That was the case last night, sadly in more autobuilds I'm
still seeing deadlock detected so I'm continuing to investigate. 

ldb: ltdb: tdb(/home/ubuntu/autobuild/b27129/samba/bin/ab/fl2008r2dc/private/sam.ldb): tdb_transaction_prepare_commit: failed to upgrade hash locks: Locking error

ldb: ltdb: tdb(/home/ubuntu/autobuild/b27129/samba/bin/ab/fl2008r2dc/private/sam.ldb): tdb_transaction_cancel: no transaction

ldb: dsdb_set_schema() failed: 51:Busy: Failure during tdb_transaction_prepare_commit(): Locking error -> Busy

I think we have this pattern:

(1) start search (allrecord read lock)
(1) transaction start (transaction read lock)
(2) transaction start (transaction read lock)
(1) traverse (requesting transaction write lock, then write locks)
(2) some operation (likely attempting to write new schema index list to
@INDEXLIST)
(2) transaction prepare commit (requesting all-record lock held by (1))
-> DEADLOCK detected, because the 

I'll write another cmocka test when I get a moment to prove it.  I
think the fix is to not request the transaction write lock in the
traverse.

Thanks,

Andrew Bartlett
-- 
Andrew Bartlett                       http://samba.org/~abartlet/
Authentication Developer, Samba Team  http://samba.org
Samba Developer, Catalyst IT          http://catalyst.net.nz/services/samba