Read corruption possible during ldb_search

[Garming Sam]

As far as I know, all the known deadlocks and data corruption should be
fixed in this patchset.

What I think Andrew was trying to say was that there was no direct test
of my patch ([PATCH 07/17] ldb_tdb: Ensure we correctly decrement
ltdb->read_lock_count) which fixes the original read consistency issue
we were attempting to resolve. Instead he wrote a cmocka test to show
that separately (prior to that we only detected it through spurious
deadlocks during make test).


Cheers,

Garming

On 26/04/17 00:20, Stefan Metzmacher via samba-technical wrote:
> Am 25.04.2017 um 13:08 schrieb Andrew Bartlett:
>> I would like to make progress on the ldb and tdb locking issue.
>>
>> I've now included a patch that demonstrates that with ldb_tdb 1.1.29 we
>> allow writes to interfere with indexed reads, meaning that if a write
>> happens during a search, the search output is not consistent. 
>>
>> To demonstrate it, run ./bin/ldb_tdb_mod_op_test in a lib/ldb
>> standalone build with this commit reverted:
>> "ldb_tdb: Ensure we correctly decrement ltdb->read_lock_count"
>>
>> Therefore the locking changes are more than just a performance issue,
>> and I would request that we move forward with it, given the inability
>> to isolate the possible decade-old locking issue on Solaris.
>>
>> In the alternate, could you please propose a route forward on this?  
> I used this branch
> https://git.samba.org/?p=abartlet/samba.git/.git;a=shortlog;h=refs/heads/multi-process-ldap-server
> (with this top commit)
> https://git.samba.org/?p=abartlet/samba.git/.git;a=commit;h=04f8106c705ed3c7573dfd5db1e41e04a1111769
> and combined that with the patches I posted here:
> https://lists.samba.org/archive/samba-technical/2017-April/119979.html
>
> The result is attached and available here:
> https://git.samba.org/?p=metze/samba/wip.git;a=shortlog;h=refs/heads/master4-ldb
>
> So my most important question:
> Do you think the patches in my branch fix all of our known deadlock/
> data corruption bugs?
>
> All mails in this thread indicated to me that there're more bugs to be
> fixed.
> But this could also be a misunderstanding.
>
> If there're no more bugs to be fixed and I start with a more deep
> review, otherwise please base your additions on top of my branch.
>
> Regarding the Solaris deadlock problem, I guess we can ignore it
> as the test we wrote also works on Solaris 10, but I'll run the
> full standalone testsuites for ldb and tdb again before it might
> be pushed to master.
>
> Thanks!
> metze


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 490 bytes
Desc: OpenPGP digital signature
URL: <http://lists.samba.org/pipermail/samba-technical/attachments/20170426/edcd57bd/signature.sig>