ldb one-level search performance issue
Andrew Bartlett
abartlet at samba.org
Fri May 25 05:39:50 UTC 2018
On Fri, 2018-05-25 at 06:25 +0200, Stefan Metzmacher wrote:
> Am 23.05.2018 um 20:55 schrieb Andrew Bartlett via samba-technical:
> > On Wed, 2018-05-23 at 12:43 +0200, Stefan Metzmacher wrote:
> > > Hi Andrew,
> > >
> > > > I've uploaded the patches with tests to
> > > > https://gitlab.com/catalyst-samba/samba/commits/tim-ldb-fix
> > > >
> > > > and the CI is here:
> > > >
> > > > https://gitlab.com/catalyst-samba/samba/pipelines/22482561
> > > >
> > > > If that passes, please review and push so we can then do the release
> > > > (after the python build fix).
> > >
> > > "ldb: Add tests for when we should expect a full scan" fails
> > > "autobuild-private ldb"
> > >
> > > metze
> >
> > Thanks!
> >
> > I see two lessons here:
> >
> > - This is a great example of why pre-submission CI is really important,
> > so we don't waste time on bung patches
> > - Doing patches in the rush out the door before dinner with the family
> > never ends well. The family dinner is more important than the patch,
> > which will always need rework in the morning anyway :-)
> >
> > https://gitlab.com/catalyst-samba/samba/pipelines/22531454
> >
> > (Also already passing a local make test on ldb).
> >
> > If this one passes, please review and push!
>
> I tried, but it seems to consistently fail with:
>
> [851(6535)/853 at 2h13m26s]
> samba4.blackbox.dbcheck(ad_dc_ntvfs)(ad_dc_ntvfs:local)
> WARNING: The "lsa over netlogon" option is deprecated
> WARNING: The "server schannel" option is deprecated
> WARNING: The "lsa over netlogon" option is deprecated
> WARNING: The "server schannel" option is deprecated
> WARNING: The "lsa over netlogon" option is deprecated
> WARNING: The "server schannel" option is deprecated
> UNEXPECTED(failure):
> samba4.blackbox.dbcheck(ad_dc_ntvfs).dbcheck(ad_dc_ntvfs:local)
> REASON: Exception: Exception: WARNING: The "lsa over netlogon" option is
> deprecated
> WARNING: The "server schannel" option is deprecated
> Checking 18786 objects
> NOTE: old (due to rename or delete) DN string component for
> defaultObjectCategory in object
> CN=DrsReplSchema-1527199312-1-cls-B-NEW,CN=Schema,CN=Configuration,DC
> =samba,DC=example,DC=com -
> <GUID=2051e2f9-6b4b-47d1-b146-4b8d7aedf49d>;CN=DrsReplSchema-1527199312-1-cls-B,CN=Schema,CN=Configuration,DC=samba,DC=example,DC=com
> Not fixing old string component
> NOTE: old (due to rename or delete) DN string component for
> defaultObjectCategory in object
> CN=DrsReplSchema-1527199258-1-cls-B-NEW,CN=Schema,CN=Configuration,DC=samba,DC=example,DC=com
Thanks. I've been trying to pretend it was just a flapping test,
fixing other flapping test and blaming gitlab for generally perturbing
the state. This seems a much more plausible.
Tim and I are working on a fix for dbcheck (see attached) and tests,
but the question of why this test now fails remains.
Andrew Bartlett
--
Andrew Bartlett
https://samba.org/~abartlet/
Authentication Developer, Samba Team https://samba.org
Samba Development and Support, Catalyst IT
https://catalyst.net.nz/services/samba
-------------- next part --------------
From c7af354cbe628564dd848fd7abf4b2f74f213f03 Mon Sep 17 00:00:00 2001
From: Tim Beale <timbeale at catalyst.net.nz>
Date: Fri, 25 May 2018 14:05:27 +1200
Subject: [PATCH] dbchecker: Fixing up incorrect DNs wasn't working
dbcheck would fail to fix up attributes where the extended DN's GUID is
correct, but the DN itself is incorrect. The code failed attempting to
remove the old/incorrect DN, e.g.
NOTE: old (due to rename or delete) DN string component for
objectCategory in object CN=alice,CN=Users,DC=samba,DC=example,DC=com -
<GUID=7bfdf9d8-62f9-420c-8a71-e3d3e931c91e>;
CN=Person,CN=Schema,CN=Configuration,DC=samba,DC=bad,DC=com
Change DN to <GUID=7bfdf9d8-62f9-420c-8a71-e3d3e931c91e>;
CN=Person,CN=Schema,CN=Configuration,DC=samba,DC=example,DC=com?
[y/N/all/none] y
Failed to fix old DN string on attribute objectCategory : (16,
"attribute 'objectCategory': no matching attribute value while deleting
attribute on 'CN=alice,CN=Users,DC=samba,DC=example,DC=com'")
The problem was the LDB message specified the value to delete with its
full DN, including the GUID. The LDB code then helpfully corrected this
value on the way through, so that the DN got updated to reflect the
correct DN (i.e. 'DC=example,DC=com') of the object matching that GUID,
rather than the incorrect DN (i.e. 'DC=bad,DC=com') that we were trying
to remove. Because the requested value and the existing DB value didn't
match, the operation failed.
We can avoid this problem by passing down just the DN (not the extended
DN) of the value we want to delete. Without the GUID portion of the DN,
the LDB code will no longer try to correct it on the way through, and
the dbcheck operation will succeed.
Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
---
python/samba/dbchecker.py | 10 ++++++++--
1 file changed, 8 insertions(+), 2 deletions(-)
diff --git a/python/samba/dbchecker.py b/python/samba/dbchecker.py
index 9d72fc6ca94..e67c7423577 100644
--- a/python/samba/dbchecker.py
+++ b/python/samba/dbchecker.py
@@ -1304,8 +1304,14 @@ newSuperior: %s""" % (str(from_dn), str(to_rdn), str(to_base)))
# otherwise, so we don't increment error_count.
if reverse_link_name is None:
if str(res[0].dn) != str(dsdb_dn.dn):
- self.err_dn_string_component_old(obj.dn, attrname, val, dsdb_dn,
- res[0].dn)
+
+ # Pass in the old/bad DN without the <GUID=...> part,
+ # otherwise the LDB code will correct it on the way through
+ # (Note: we still want to preserve the DSDB DN prefix in the
+ # case of binary DNs)
+ bad_dn = dsdb_dn.prefix + dsdb_dn.dn.get_linearized()
+ self.err_dn_string_component_old(obj.dn, attrname, bad_dn,
+ dsdb_dn, res[0].dn)
continue
# check the reverse_link is correct if there should be one
--
2.11.0
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 862 bytes
Desc: This is a digitally signed message part
URL: <http://lists.samba.org/pipermail/samba-technical/attachments/20180525/d861cec9/signature.sig>
More information about the samba-technical
mailing list