[SCM] Samba Shared Repository - branch v4-7-test updated
Karolin Seeger
kseeger at samba.org
Wed Nov 29 12:44:02 UTC 2017
The branch, v4-7-test has been updated
via 00dfe4c repl_meta_data: Fix removing of backlink on deleted objects
via f2c4754 repl_meta_data: Allow delete of an object with dangling backlinks
via fe3ae81 selftest: Add more corruption cases for runtime and dbcheck
via 09a311c selftest: add dbcheck tests for duplicate links
via ece46f7 dbcheck: detect and fix duplicate links
via c1860d1 dbcheck: only calculate linked attribute helper variables once in check_dn()
via 7b67da4 dbcheck: remove indentation level
via 27cdadf dsdb:extended_dn_store: implement DSDB_CONTROL_DBCHECK_FIX_DUPLICATE_LINKS control
via aafcb73 dsdb:repl_meta_data: implement DSDB_CONTROL_DBCHECK_FIX_DUPLICATE_LINKS control
via f0920ea s4:dsdb: allocate DSDB_CONTROL_DBCHECK_FIX_DUPLICATE_LINKS oid
via 67270b3 s4:schema_samba4: mark DSDB_CONTROL_INVALID_NOT_IMPLEMENTED 1.3.6.1.4.1.7165.4.3.32 as allocated
via ee05047 selftest: Additional check for a backlink pointing at a deleted object
via df23dc8 selftest: Split out creation of complex (often invalid) links
via 505936a selftest: Split out dbcheck runs from dangling_multi_valued test
via 74688a2 selftest: add more dbcheck tests
via 68bf275 dbcheck: Use the GUID as the DN to fix replPropertyMetaData
via b7c9edb dbcheck: Clarify error count bumping in deleted/gone DN handling
via ed4189f selftest: sort dbcheck output to avoid sort order impacting results
via 45911b0 dbcheck: Allow removal of one-way links to missing objects
via e90e3d7 systemd: Start processes in forground and without a process group
from fe6da67 libnet_join: fix "net rpc oldjoin"
https://git.samba.org/?p=samba.git;a=shortlog;h=v4-7-test
- Log -----------------------------------------------------------------
commit 00dfe4cf9818abdc50da034fd931e29bc871b10c
Author: Andrej Gessel <Andrej.Gessel at janztec.com>
Date: Thu Oct 19 17:16:37 2017 +0200
repl_meta_data: Fix removing of backlink on deleted objects
USER is memberOf GROUP and they both were deleted on W2K8R2 AD. Domain join ends
with error below.
Failed to apply records: ../source4/dsdb/samdb/ldb_modules/repl_meta_data.c:421
8: Failed to remove backlink of memberOf when deleting CN=USER\0ADEL:a1f2a2cc-1
179-4734-b753-c121ed02a34c,CN=Deleted Objects,DC=samdom,DC=intern: dsdb_module_
search_dn: did not find base dn CN=GROUP\0ADEL:030d0be1-3ada-4b93-8371-927f2092
3116,CN=Deleted Objects,DC=samdom,DC=intern (0 results): Operations error
Failed to commit objects: WERR_GEN_FAILURE/NT_STATUS_INVALID_NETWORK_RESPONSE
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13120
Signed-off-by: Andrej Gessel <Andrej.Gessel at janztec.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>
(cherry picked from commit 40bd7e145a68c9a58d6bc3c5526a12fdf0027729)
Autobuild-User(v4-7-test): Karolin Seeger <kseeger at samba.org>
Autobuild-Date(v4-7-test): Wed Nov 29 13:43:37 CET 2017 on sn-devel-144
commit f2c47544dc8fd42680b1f90bc962e57fe915de45
Author: Andrew Bartlett <abartlet at samba.org>
Date: Wed Nov 1 08:22:22 2017 +1300
repl_meta_data: Allow delete of an object with dangling backlinks
This should not happen, but stopping all replication because of it is a pain.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13095
Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>
Autobuild-User(master): Stefan Metzmacher <metze at samba.org>
Autobuild-Date(master): Fri Nov 24 19:53:50 CET 2017 on sn-devel-144
(cherry picked from commit 6cf7abbcfdad84fee57852862ebe44aa6115ca25)
commit fe3ae81cd9a5cf263da9cb373d07ebad4541727b
Author: Andrew Bartlett <abartlet at samba.org>
Date: Wed Nov 1 09:02:01 2017 +1300
selftest: Add more corruption cases for runtime and dbcheck
These tests now confirm we can handle these issues at runtime
as well as at dbcheck
Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>
(cherry picked from commit 4815efc0e3f89079e7c9b868b7514ea7c49a807c)
commit 09a311caa602265b18067d8cd155e91f33918b1a
Author: Stefan Metzmacher <metze at samba.org>
Date: Thu Oct 26 14:42:23 2017 +0200
selftest: add dbcheck tests for duplicate links
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13095
Pair-Programmed-With: Andrew Bartlett <abartlet at samba.org>
Signed-off-by: Stefan Metzmacher <metze at samba.org>
Signed-off-by: Andrew Bartlett <abartlet at samba.org>
(cherry picked from commit 70bf809e0cdf84029022ca95fb83d17a0d6e36c0)
commit ece46f7278ed2633167d01f483cdb44858d0dd51
Author: Stefan Metzmacher <metze at samba.org>
Date: Fri Oct 27 10:21:26 2017 +0200
dbcheck: detect and fix duplicate links
Check with git show -w
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13095
Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
(cherry picked from commit 239fbeb163c24b0f08e1bd9d8f7a9f73443d4b90)
commit c1860d169cb60a1d4e40800ad6e87e81fa24fcac
Author: Stefan Metzmacher <metze at samba.org>
Date: Fri Oct 27 10:21:26 2017 +0200
dbcheck: only calculate linked attribute helper variables once in check_dn()
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13095
Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
(cherry picked from commit 9a631560c9358e4f28b96fecf23a545e1d04098c)
commit 7b67da4d84ec4b462eaca36b80a94e49f9f45739
Author: Stefan Metzmacher <metze at samba.org>
Date: Thu Oct 26 16:30:28 2017 +0200
dbcheck: remove indentation level
Check with git show -w
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13095
Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
(cherry picked from commit eb6bd6511a98dafebaa0d3951fe78c77acf13e3a)
commit 27cdadf1d0791454747a0e6ac7f2e7ef8ba881fc
Author: Stefan Metzmacher <metze at samba.org>
Date: Thu Oct 26 07:47:48 2017 +0200
dsdb:extended_dn_store: implement DSDB_CONTROL_DBCHECK_FIX_DUPLICATE_LINKS control
This will be used by dbcheck to fix duplicate link values.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13095
Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
(cherry picked from commit 83aa22260bd39ad0e6aab7764f9a7fc915d02a4b)
commit aafcb7381a8f3b8d6d44528adee5e7fa2cf8b62a
Author: Stefan Metzmacher <metze at samba.org>
Date: Wed Oct 25 16:48:44 2017 +0200
dsdb:repl_meta_data: implement DSDB_CONTROL_DBCHECK_FIX_DUPLICATE_LINKS control
This will be used by dbcheck to fix duplicate link values.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13095
Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
(cherry picked from commit 9a1e23a1f67c38248e41e0d3aa2af8a682477364)
commit f0920ea48b9cdffaaf0a9f0912543e2289a2dcca
Author: Stefan Metzmacher <metze at samba.org>
Date: Wed Oct 25 16:47:36 2017 +0200
s4:dsdb: allocate DSDB_CONTROL_DBCHECK_FIX_DUPLICATE_LINKS oid
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13095
Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
(cherry picked from commit 1eb8d8ec5a62baf1b8e3c7cb1856787de4a3ccb2)
commit 67270b3f7ed0963f26f65298c88791d0d97c643b
Author: Stefan Metzmacher <metze at samba.org>
Date: Wed Oct 25 16:26:16 2017 +0200
s4:schema_samba4: mark DSDB_CONTROL_INVALID_NOT_IMPLEMENTED 1.3.6.1.4.1.7165.4.3.32 as allocated
Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
(cherry picked from commit 126d28d0b58740a8abd4137562dda685a57449bb)
commit ee05047cb6a4c6be6a17a7c39a46b93ddcc7f98f
Author: Andrew Bartlett <abartlet at samba.org>
Date: Tue Oct 31 11:20:34 2017 +1300
selftest: Additional check for a backlink pointing at a deleted object
Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>
(cherry picked from commit a784cc3a7f2043a5762d426e904a90e44b101ecd)
commit df23dc813296b836ca98b5575cc5974b6a781692
Author: Andrew Bartlett <abartlet at samba.org>
Date: Tue Oct 31 08:23:39 2017 +1300
selftest: Split out creation of complex (often invalid) links
This will allow us to test other run-time behaviour with broken
databases.
Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>
(cherry picked from commit 25ae8d72d66cbe7342b50254ede7e5890bc23b73)
commit 505936a986a7703eb3349603a560bc70eccbead9
Author: Andrew Bartlett <abartlet at samba.org>
Date: Tue Oct 31 08:21:15 2017 +1300
selftest: Split out dbcheck runs from dangling_multi_valued test
Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>
(cherry picked from commit b99d2ee122991d0bf1742fa5665656bbbba44057)
commit 74688a256ad1ae78adff08fdab22e480f183490c
Author: Andrew Bartlett <abartlet at samba.org>
Date: Mon Oct 30 15:29:36 2017 +1300
selftest: add more dbcheck tests
This validates some more combinations and ensures that the changes
in 962a1b32201fce0a49c6be55943d4fbb57ed781e are tested.
Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>
(cherry picked from commit 7be38c605468786894a373e15068b8017323da78)
commit 68bf2752e22deec5bed1f7ea3aad8850d53bd0c8
Author: Andrew Bartlett <abartlet at samba.org>
Date: Mon Oct 30 10:51:35 2017 +1300
dbcheck: Use the GUID as the DN to fix replPropertyMetaData
This allows this to still work after an object is renamed under the deleted objects container.
Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>
(cherry picked from commit 527f2c95cfe24d29fa34ed19db3b073781d96d9a)
commit b7c9edb392f76d557f46cd119c398b874c80ce1e
Author: Andrew Bartlett <abartlet at samba.org>
Date: Mon Oct 30 09:48:43 2017 +1300
dbcheck: Clarify error count bumping in deleted/gone DN handling
Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>
(cherry picked from commit 3b111fbdbed99d5d90c1120243200baae9867534)
commit ed4189f3bb1db352ddff396a8ee6d1e103d67efa
Author: Andrew Bartlett <abartlet at samba.org>
Date: Fri Aug 25 17:37:05 2017 +1200
selftest: sort dbcheck output to avoid sort order impacting results
The GUID index code will change the returned results order
Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Garming Sam <garming at catalyst.net.nz>
(cherry picked from commit da575f01313673fedfc7d15ec11ba6818dbd30d8)
commit 45911b030072cbfb305b104e304996dc2b5ee4ff
Author: Andrew Bartlett <abartlet at samba.org>
Date: Tue Oct 17 23:01:51 2017 +1300
dbcheck: Allow removal of one-way links to missing objects
If dbcheck is not run within the tombstone lifetime, these links can
persist in the database forever. The risk of unintentional information loss
is why these links are only removed within the same partition. A
replication may be in progress which has created only one end of
the link, so we must keep that.
Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Rowland Penny <rpenny at samba.org>
Autobuild-User(master): Jeremy Allison <jra at samba.org>
Autobuild-Date(master): Thu Oct 19 00:50:19 CEST 2017 on sn-devel-144
(cherry picked from commit 962a1b32201fce0a49c6be55943d4fbb57ed781e)
commit e90e3d7efb555dc430ba47c433c96f4f4835f33f
Author: Andreas Schneider <asn at samba.org>
Date: Fri Nov 10 09:32:27 2017 +0100
systemd: Start processes in forground and without a process group
We should not double fork in notify mode or systemd think something
during startup will be wrong and send SIGTERM to the process. So
sometimes the daemon will not start up correctly.
systemd will also handle the process group.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13129
Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
(cherry picked from commit 8b6f58194da7e849cdb9d20712dff49b17a93a77)
-----------------------------------------------------------------------
Summary of changes:
packaging/systemd/nmb.service | 2 +-
packaging/systemd/samba.service | 2 +-
packaging/systemd/smb.service | 2 +-
packaging/systemd/winbind.service | 2 +-
python/samba/dbchecker.py | 305 +++++++++++++++------
selftest/knownfail | 2 +-
selftest/tests.py | 5 +
source4/dsdb/samdb/ldb_modules/extended_dn_store.c | 7 +
source4/dsdb/samdb/ldb_modules/repl_meta_data.c | 81 +++++-
source4/dsdb/samdb/samdb.h | 3 +
.../add-dangling-deleted-link.ldif | 5 +
.../add-deleted-backlink-user.ldif | 3 +
...ing-backlink.ldif => add-deleted-backlink.ldif} | 3 +-
.../add-deleted-target-backlink-user.ldif | 3 +
.../add-deleted-target-backlink.ldif | 4 +
...-one-way-link.ldif => dangling-one-way-dn.ldif} | 0
.../release-4-5-0-pre1/dangling-one-way-link.ldif | 22 +-
.../release-4-5-0-pre1/deleted-one-way-dn.ldif | 13 +
.../expected-dbcheck-link-output.txt | 58 ++--
...pected-dbcheck-link-output_duplicate_member.txt | 8 +
.../expected-dbcheck-link-output_one_way.txt | 7 +
.../expected-duplicates-after-link-dbcheck.ldif | 28 ++
.../revive-backlink-on-deleted-group.ldif | 5 +
.../revive-links-on-deleted-group.ldif | 20 ++
source4/setup/schema_samba4.ldif | 2 +
testprogs/blackbox/common-links.sh | 215 +++++++++++++++
testprogs/blackbox/dbcheck-links.sh | 212 +++++---------
testprogs/blackbox/runtime-links.sh | 74 +++++
28 files changed, 816 insertions(+), 277 deletions(-)
create mode 100644 source4/selftest/provisions/release-4-5-0-pre1/add-dangling-deleted-link.ldif
create mode 100644 source4/selftest/provisions/release-4-5-0-pre1/add-deleted-backlink-user.ldif
copy source4/selftest/provisions/release-4-5-0-pre1/{add-dangling-backlink.ldif => add-deleted-backlink.ldif} (67%)
create mode 100644 source4/selftest/provisions/release-4-5-0-pre1/add-deleted-target-backlink-user.ldif
create mode 100644 source4/selftest/provisions/release-4-5-0-pre1/add-deleted-target-backlink.ldif
copy source4/selftest/provisions/release-4-5-0-pre1/{dangling-one-way-link.ldif => dangling-one-way-dn.ldif} (100%)
create mode 100644 source4/selftest/provisions/release-4-5-0-pre1/deleted-one-way-dn.ldif
create mode 100644 source4/selftest/provisions/release-4-5-0-pre1/expected-dbcheck-link-output_duplicate_member.txt
create mode 100644 source4/selftest/provisions/release-4-5-0-pre1/expected-dbcheck-link-output_one_way.txt
create mode 100644 source4/selftest/provisions/release-4-5-0-pre1/expected-duplicates-after-link-dbcheck.ldif
create mode 100644 source4/selftest/provisions/release-4-5-0-pre1/revive-backlink-on-deleted-group.ldif
create mode 100644 source4/selftest/provisions/release-4-5-0-pre1/revive-links-on-deleted-group.ldif
create mode 100644 testprogs/blackbox/common-links.sh
create mode 100755 testprogs/blackbox/runtime-links.sh
Changeset truncated at 500 lines:
diff --git a/packaging/systemd/nmb.service b/packaging/systemd/nmb.service
index 992c0cd..71c93d6 100644
--- a/packaging/systemd/nmb.service
+++ b/packaging/systemd/nmb.service
@@ -7,7 +7,7 @@ Type=notify
NotifyAccess=all
PIDFile=/run/nmbd.pid
EnvironmentFile=-/etc/sysconfig/samba
-ExecStart=/usr/sbin/nmbd $NMBDOPTIONS
+ExecStart=/usr/sbin/nmbd --foreground --no-process-group $NMBDOPTIONS
ExecReload=/usr/bin/kill -HUP $MAINPID
LimitCORE=infinity
diff --git a/packaging/systemd/samba.service b/packaging/systemd/samba.service
index 824f89c..1b64c3b 100644
--- a/packaging/systemd/samba.service
+++ b/packaging/systemd/samba.service
@@ -8,7 +8,7 @@ NotifyAccess=all
PIDFile=/run/samba.pid
LimitNOFILE=16384
EnvironmentFile=-/etc/sysconfig/samba
-ExecStart=/usr/sbin/samba $SAMBAOPTIONS
+ExecStart=/usr/sbin/samba --foreground --no-process-group $SAMBAOPTIONS
ExecReload=/usr/bin/kill -HUP $MAINPID
[Install]
diff --git a/packaging/systemd/smb.service b/packaging/systemd/smb.service
index 6053a5c..adf6684 100644
--- a/packaging/systemd/smb.service
+++ b/packaging/systemd/smb.service
@@ -8,7 +8,7 @@ NotifyAccess=all
PIDFile=/run/smbd.pid
LimitNOFILE=16384
EnvironmentFile=-/etc/sysconfig/samba
-ExecStart=/usr/sbin/smbd $SMBDOPTIONS
+ExecStart=/usr/sbin/smbd --foreground --no-process-group $SMBDOPTIONS
ExecReload=/usr/bin/kill -HUP $MAINPID
LimitCORE=infinity
diff --git a/packaging/systemd/winbind.service b/packaging/systemd/winbind.service
index c511488..46b3797 100644
--- a/packaging/systemd/winbind.service
+++ b/packaging/systemd/winbind.service
@@ -7,7 +7,7 @@ Type=notify
NotifyAccess=all
PIDFile=/run/winbindd.pid
EnvironmentFile=-/etc/sysconfig/samba
-ExecStart=/usr/sbin/winbindd "$WINBINDOPTIONS"
+ExecStart=/usr/sbin/winbindd --foreground --no-process-group "$WINBINDOPTIONS"
ExecReload=/usr/bin/kill -HUP $MAINPID
LimitCORE=infinity
diff --git a/python/samba/dbchecker.py b/python/samba/dbchecker.py
index 1a73fe0..1933740 100644
--- a/python/samba/dbchecker.py
+++ b/python/samba/dbchecker.py
@@ -65,6 +65,7 @@ class dbcheck(object):
self.fix_undead_linked_attributes = False
self.fix_all_missing_backlinks = False
self.fix_all_orphaned_backlinks = False
+ self.fix_all_duplicate_links = False
self.fix_rmd_flags = False
self.fix_ntsecuritydescriptor = False
self.fix_ntsecuritydescriptor_owner_group = False
@@ -498,13 +499,15 @@ newSuperior: %s""" % (str(from_dn), str(to_rdn), str(to_base)))
def err_deleted_dn(self, dn, attrname, val, dsdb_dn, correct_dn, remove_plausible=False):
"""handle a DN pointing to a deleted object"""
- self.report("ERROR: target DN is deleted for %s in object %s - %s" % (attrname, dn, val))
- self.report("Target GUID points at deleted DN %r" % str(correct_dn))
if not remove_plausible:
+ self.report("ERROR: target DN is deleted for %s in object %s - %s" % (attrname, dn, val))
+ self.report("Target GUID points at deleted DN %r" % str(correct_dn))
if not self.confirm_all('Remove DN link?', 'remove_implausible_deleted_DN_links'):
self.report("Not removing")
return
else:
+ self.report("WARNING: target DN is deleted for %s in object %s - %s" % (attrname, dn, val))
+ self.report("Target GUID points at deleted DN %r" % str(correct_dn))
if not self.confirm_all('Remove stale DN link?', 'remove_plausible_deleted_DN_links'):
self.report("Not removing")
return
@@ -523,9 +526,49 @@ newSuperior: %s""" % (str(from_dn), str(to_rdn), str(to_base)))
# check if its a backlink
linkID, _ = self.get_attr_linkID_and_reverse_name(attrname)
if (linkID & 1 == 0) and str(dsdb_dn).find('\\0ADEL') == -1:
- self.report("Not removing dangling forward link")
- return
+
+ linkID, reverse_link_name \
+ = self.get_attr_linkID_and_reverse_name(attrname)
+ if reverse_link_name is not None:
+ self.report("WARNING: no target object found for GUID "
+ "component for one-way forward link "
+ "%s in object "
+ "%s - %s" % (attrname, dn, val))
+ self.report("Not removing dangling forward link")
+ return 0
+
+ nc_root = self.samdb.get_nc_root(dn)
+ target_nc_root = self.samdb.get_nc_root(dsdb_dn.dn)
+ if nc_root != target_nc_root:
+ # We don't bump the error count as Samba produces these
+ # in normal operation
+ self.report("WARNING: no target object found for GUID "
+ "component for cross-partition link "
+ "%s in object "
+ "%s - %s" % (attrname, dn, val))
+ self.report("Not removing dangling one-way "
+ "cross-partition link "
+ "(we might be mid-replication)")
+ return 0
+
+ # Due to our link handling one-way links pointing to
+ # missing objects are plausible.
+ #
+ # We don't bump the error count as Samba produces these
+ # in normal operation
+ self.report("WARNING: no target object found for GUID "
+ "component for DN value %s in object "
+ "%s - %s" % (attrname, dn, val))
+ self.err_deleted_dn(dn, attrname, val,
+ dsdb_dn, dsdb_dn, True)
+ return 0
+
+ # We bump the error count here, as we should have deleted this
+ self.report("ERROR: no target object found for GUID "
+ "component for link %s in object "
+ "%s - %s" % (attrname, dn, val))
self.err_deleted_dn(dn, attrname, val, dsdb_dn, dsdb_dn, False)
+ return 1
def err_missing_dn_GUID_component(self, dn, attrname, val, dsdb_dn, errstr):
"""handle a missing GUID extended DN component"""
@@ -678,6 +721,19 @@ newSuperior: %s""" % (str(from_dn), str(to_rdn), str(to_base)))
"Failed to fix orphaned backlink %s" % attrname):
self.report("Fixed orphaned backlink %s" % (attrname))
+ def err_duplicate_links(self, obj, attrname, vals):
+ '''handle a duplicate links value'''
+
+ if not self.confirm_all("Remove duplicate links in attribute '%s'" % attrname, 'fix_all_duplicate_links'):
+ self.report("Not removing duplicate links in attribute '%s'" % attrname)
+ return
+ m = ldb.Message()
+ m.dn = obj.dn
+ m['value'] = ldb.MessageElement(vals, ldb.FLAG_MOD_REPLACE, attrname)
+ if self.do_modify(m, ["local_oid:1.3.6.1.4.1.7165.4.3.19.2:1"],
+ "Failed to fix duplicate links in attribute '%s'" % attrname):
+ self.report("Fixed duplicate links in attribute '%s'" % (attrname))
+
def err_no_fsmoRoleOwner(self, obj):
'''handle a missing fSMORoleOwner'''
self.report("ERROR: fSMORoleOwner not found for role %s" % (obj.dn))
@@ -834,6 +890,81 @@ newSuperior: %s""" % (str(from_dn), str(to_rdn), str(to_base)))
error_count = 0
obj_guid = obj['objectGUID'][0]
+ linkID, reverse_link_name = self.get_attr_linkID_and_reverse_name(attrname)
+ if reverse_link_name is not None:
+ reverse_syntax_oid = self.samdb_schema.get_syntax_oid_from_lDAPDisplayName(reverse_link_name)
+ else:
+ reverse_syntax_oid = None
+
+ duplicate_dict = dict()
+ duplicate_list = list()
+ unique_dict = dict()
+ unique_list = list()
+ for val in obj[attrname]:
+ if linkID & 1:
+ #
+ # Only cleanup forward links here,
+ # back links are handled below.
+ break
+
+ dsdb_dn = dsdb_Dn(self.samdb, val, syntax_oid)
+
+ # all DNs should have a GUID component
+ guid = dsdb_dn.dn.get_extended_component("GUID")
+ if guid is None:
+ continue
+ guidstr = str(misc.GUID(guid))
+ keystr = guidstr + dsdb_dn.prefix
+ if keystr not in unique_dict:
+ unique_dict[keystr] = dsdb_dn
+ unique_list.append(keystr)
+ continue
+ error_count += 1
+ if keystr not in duplicate_dict:
+ duplicate_dict[keystr] = dict()
+ duplicate_dict[keystr]["keep"] = None
+ duplicate_dict[keystr]["delete"] = list()
+ duplicate_list.append(keystr)
+
+ # Now check for the highest RMD_VERSION
+ v1 = int(unique_dict[keystr].dn.get_extended_component("RMD_VERSION"))
+ v2 = int(dsdb_dn.dn.get_extended_component("RMD_VERSION"))
+ if v1 > v2:
+ duplicate_dict[keystr]["keep"] = unique_dict[keystr]
+ duplicate_dict[keystr]["delete"].append(dsdb_dn)
+ continue
+ if v1 < v2:
+ duplicate_dict[keystr]["keep"] = dsdb_dn
+ duplicate_dict[keystr]["delete"].append(unique_dict[keystr])
+ unique_dict[keystr] = dsdb_dn
+ continue
+ # Fallback to the highest RMD_LOCAL_USN
+ u1 = int(unique_dict[keystr].dn.get_extended_component("RMD_LOCAL_USN"))
+ u2 = int(dsdb_dn.dn.get_extended_component("RMD_LOCAL_USN"))
+ if u1 >= u2:
+ duplicate_dict[keystr]["keep"] = unique_dict[keystr]
+ duplicate_dict[keystr]["delete"].append(dsdb_dn)
+ continue
+ duplicate_dict[keystr]["keep"] = dsdb_dn
+ duplicate_dict[keystr]["delete"].append(unique_dict[keystr])
+ unique_dict[keystr] = dsdb_dn
+
+ if len(duplicate_list) != 0:
+ self.report("ERROR: Duplicate link values for attribute '%s' in '%s'" % (attrname, obj.dn))
+ for keystr in duplicate_list:
+ d = duplicate_dict[keystr]
+ for dd in d["delete"]:
+ self.report("Duplicate link '%s'" % dd)
+ self.report("Correct link '%s'" % d["keep"])
+
+ vals = []
+ for keystr in unique_list:
+ dsdb_dn = unique_dict[keystr]
+ vals.append(str(dsdb_dn))
+ self.err_duplicate_links(obj, attrname, vals)
+ # We should continue with the fixed values
+ obj[attrname] = ldb.MessageElement(vals, ldb.FLAG_MOD_REPLACE, attrname)
+
for val in obj[attrname]:
dsdb_dn = dsdb_Dn(self.samdb, val, syntax_oid)
@@ -854,7 +985,6 @@ newSuperior: %s""" % (str(from_dn), str(to_rdn), str(to_base)))
else:
fixing_msDS_HasInstantiatedNCs = False
- linkID, reverse_link_name = self.get_attr_linkID_and_reverse_name(attrname)
if reverse_link_name is not None:
attrs.append(reverse_link_name)
@@ -865,12 +995,14 @@ newSuperior: %s""" % (str(from_dn), str(to_rdn), str(to_base)))
"reveal_internals:0"
])
except ldb.LdbError, (enum, estr):
- error_count += 1
- self.report("ERROR: no target object found for GUID component for %s in object %s - %s" % (attrname, obj.dn, val))
if enum != ldb.ERR_NO_SUCH_OBJECT:
raise
- self.err_missing_target_dn_or_GUID(obj.dn, attrname, val, dsdb_dn)
+ # We don't always want to
+ error_count += self.err_missing_target_dn_or_GUID(obj.dn,
+ attrname,
+ val,
+ dsdb_dn)
continue
if fixing_msDS_HasInstantiatedNCs:
@@ -926,16 +1058,15 @@ newSuperior: %s""" % (str(from_dn), str(to_rdn), str(to_base)))
# components on deleted links, as these are allowed to
# go stale (we just need the GUID, not the name)
rmd_blob = dsdb_dn.dn.get_extended_component("RMD_FLAGS")
+ rmd_flags = 0
if rmd_blob is not None:
rmd_flags = int(rmd_blob)
- if rmd_flags & 1:
- continue
# assert the DN matches in string form, where a reverse
# link exists, otherwise (below) offer to fix it as a non-error.
# The string form is essentially only kept for forensics,
# as we always re-resolve by GUID in normal operations.
- if reverse_link_name is not None:
+ if not rmd_flags & 1 and reverse_link_name is not None:
if str(res[0].dn) != str(dsdb_dn.dn):
error_count += 1
self.err_dn_component_target_mismatch(obj.dn, attrname, val, dsdb_dn,
@@ -964,76 +1095,93 @@ newSuperior: %s""" % (str(from_dn), str(to_rdn), str(to_base)))
res[0].dn)
continue
- else:
- # check the reverse_link is correct if there should be one
- match_count = 0
- if reverse_link_name in res[0]:
- for v in res[0][reverse_link_name]:
- v_guid = dsdb_Dn(self.samdb, v).dn.get_extended_component("GUID")
- if v_guid == obj_guid:
- match_count += 1
- if match_count != 1:
- reverse_syntax_oid = self.samdb_schema.get_syntax_oid_from_lDAPDisplayName(reverse_link_name)
- if syntax_oid == dsdb.DSDB_SYNTAX_BINARY_DN or reverse_syntax_oid == dsdb.DSDB_SYNTAX_BINARY_DN:
- if not linkID & 1:
- # Forward binary multi-valued linked attribute
- forward_count = 0
- for w in obj[attrname]:
- w_guid = dsdb_Dn(self.samdb, w).dn.get_extended_component("GUID")
- if w_guid == guid:
- forward_count += 1
-
- if match_count == forward_count:
- continue
-
- error_count += 1
-
- # Add or remove the missing number of backlinks
- diff_count = forward_count - match_count
-
- # Loop until the difference between the forward and
- # the backward links is resolved.
- while diff_count != 0:
- if diff_count > 0:
- # self.err_missing_backlink(obj, attrname,
- # obj.dn.extended_str(),
- # reverse_link_name,
- # dsdb_dn.dn)
- # diff_count -= 1
- # TODO no method to fix these right now
- self.report("ERROR: Can't fix missing "
- "multi-valued backlinks on %s" % str(dsdb_dn.dn))
- break
- else:
- self.err_orphaned_backlink(res[0], reverse_link_name,
- obj.dn.extended_str(), attrname,
- dsdb_dn.dn)
- diff_count += 1
+ # check the reverse_link is correct if there should be one
+ match_count = 0
+ if reverse_link_name in res[0]:
+ for v in res[0][reverse_link_name]:
+ v_dn = dsdb_Dn(self.samdb, v)
+ v_guid = v_dn.dn.get_extended_component("GUID")
+ v_blob = v_dn.dn.get_extended_component("RMD_FLAGS")
+ v_rmd_flags = 0
+ if v_blob is not None:
+ v_rmd_flags = int(v_blob)
+ if v_rmd_flags & 1:
+ continue
+ if v_guid == obj_guid:
+ match_count += 1
+
+ if match_count != 1:
+ if syntax_oid == dsdb.DSDB_SYNTAX_BINARY_DN or reverse_syntax_oid == dsdb.DSDB_SYNTAX_BINARY_DN:
+ if not linkID & 1:
+ # Forward binary multi-valued linked attribute
+ forward_count = 0
+ for w in obj[attrname]:
+ w_guid = dsdb_Dn(self.samdb, w).dn.get_extended_component("GUID")
+ if w_guid == guid:
+ forward_count += 1
+
+ if match_count == forward_count:
+ continue
+ expected_count = 0
+ for v in obj[attrname]:
+ v_dn = dsdb_Dn(self.samdb, v)
+ v_guid = v_dn.dn.get_extended_component("GUID")
+ v_blob = v_dn.dn.get_extended_component("RMD_FLAGS")
+ v_rmd_flags = 0
+ if v_blob is not None:
+ v_rmd_flags = int(v_blob)
+ if v_rmd_flags & 1:
+ continue
+ if v_guid == guid:
+ expected_count += 1
- else:
- # If there's a backward link on binary multi-valued linked attribute,
- # let the check on the forward link remedy the value.
- # UNLESS, there is no forward link detected.
- if match_count == 0:
- self.err_orphaned_backlink(obj, attrname,
- val, reverse_link_name,
- dsdb_dn.dn)
+ if match_count == expected_count:
+ continue
- continue
+ diff_count = expected_count - match_count
+ if linkID & 1:
+ # If there's a backward link on binary multi-valued linked attribute,
+ # let the check on the forward link remedy the value.
+ # UNLESS, there is no forward link detected.
+ if match_count == 0:
error_count += 1
- if linkID & 1:
- # Backlink exists, but forward link does not
- # Delete the hanging backlink
- self.err_orphaned_backlink(obj, attrname, val, reverse_link_name, dsdb_dn.dn)
- else:
- # Forward link exists, but backlink does not
- # Add the missing backlink (if the target object is not Deleted Objects?)
- if not target_is_deleted:
- self.err_missing_backlink(obj, attrname, obj.dn.extended_str(), reverse_link_name, dsdb_dn.dn)
+ self.err_orphaned_backlink(obj, attrname,
+ val, reverse_link_name,
+ dsdb_dn.dn)
continue
+ # Only warn here and let the forward link logic fix it.
+ self.report("WARNING: Link (back) mismatch for '%s' (%d) on '%s' to '%s' (%d) on '%s'" % (
+ attrname, expected_count, str(obj.dn),
+ reverse_link_name, match_count, str(dsdb_dn.dn)))
+ continue
+
+ assert not target_is_deleted
+ self.report("ERROR: Link (forward) mismatch for '%s' (%d) on '%s' to '%s' (%d) on '%s'" % (
+ attrname, expected_count, str(obj.dn),
+ reverse_link_name, match_count, str(dsdb_dn.dn)))
+ # Loop until the difference between the forward and
+ # the backward links is resolved.
+ while diff_count != 0:
+ error_count += 1
+ if diff_count > 0:
+ if match_count > 0 or diff_count > 1:
+ # TODO no method to fix these right now
+ self.report("ERROR: Can't fix missing "
+ "multi-valued backlinks on %s" % str(dsdb_dn.dn))
+ break
+ self.err_missing_backlink(obj, attrname,
+ obj.dn.extended_str(),
+ reverse_link_name,
+ dsdb_dn.dn)
+ diff_count -= 1
+ else:
+ self.err_orphaned_backlink(res[0], reverse_link_name,
+ obj.dn.extended_str(), attrname,
+ obj.dn)
+ diff_count += 1
return error_count
@@ -1079,11 +1227,14 @@ newSuperior: %s""" % (str(from_dn), str(to_rdn), str(to_base)))
return (set_att, list_attid, wrong_attids)
- def fix_metadata(self, dn, attr):
+ def fix_metadata(self, obj, attr):
'''re-write replPropertyMetaData elements for a single attribute for a
object. This is used to fix missing replPropertyMetaData elements'''
+ guid_str = str(ndr_unpack(misc.GUID, obj['objectGUID'][0]))
+ dn = ldb.Dn(self.samdb, "<GUID=%s>" % guid_str)
res = self.samdb.search(base = dn, scope=ldb.SCOPE_BASE, attrs = [attr],
- controls = ["search_options:1:2", "show_recycled:1"])
+ controls = ["search_options:1:2",
+ "show_recycled:1"])
msg = res[0]
nmsg = ldb.Message()
nmsg.dn = dn
@@ -1925,7 +2076,7 @@ newSuperior: %s""" % (str(from_dn), str(to_rdn), str(to_base)))
if not self.confirm_all("Fix missing replPropertyMetaData element '%s'" % att, 'fix_all_metadata'):
self.report("Not fixing missing replPropertyMetaData element '%s'" % att)
continue
- self.fix_metadata(dn, att)
+ self.fix_metadata(obj, att)
if self.is_fsmo_role(dn):
if "fSMORoleOwner" not in obj and ("*" in attrs or "fsmoroleowner" in map(str.lower, attrs)):
diff --git a/selftest/knownfail b/selftest/knownfail
index dfa055f..dd23c7d 100644
--- a/selftest/knownfail
+++ b/selftest/knownfail
@@ -318,7 +318,7 @@
^samba3.smb2.credits.session_setup_credits_granted.*
^samba3.smb2.credits.single_req_credits_granted.*
^samba3.smb2.credits.skipped_mid.*
-^samba4.blackbox.dbcheck-links.release-4-5-0-pre1.dangling_multi_valued_dbcheck
+^samba4.blackbox.dbcheck-links.release-4-5-0-pre1.dbcheck_dangling_multi_valued_clean
^samba4.blackbox.dbcheck-links.release-4-5-0-pre1.dangling_multi_valued_check_missing
#
# rap password tests don't function in the ad_dc_ntvfs:local environment
diff --git a/selftest/tests.py b/selftest/tests.py
index 175b56c..a94fc8c 100644
--- a/selftest/tests.py
+++ b/selftest/tests.py
@@ -123,6 +123,11 @@ plantestsuite(
["PYTHON=%s" % python,
os.path.join(bbdir, "dbcheck-links.sh"),
'$PREFIX_ABS/provision', 'release-4-5-0-pre1', configuration])
+plantestsuite(
+ "samba4.blackbox.runtime-links.release-4-5-0-pre1", "none",
+ ["PYTHON=%s" % python,
+ os.path.join(bbdir, "runtime-links.sh"),
+ '$PREFIX_ABS/provision', 'release-4-5-0-pre1', configuration])
planpythontestsuite("none", "samba.tests.upgradeprovision")
planpythontestsuite("none", "samba.tests.xattr")
planpythontestsuite("none", "samba.tests.ntacls")
diff --git a/source4/dsdb/samdb/ldb_modules/extended_dn_store.c b/source4/dsdb/samdb/ldb_modules/extended_dn_store.c
index 28ad9d0..a32ab8d 100644
--- a/source4/dsdb/samdb/ldb_modules/extended_dn_store.c
+++ b/source4/dsdb/samdb/ldb_modules/extended_dn_store.c
@@ -375,6 +375,7 @@ static int extended_dn_modify(struct ldb_module *module, struct ldb_request *req
unsigned int i, j;
--
Samba Shared Repository
More information about the samba-cvs
mailing list