[SCM] Samba Shared Repository - branch v4-8-test updated
Karolin Seeger
kseeger at samba.org
Thu Mar 28 17:43:03 UTC 2019
The branch, v4-8-test has been updated
via 77b4430bd5e s3:waf: Fix the detection of makdev() macro on Linux
via cf7d657a4d0 dbcheck: use the str() value of the "name" attribute
via a41fa4dd1e9 dbcheck: don't check expired tombstone objects by default anymore
via e0f6e6cff3e blackbox/dbcheck-links.sh: prepare regression test for skipping expired tombstones
via 57f7ec5c1ca blackbox/dbcheck*.sh: pass --selftest-check-expired-tombstones to dbcheck
via b388052af91 dbcheck: add --selftest-check-expired-tombstones cmdline option
via f6f2efd080b python/samba/netcmd: provide SUPPRESS_HELP via Option class
via 42c9e569e81 dbcheck: detect the change after deletion bug
via 08f7f33acb9 blackbox/dbcheck-links.sh: add regression test for lost deleted object repair
via 2272dea483e dbcheck: add find_repl_attid() helper function
via 0473eab6862 dbcheck: don't remove dangling one-way links on already deleted objects
via 0fd3f38c1cf dbcheck: don't move already deleted objects to LostAndFound
via ac900c23b5b dbcheck: do isDeleted, systemFlags and replPropertyMetaData detection first
via 3136a2cc546 dbcheck: use DSDB_CONTROL_DBCHECK_FIX_LINK_DN_NAME when renaming deleted objects
via 9daeafbfec8 dsdb:repl_meta_data: allow CONTROL_DBCHECK_FIX_LINK_DN_NAME to by pass rename
via f91050ee547 blackbox/dbcheck-links.sh: reproduce lost deleted object problem
via 833d543717c selftest: force running with TZ=UTC
via 6da5ef15ec1 python/samba: extra ndr_unpack needs bytes function
via 19a77a10b76 python/samba: PY3 port for ridalloc_exop test to work
from 300d52de7e5 s4:librpc: Fix installation of Samba
https://git.samba.org/?p=samba.git;a=shortlog;h=v4-8-test
- Log -----------------------------------------------------------------
commit 77b4430bd5ed540a8b3b68c814920abe5621b66d
Author: Andreas Schneider <asn at samba.org>
Date: Thu Mar 21 11:55:46 2019 +0100
s3:waf: Fix the detection of makdev() macro on Linux
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13853
Signed-off-by: Andreas Schneider <asn at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>
(cherry picked from commit eace58b539a382c61edd7c2be6fdfab31114719f)
Autobuild-User(v4-8-test): Karolin Seeger <kseeger at samba.org>
Autobuild-Date(v4-8-test): Thu Mar 28 17:42:25 UTC 2019 on sn-devel-144
commit cf7d657a4d04cb3016b9f1c902f767123d3d1c34
Author: Stefan Metzmacher <metze at samba.org>
Date: Tue Mar 19 13:05:16 2019 +0100
dbcheck: use the str() value of the "name" attribute
We do the same with the rdn attribute value
and we need the same logic on both in order to
check they are the same.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13816
Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Noel Power <npower at samba.org>
(cherry picked from commit dd6f0dad218ec1d5aa38ea8aa6848ec81035cb3f)
commit a41fa4dd1e9b1883397cc9dc1b349cc3657830d5
Author: Stefan Metzmacher <metze at samba.org>
Date: Tue Mar 12 11:41:01 2019 +0100
dbcheck: don't check expired tombstone objects by default anymore
These will be removed anyway and any change on them risks to
be an originating update that causes replication problems.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13816
Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Thu Mar 14 03:12:27 UTC 2019 on sn-devel-144
(cherry picked from commit a2c5f8cf41c2dfdc4f122e8427d1dfeabb6ba311)
commit e0f6e6cff3e74c5f8c2f521866f7e4962d988b6f
Author: Stefan Metzmacher <metze at samba.org>
Date: Tue Mar 12 11:38:22 2019 +0100
blackbox/dbcheck-links.sh: prepare regression test for skipping expired tombstones
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13816
Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
(cherry picked from commit b096a3117ed9249fd6f65f3221a26c88efbba3b8)
commit 57f7ec5c1ca16c6e8cfa67ac48bc05cead20e271
Author: Stefan Metzmacher <metze at samba.org>
Date: Tue Mar 12 11:04:33 2019 +0100
blackbox/dbcheck*.sh: pass --selftest-check-expired-tombstones to dbcheck
These tests operate on provision dumps created long ago, they still
want to run tests on deleted objects, when the next commits remove
processing expired tombstone objects in dbcheck.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13816
Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
(cherry picked from commit 5fccc4e9044d2e57be33471f5e6b9be7cc37ac3a)
commit b388052af91a34e5df95ebcffc218c9bd4e1d125
Author: Stefan Metzmacher <metze at samba.org>
Date: Tue Mar 12 11:02:18 2019 +0100
dbcheck: add --selftest-check-expired-tombstones cmdline option
This will be used by dbcheck tests which operate on static/old provision
dumps in the following commits.
Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
(cherry picked from commit 6f9c5ed8de47bb98e21e8064d8e90f963f2f71ca)
commit f6f2efd080b935c1b809da74518ee9b96814b2b0
Author: Stefan Metzmacher <metze at samba.org>
Date: Tue Mar 12 10:25:40 2019 +0100
python/samba/netcmd: provide SUPPRESS_HELP via Option class
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13816
Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
(cherry picked from commit b61d580fb7dba8ff94e9e98c958e324865cd2f1d)
commit 42c9e569e81238956b68f37bd445cc6c76268230
Author: Stefan Metzmacher <metze at samba.org>
Date: Thu Feb 28 18:22:18 2019 +0100
dbcheck: detect the change after deletion bug
Old versions of 'samba-tool dbcheck' could reanimate
deleted objects, when running at the same time as the
tombstone garbage collection.
When the (deleted) parent of a deleted object
(with the DISALLOW_MOVE_ON_DELETE bit in systemFlags),
is removed before the object itself, dbcheck moved
it in the LostAndFound[Config] subtree of the partition
as an originating change. That means that the object
will be in tombstone state again for 180 days on the local
DC. And other DCs fail to replicate the object as
it's already removed completely there and the replication
only gives the name and lastKnownParent attributes, because
all other attributes should already be known to the other DC.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13816
Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
(cherry picked from commit a1658b306d85452407388b91a745078c9c1f7dc7)
commit 08f7f33acb9a739505ed621d962913a4f212a190
Author: Stefan Metzmacher <metze at samba.org>
Date: Mon Mar 11 23:14:02 2019 +0100
blackbox/dbcheck-links.sh: add regression test for lost deleted object repair
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13816
Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
(cherry picked from commit 1ccc21a34d295be3bb2ab481a5918003eae88bf4)
commit 2272dea483ecb0d43cc4d155b92e34a07a49146d
Author: Stefan Metzmacher <metze at samba.org>
Date: Thu Feb 28 18:16:27 2019 +0100
dbcheck: add find_repl_attid() helper function
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13816
Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
(cherry picked from commit 598e38d2a5e0832429ba65b4e55bf7127618f894)
commit 0473eab6862b41a68e980d451f3008568ce634f2
Author: Stefan Metzmacher <metze at samba.org>
Date: Mon Feb 25 15:35:22 2019 +0100
dbcheck: don't remove dangling one-way links on already deleted objects
This would typically happen when the garbage collection
removed a parent object before a child object (both with
the DISALLOW_MOVE_ON_DELETE bit set in systemFlags),
while dbcheck is running at the same time as the garbage collection.
In this case the lastKnownParent attributes points a non existing
object.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13816
Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
(cherry picked from commit e388e599495b6d7c38b8b6966332e27f8b958783)
commit 0fd3f38c1cfa717c1e9a2d320030ee29102e5005
Author: Stefan Metzmacher <metze at samba.org>
Date: Mon Feb 25 15:35:22 2019 +0100
dbcheck: don't move already deleted objects to LostAndFound
This would typically happen when the garbage collection
removed a parent object before a child object (both with
the DISALLOW_MOVE_ON_DELETE bit set in systemFlags),
while dbcheck is running at the same time as the garbage collection.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13816
Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
(cherry picked from commit 6d50ee74920c39cdb18b427bfaaf200775bf2d73)
commit ac900c23b5bc2fa8fe1bbb0b6f195fcd43ec491f
Author: Stefan Metzmacher <metze at samba.org>
Date: Mon Feb 25 15:09:36 2019 +0100
dbcheck: do isDeleted, systemFlags and replPropertyMetaData detection first
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13816
Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
(cherry picked from commit 9afcd5331ce567bd80d35175f8e4e21c506e9347)
commit 3136a2cc546e5674a810584323944246d7466855
Author: Stefan Metzmacher <metze at samba.org>
Date: Mon Mar 11 22:45:46 2019 +0100
dbcheck: use DSDB_CONTROL_DBCHECK_FIX_LINK_DN_NAME when renaming deleted objects
We should never do originating updates on deleted objects.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13816
Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
(cherry picked from commit 07a8326746f0c444eedf3860b178fc29d84e8d16)
commit 9daeafbfec8bde7a4c2e0057fb4f06cb54624cd2
Author: Stefan Metzmacher <metze at samba.org>
Date: Mon Mar 11 22:38:38 2019 +0100
dsdb:repl_meta_data: allow CONTROL_DBCHECK_FIX_LINK_DN_NAME to by pass rename
We need a way to rename an object without updating the replication meta
data.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13816
Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
(cherry picked from commit 3e8a435d27da899d0e3dab7cbc0a1c738067eba3)
commit f91050ee547ff645933da1a3ceeecf3d1052045e
Author: Stefan Metzmacher <metze at samba.org>
Date: Mon Mar 11 14:52:57 2019 +0100
blackbox/dbcheck-links.sh: reproduce lost deleted object problem
When a parent object is removed during the tombstone garbage collection
before a child object and samba-tool dbcheck runs at the same time, the
following can happen:
- If the object child had DISALLOW_MOVE_ON_DELETE in systemFlags,
samba-tool dbcheck moves the object under the LostAndFound[Config]
object (as an originating update!)
- The lastKnownParent attribute is removed (as an originating update!)
These originating updates cause the object to have an extended time
as tombstone. And these changes are replicated to other DCs,
which very likely already removed the object completely!
This means the destination DC of replication has no chance to handle
the object it gets from the source DC with just 2 attributes (name, lastKnownParent).
The destination logs something like:
No objectClass found in replPropertyMetaData
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13816
Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
(cherry picked from commit 5357f591accffbf8c62335c308b985811b66f0b5)
commit 833d543717ca7f59270a384397dd5a15d3ab79ae
Author: Stefan Metzmacher <metze at samba.org>
Date: Wed Feb 27 08:22:09 2019 +0100
selftest: force running with TZ=UTC
Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andreas Schneider <asn at samba.org>
Autobuild-User(master): Stefan Metzmacher <metze at samba.org>
Autobuild-Date(master): Wed Feb 27 11:24:59 UTC 2019 on sn-devel-144
(cherry picked from commit 4f307f2302b0fe8fd0fc6379eb8e6491faf8520c)
commit 6da5ef15ec104d87530ec4fe4765e98ee4288dfe
Author: Noel Power <noel.power at suse.com>
Date: Fri Sep 7 14:38:54 2018 +0100
python/samba: extra ndr_unpack needs bytes function
(cherry picked from commit 8db43696e70d7c4cb21172b7e7461cf6a72914a2)
commit 19a77a10b76c09d86000881cafd031bbcffd2985
Author: Noel Power <noel.power at suse.com>
Date: Fri Sep 7 12:42:19 2018 +0100
python/samba: PY3 port for ridalloc_exop test to work
Signed-off-by: Noel Power <noel.power at suse.com>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
(cherry picked from commit fc13a1268a4a9de94efd312a8309aa55d331ae19)
-----------------------------------------------------------------------
Summary of changes:
python/samba/dbchecker.py | 234 ++++++++++++--
python/samba/netcmd/__init__.py | 1 +
python/samba/netcmd/dbcheck.py | 15 +-
python/samba/remove_dc.py | 4 +-
selftest/selftest.pl | 3 +
source3/wscript | 3 +
source4/dsdb/samdb/ldb_modules/repl_meta_data.c | 7 +
...cted-dbcheck-link-output-lost-deleted-user1.txt | 9 +
...cted-dbcheck-link-output-lost-deleted-user2.txt | 8 +
...cted-dbcheck-link-output-lost-deleted-user3.txt | 19 ++
testprogs/blackbox/dbcheck-links.sh | 342 ++++++++++++++++++++-
testprogs/blackbox/dbcheck-oldrelease.sh | 14 +-
12 files changed, 611 insertions(+), 48 deletions(-)
create mode 100644 source4/selftest/provisions/release-4-5-0-pre1/expected-dbcheck-link-output-lost-deleted-user1.txt
create mode 100644 source4/selftest/provisions/release-4-5-0-pre1/expected-dbcheck-link-output-lost-deleted-user2.txt
create mode 100644 source4/selftest/provisions/release-4-5-0-pre1/expected-dbcheck-link-output-lost-deleted-user3.txt
Changeset truncated at 500 lines:
diff --git a/python/samba/dbchecker.py b/python/samba/dbchecker.py
index 2619b9bc72e..f17ff39ae02 100644
--- a/python/samba/dbchecker.py
+++ b/python/samba/dbchecker.py
@@ -40,7 +40,8 @@ class dbcheck(object):
def __init__(self, samdb, samdb_schema=None, verbose=False, fix=False,
yes=False, quiet=False, in_transaction=False,
- reset_well_known_acls=False):
+ reset_well_known_acls=False,
+ check_expired_tombstones=False):
self.samdb = samdb
self.dict_oid_name = None
self.samdb_schema = (samdb_schema or samdb)
@@ -87,6 +88,8 @@ class dbcheck(object):
self.fix_doubled_userparameters = False
self.fix_sid_rid_set_conflict = False
self.reset_well_known_acls = reset_well_known_acls
+ self.check_expired_tombstones = check_expired_tombstones
+ self.expired_tombstones = 0
self.reset_all_well_known_acls = False
self.in_transaction = in_transaction
self.infrastructure_dn = ldb.Dn(samdb, "CN=Infrastructure," + samdb.domain_dn())
@@ -100,6 +103,7 @@ class dbcheck(object):
self.fix_missing_deleted_objects = False
self.fix_replica_locations = False
self.fix_missing_rid_set_master = False
+ self.fix_changes_after_deletion_bug = False
self.dn_set = set()
self.link_id_cache = {}
@@ -187,6 +191,14 @@ class dbcheck(object):
else:
self.rid_set_dn = None
+ ntds_service_dn = "CN=Directory Service,CN=Windows NT,CN=Services,%s" % \
+ self.samdb.get_config_basedn().get_linearized()
+ res = samdb.search(base=ntds_service_dn,
+ scope=ldb.SCOPE_BASE,
+ expression="(objectClass=nTDSService)",
+ attrs=["tombstoneLifetime"])
+ self.tombstoneLifetime = int(res[0]["tombstoneLifetime"][0])
+
self.compatibleFeatures = []
self.requiredFeatures = []
@@ -221,6 +233,13 @@ class dbcheck(object):
if DN is None:
error_count += self.check_rootdse()
+ if self.expired_tombstones > 0:
+ self.report("NOTICE: found %d expired tombstones, "
+ "'samba' will remove them daily, "
+ "'samba-tool domain tombstones expunge' "
+ "would do that immediately." % (
+ self.expired_tombstones))
+
if error_count != 0 and not self.fix:
self.report("Please use --fix to fix these errors")
@@ -544,6 +563,19 @@ newSuperior: %s""" % (str(from_dn), str(to_rdn), str(to_base)))
def err_missing_target_dn_or_GUID(self, dn, attrname, val, dsdb_dn):
"""handle a missing target DN (if specified, GUID form can't be found,
and otherwise DN string form can't be found)"""
+
+ # Don't change anything if the object itself is deleted
+ if str(dn).find('\\0ADEL') != -1:
+ # We don't bump the error count as Samba produces these
+ # in normal operation
+ self.report("WARNING: no target object found for GUID "
+ "component link %s in deleted object "
+ "%s - %s" % (attrname, dn, val))
+ self.report("Not removing dangling one-way "
+ "link on deleted object "
+ "(tombstone garbage collection in progress?)")
+ return 0
+
# check if its a backlink
linkID, _ = self.get_attr_linkID_and_reverse_name(attrname)
if (linkID & 1 == 0) and str(dsdb_dn).find('\\0ADEL') == -1:
@@ -853,7 +885,7 @@ newSuperior: %s""" % (str(from_dn), str(to_rdn), str(to_base)))
else:
self.samdb.transaction_cancel()
- def err_wrong_dn(self, obj, new_dn, rdn_attr, rdn_val, name_val):
+ def err_wrong_dn(self, obj, new_dn, rdn_attr, rdn_val, name_val, controls):
'''handle a wrong dn'''
new_rdn = ldb.Dn(self.samdb, str(new_dn))
@@ -870,7 +902,7 @@ newSuperior: %s""" % (str(from_dn), str(to_rdn), str(to_base)))
self.report("Not renaming %s to %s" % (obj.dn, new_dn))
return
- if self.do_rename(obj.dn, new_rdn, new_parent, ["show_recycled:1", "relax:0"],
+ if self.do_rename(obj.dn, new_rdn, new_parent, controls,
"Failed to rename object %s into %s" % (obj.dn, new_dn)):
self.report("Renamed %s into %s" % (obj.dn, new_dn))
@@ -1441,6 +1473,12 @@ newSuperior: %s""" % (str(from_dn), str(to_rdn), str(to_base)))
return error_count
+ def find_repl_attid(self, repl, attid):
+ for o in repl.ctr.array:
+ if o.attid == attid:
+ return o
+
+ return None
def get_originating_time(self, val, attid):
'''Read metadata properties and return the originating time for
@@ -1450,12 +1488,9 @@ newSuperior: %s""" % (str(from_dn), str(to_rdn), str(to_base)))
'''
repl = ndr_unpack(drsblobs.replPropertyMetaDataBlob, str(val))
- obj = repl.ctr
-
- for o in repl.ctr.array:
- if o.attid == attid:
- return o.originating_change_time
-
+ o = self.find_repl_attid(repl, attid)
+ if o is not None:
+ return o.originating_change_time
return 0
def process_metadata(self, dn, val):
@@ -1708,6 +1743,131 @@ newSuperior: %s""" % (str(from_dn), str(to_rdn), str(to_base)))
self.report("Fixed attribute '%s' of '%s'\n" % (sd_attr, dn))
self.samdb.set_session_info(self.system_session_info)
+ def is_expired_tombstone(self, dn, repl_val):
+ if self.check_expired_tombstones:
+ # This is not the default, it's just
+ # used to keep dbcheck tests work with
+ # old static provision dumps
+ return False
+
+ repl = ndr_unpack(drsblobs.replPropertyMetaDataBlob, repl_val)
+
+ isDeleted = self.find_repl_attid(repl, drsuapi.DRSUAPI_ATTID_isDeleted)
+
+ delete_time = samba.nttime2unix(isDeleted.originating_change_time)
+ current_time = time.time()
+
+ tombstone_delta = self.tombstoneLifetime * (24 * 60 * 60)
+
+ delta = current_time - delete_time
+ if delta <= tombstone_delta:
+ return False
+
+ self.report("SKIPING: object %s is an expired tombstone" % dn)
+ self.report("isDeleted: attid=0x%08x version=%d invocation=%s usn=%s (local=%s) at %s" % (
+ isDeleted.attid,
+ isDeleted.version,
+ isDeleted.originating_invocation_id,
+ isDeleted.originating_usn,
+ isDeleted.local_usn,
+ time.ctime(samba.nttime2unix(isDeleted.originating_change_time))))
+ self.expired_tombstones += 1
+ return True
+
+ def find_changes_after_deletion(self, repl_val):
+ repl = ndr_unpack(drsblobs.replPropertyMetaDataBlob, repl_val)
+
+ isDeleted = self.find_repl_attid(repl, drsuapi.DRSUAPI_ATTID_isDeleted)
+
+ delete_time = samba.nttime2unix(isDeleted.originating_change_time)
+
+ tombstone_delta = self.tombstoneLifetime * (24 * 60 * 60)
+
+ found = []
+ for o in repl.ctr.array:
+ if o.attid == drsuapi.DRSUAPI_ATTID_isDeleted:
+ continue
+
+ if o.local_usn <= isDeleted.local_usn:
+ continue
+
+ if o.originating_change_time <= isDeleted.originating_change_time:
+ continue
+
+ change_time = samba.nttime2unix(o.originating_change_time)
+
+ delta = change_time - delete_time
+ if delta <= tombstone_delta:
+ continue
+
+ # If the modification happened after the tombstone lifetime
+ # has passed, we have a bug as the object might be deleted
+ # already on other DCs and won't be able to replicate
+ # back
+ found.append(o)
+
+ return found, isDeleted
+
+ def has_changes_after_deletion(self, dn, repl_val):
+ found, isDeleted = self.find_changes_after_deletion(repl_val)
+ if len(found) == 0:
+ return False
+
+ def report_attid(o):
+ try:
+ attname = self.samdb_schema.get_lDAPDisplayName_by_attid(o.attid)
+ except KeyError:
+ attname = "<unknown:0x%x08x>" % o.attid
+
+ self.report("%s: attid=0x%08x version=%d invocation=%s usn=%s (local=%s) at %s" % (
+ attname, o.attid, o.version,
+ o.originating_invocation_id,
+ o.originating_usn,
+ o.local_usn,
+ time.ctime(samba.nttime2unix(o.originating_change_time))))
+
+ self.report("ERROR: object %s, has changes after deletion" % dn)
+ report_attid(isDeleted)
+ for o in found:
+ report_attid(o)
+
+ return True
+
+ def err_changes_after_deletion(self, dn, repl_val):
+ found, isDeleted = self.find_changes_after_deletion(repl_val)
+
+ in_schema_nc = dn.is_child_of(self.schema_dn)
+ rdn_attr = dn.get_rdn_name()
+ rdn_attid = self.samdb_schema.get_attid_from_lDAPDisplayName(rdn_attr,
+ is_schema_nc=in_schema_nc)
+
+ unexpected = []
+ for o in found:
+ if o.attid == rdn_attid:
+ continue
+ if o.attid == drsuapi.DRSUAPI_ATTID_name:
+ continue
+ if o.attid == drsuapi.DRSUAPI_ATTID_lastKnownParent:
+ continue
+ try:
+ attname = self.samdb_schema.get_lDAPDisplayName_by_attid(o.attid)
+ except KeyError:
+ attname = "<unknown:0x%x08x>" % o.attid
+ unexpected.append(attname)
+
+ if len(unexpected) > 0:
+ self.report('Unexpeted attributes: %s' % ",".join(unexpected))
+ self.report('Not fixing changes after deletion bug')
+ return
+
+ if not self.confirm_all('Delete broken tombstone object %s deleted %s days ago?' % (
+ dn, self.tombstoneLifetime), 'fix_changes_after_deletion_bug'):
+ self.report('Not fixing changes after deletion bug')
+ return
+
+ if self.do_delete(dn, ["relax:0"],
+ "Failed to remove DN %s" % dn):
+ self.report("Removed DN %s" % dn)
def has_replmetadata_zero_invocationid(self, dn, repl_meta_data):
repl = ndr_unpack(drsblobs.replPropertyMetaDataBlob,
@@ -1979,8 +2139,7 @@ newSuperior: %s""" % (str(from_dn), str(to_rdn), str(to_base)))
raise
else:
instancetype |= dsdb.INSTANCE_TYPE_NC_ABOVE
-
- if self.write_ncs is not None and str(nc_root) in self.write_ncs:
+ if self.write_ncs is not None and str(nc_root) in [str(x) for x in self.write_ncs]:
instancetype |= dsdb.INSTANCE_TYPE_WRITE
return instancetype
@@ -2059,7 +2218,6 @@ newSuperior: %s""" % (str(from_dn), str(to_rdn), str(to_base)))
error_count = 0
set_attrs_from_md = set()
set_attrs_seen = set()
- got_repl_property_meta_data = False
got_objectclass = False
nc_dn = self.samdb.get_nc_root(obj.dn)
@@ -2077,6 +2235,26 @@ newSuperior: %s""" % (str(from_dn), str(to_rdn), str(to_base)))
name_val = None
isDeleted = False
systemFlags = 0
+ repl_meta_data_val = None
+
+ for attrname in obj:
+ if str(attrname).lower() == 'isdeleted':
+ if str(obj[attrname][0]) != "FALSE":
+ isDeleted = True
+
+ if str(attrname).lower() == 'systemflags':
+ systemFlags = int(obj[attrname][0])
+
+ if str(attrname).lower() == 'replpropertymetadata':
+ repl_meta_data_val = obj[attrname][0]
+
+ if isDeleted and repl_meta_data_val:
+ if self.has_changes_after_deletion(dn, repl_meta_data_val):
+ error_count += 1
+ self.err_changes_after_deletion(dn, repl_meta_data_val)
+ return error_count
+ if self.is_expired_tombstone(dn, repl_meta_data_val):
+ return error_count
for attrname in obj:
if attrname == 'dn' or attrname == "distinguishedName":
@@ -2091,7 +2269,7 @@ newSuperior: %s""" % (str(from_dn), str(to_rdn), str(to_base)))
self.report("ERROR: Not fixing num_values(%d) for '%s' on '%s'" %
(len(obj[attrname]), attrname, str(obj.dn)))
else:
- name_val = obj[attrname][0]
+ name_val = str(obj[attrname][0])
if str(attrname).lower() == str(obj.dn.get_rdn_name()).lower():
object_rdn_attr = attrname
@@ -2100,14 +2278,7 @@ newSuperior: %s""" % (str(from_dn), str(to_rdn), str(to_base)))
self.report("ERROR: Not fixing num_values(%d) for '%s' on '%s'" %
(len(obj[attrname]), attrname, str(obj.dn)))
else:
- object_rdn_val = obj[attrname][0]
-
- if str(attrname).lower() == 'isdeleted':
- if obj[attrname][0] != "FALSE":
- isDeleted = True
-
- if str(attrname).lower() == 'systemflags':
- systemFlags = int(obj[attrname][0])
+ object_rdn_val = str(obj[attrname][0])
if str(attrname).lower() == 'replpropertymetadata':
if self.has_replmetadata_zero_invocationid(dn, obj[attrname]):
@@ -2138,7 +2309,6 @@ newSuperior: %s""" % (str(from_dn), str(to_rdn), str(to_base)))
self.report("ERROR: Not fixing incorrect inital attributeID in '%s' on '%s', it should be objectClass" %
(attrname, str(dn)))
- got_repl_property_meta_data = True
continue
if str(attrname).lower() == 'ntsecuritydescriptor':
@@ -2278,7 +2448,7 @@ newSuperior: %s""" % (str(from_dn), str(to_rdn), str(to_base)))
if str(attrname).lower() == "instancetype":
calculated_instancetype = self.calculate_instancetype(dn)
- if len(obj["instanceType"]) != 1 or obj["instanceType"][0] != str(calculated_instancetype):
+ if len(obj["instanceType"]) != 1 or int(obj["instanceType"][0]) != calculated_instancetype:
error_count += 1
self.err_wrong_instancetype(obj, calculated_instancetype)
@@ -2296,9 +2466,11 @@ newSuperior: %s""" % (str(from_dn), str(to_rdn), str(to_base)))
if name_val is not None:
parent_dn = None
+ controls = ["show_recycled:1", "relax:0"]
if isDeleted:
if not (systemFlags & samba.dsdb.SYSTEM_FLAG_DISALLOW_MOVE_ON_DELETE):
parent_dn = deleted_objects_dn
+ controls += ["local_oid:%s:1" % dsdb.DSDB_CONTROL_DBCHECK_FIX_LINK_DN_NAME]
if parent_dn is None:
parent_dn = obj.dn.parent()
expected_dn = ldb.Dn(self.samdb, "RDN=RDN,%s" % (parent_dn))
@@ -2309,19 +2481,20 @@ newSuperior: %s""" % (str(from_dn), str(to_rdn), str(to_base)))
if expected_dn != obj.dn:
error_count += 1
- self.err_wrong_dn(obj, expected_dn, object_rdn_attr, object_rdn_val, name_val)
+ self.err_wrong_dn(obj, expected_dn, object_rdn_attr,
+ object_rdn_val, name_val, controls)
elif obj.dn.get_rdn_value() != object_rdn_val:
error_count += 1
self.report("ERROR: Not fixing %s=%r on '%s'" % (object_rdn_attr, object_rdn_val, str(obj.dn)))
show_dn = True
- if got_repl_property_meta_data:
+ if repl_meta_data_val:
if obj.dn == deleted_objects_dn:
isDeletedAttId = 131120
# It's 29/12/9999 at 23:59:59 UTC as specified in MS-ADTS 7.1.1.4.2 Deleted Objects Container
expectedTimeDo = 2650466015990000000
- originating = self.get_originating_time(obj["replPropertyMetaData"], isDeletedAttId)
+ originating = self.get_originating_time(repl_meta_data_val, isDeletedAttId)
if originating != expectedTimeDo:
if self.confirm_all("Fix isDeleted originating_change_time on '%s'" % str(dn), 'fix_time_metadata'):
nmsg = ldb.Message()
@@ -2355,8 +2528,13 @@ newSuperior: %s""" % (str(from_dn), str(to_rdn), str(to_base)))
controls=["show_recycled:1", "show_deleted:1"])
except ldb.LdbError, (enum, estr):
if enum == ldb.ERR_NO_SUCH_OBJECT:
- self.err_missing_parent(obj)
- error_count += 1
+ if isDeleted:
+ self.report("WARNING: parent object not found for %s" % (obj.dn))
+ self.report("Not moving to LostAndFound "
+ "(tombstone garbage collection in progress?)")
+ else:
+ self.err_missing_parent(obj)
+ error_count += 1
else:
raise
diff --git a/python/samba/netcmd/__init__.py b/python/samba/netcmd/__init__.py
index 05ecc432ea6..9dd6748447f 100644
--- a/python/samba/netcmd/__init__.py
+++ b/python/samba/netcmd/__init__.py
@@ -23,6 +23,7 @@ import sys, traceback
import textwrap
class Option(optparse.Option):
+ SUPPRESS_HELP = optparse.SUPPRESS_HELP
pass
# This help formatter does text wrapping and preserves newlines
diff --git a/python/samba/netcmd/dbcheck.py b/python/samba/netcmd/dbcheck.py
index 2217366cc8d..57030274cad 100644
--- a/python/samba/netcmd/dbcheck.py
+++ b/python/samba/netcmd/dbcheck.py
@@ -74,13 +74,18 @@ class cmd_dbcheck(Command):
Option("--reset-well-known-acls", dest="reset_well_known_acls", default=False, action="store_true", help="reset ACLs on objects with well known default ACL values to the default"),
Option("-H", "--URL", help="LDB URL for database or target server (defaults to local SAM database)",
type=str, metavar="URL", dest="H"),
- ]
+ Option("--selftest-check-expired-tombstones",
+ dest="selftest_check_expired_tombstones", default=False, action="store_true",
+ help=Option.SUPPRESS_HELP), # This is only used by tests
+ ]
def run(self, DN=None, H=None, verbose=False, fix=False, yes=False,
cross_ncs=False, quiet=False,
scope="SUB", credopts=None, sambaopts=None, versionopts=None,
attrs=None, reindex=False, force_modules=False,
- reset_well_known_acls=False, yes_rules=[]):
+ reset_well_known_acls=False,
+ selftest_check_expired_tombstones=False,
+ yes_rules=[]):
lp = sambaopts.get_loadparm()
@@ -131,8 +136,10 @@ class cmd_dbcheck(Command):
started_transaction = True
try:
chk = dbcheck(samdb, samdb_schema=samdb_schema, verbose=verbose,
- fix=fix, yes=yes, quiet=quiet, in_transaction=started_transaction,
- reset_well_known_acls=reset_well_known_acls)
+ fix=fix, yes=yes, quiet=quiet,
+ in_transaction=started_transaction,
+ reset_well_known_acls=reset_well_known_acls,
+ check_expired_tombstones=selftest_check_expired_tombstones)
for option in yes_rules:
if hasattr(chk, option):
diff --git a/python/samba/remove_dc.py b/python/samba/remove_dc.py
index 4c8ee892464..84360ed1503 100644
--- a/python/samba/remove_dc.py
+++ b/python/samba/remove_dc.py
@@ -221,7 +221,7 @@ def offline_remove_server(samdb, logger,
computer_dn = None
try:
- dnsHostName = msgs[0]["dnsHostName"][0]
+ dnsHostName = str(msgs[0]["dnsHostName"][0])
except KeyError:
dnsHostName = None
@@ -251,7 +251,7 @@ def offline_remove_server(samdb, logger,
samdb.delete(computer_dn, ["tree_delete:0"])
if "dnsHostName" in msgs[0]:
- dnsHostName = msgs[0]["dnsHostName"][0]
+ dnsHostName = str(msgs[0]["dnsHostName"][0])
if remove_dns_account:
res = samdb.search(expression="(&(objectclass=user)(cn=dns-%s)(servicePrincipalName=DNS/%s))" %
diff --git a/selftest/selftest.pl b/selftest/selftest.pl
index 0e56e6a13ef..e35e67798ed 100755
--- a/selftest/selftest.pl
+++ b/selftest/selftest.pl
@@ -286,6 +286,9 @@ unless (defined($ENV{VALGRIND})) {
# make all our python scripts unbuffered
$ENV{PYTHONUNBUFFERED} = 1;
+# do not depend on the users setup
+$ENV{TZ} = "UTC";
+
my $bindir_abs = abs_path($bindir);
# Backwards compatibility:
diff --git a/source3/wscript b/source3/wscript
index 799ee08dbb5..a32899d89cd 100644
--- a/source3/wscript
+++ b/source3/wscript
@@ -1232,6 +1232,9 @@ main() {
#include <unistd.h>
#endif
#include <sys/types.h>
+#if defined(HAVE_SYS_SYSMACROS_H)
+#include <sys/sysmacros.h>
+#endif
main() { dev_t dev = makedev(1,2); return 0; }
''',
'HAVE_MAKEDEV',
diff --git a/source4/dsdb/samdb/ldb_modules/repl_meta_data.c b/source4/dsdb/samdb/ldb_modules/repl_meta_data.c
index c2eafd0a521..b8e03d7ca15 100644
--
Samba Shared Repository
More information about the samba-cvs
mailing list