[SCM] Samba Shared Repository - branch master updated
Andrew Bartlett
abartlet at samba.org
Thu Mar 14 03:13:03 UTC 2019
The branch, master has been updated
via a2c5f8cf41c dbcheck: don't check expired tombstone objects by default anymore
via b096a3117ed blackbox/dbcheck-links.sh: prepare regression test for skipping expired tombstones
via 5fccc4e9044 blackbox/dbcheck*.sh: pass --selftest-check-expired-tombstones to dbcheck
via 6f9c5ed8de4 dbcheck: add --selftest-check-expired-tombstones cmdline option
via b61d580fb7d python/samba/netcmd: provide SUPPRESS_HELP via Option class
via a1658b306d8 dbcheck: detect the change after deletion bug
via 1ccc21a34d2 blackbox/dbcheck-links.sh: add regression test for lost deleted object repair
via 598e38d2a5e dbcheck: add find_repl_attid() helper function
via e388e599495 dbcheck: don't remove dangling one-way links on already deleted objects
via 6d50ee74920 dbcheck: don't move already deleted objects to LostAndFound
via 9afcd5331ce dbcheck: do isDeleted, systemFlags and replPropertyMetaData detection first
via 07a8326746f dbcheck: use DSDB_CONTROL_DBCHECK_FIX_LINK_DN_NAME when renaming deleted objects
via 3e8a435d27d dsdb:repl_meta_data: allow CONTROL_DBCHECK_FIX_LINK_DN_NAME to by pass rename
via 5357f591acc blackbox/dbcheck-links.sh: reproduce lost deleted object problem
via 8ba6f1c895e blackbox/*.sh: pass -u to 'diff'
via 3075b85c1ab selftest: Remove unnecessary dns_hub hashmap entries
via 50abab31f8b selftest: Use consistent env variables for dns_hub
via 18ebdc31a5f selftest: Use new helper function for client's smb.conf IP
via 1fb01c5b11f tests: Make IPv4 assumption explicit
via e9c01fdbb8e selftest: Add helper function to get interfaces config
via a3da2f63be4 selftest: Remove secondary client interfaces
via c69092e9bad s4/scripting: remove obsolete w32err_code.py
from a68e8af2d1b testsuite: Remove build_farm testsuites
https://git.samba.org/?p=samba.git;a=shortlog;h=master
- Log -----------------------------------------------------------------
commit a2c5f8cf41c2dfdc4f122e8427d1dfeabb6ba311
Author: Stefan Metzmacher <metze at samba.org>
Date: Tue Mar 12 11:41:01 2019 +0100
dbcheck: don't check expired tombstone objects by default anymore
These will be removed anyway and any change on them risks to
be an originating update that causes replication problems.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13816
Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Thu Mar 14 03:12:27 UTC 2019 on sn-devel-144
commit b096a3117ed9249fd6f65f3221a26c88efbba3b8
Author: Stefan Metzmacher <metze at samba.org>
Date: Tue Mar 12 11:38:22 2019 +0100
blackbox/dbcheck-links.sh: prepare regression test for skipping expired tombstones
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13816
Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
commit 5fccc4e9044d2e57be33471f5e6b9be7cc37ac3a
Author: Stefan Metzmacher <metze at samba.org>
Date: Tue Mar 12 11:04:33 2019 +0100
blackbox/dbcheck*.sh: pass --selftest-check-expired-tombstones to dbcheck
These tests operate on provision dumps created long ago, they still
want to run tests on deleted objects, when the next commits remove
processing expired tombstone objects in dbcheck.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13816
Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
commit 6f9c5ed8de47bb98e21e8064d8e90f963f2f71ca
Author: Stefan Metzmacher <metze at samba.org>
Date: Tue Mar 12 11:02:18 2019 +0100
dbcheck: add --selftest-check-expired-tombstones cmdline option
This will be used by dbcheck tests which operate on static/old provision
dumps in the following commits.
Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
commit b61d580fb7dba8ff94e9e98c958e324865cd2f1d
Author: Stefan Metzmacher <metze at samba.org>
Date: Tue Mar 12 10:25:40 2019 +0100
python/samba/netcmd: provide SUPPRESS_HELP via Option class
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13816
Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
commit a1658b306d85452407388b91a745078c9c1f7dc7
Author: Stefan Metzmacher <metze at samba.org>
Date: Thu Feb 28 18:22:18 2019 +0100
dbcheck: detect the change after deletion bug
Old versions of 'samba-tool dbcheck' could reanimate
deleted objects, when running at the same time as the
tombstone garbage collection.
When the (deleted) parent of a deleted object
(with the DISALLOW_MOVE_ON_DELETE bit in systemFlags),
is removed before the object itself, dbcheck moved
it in the LostAndFound[Config] subtree of the partition
as an originating change. That means that the object
will be in tombstone state again for 180 days on the local
DC. And other DCs fail to replicate the object as
it's already removed completely there and the replication
only gives the name and lastKnownParent attributes, because
all other attributes should already be known to the other DC.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13816
Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
commit 1ccc21a34d295be3bb2ab481a5918003eae88bf4
Author: Stefan Metzmacher <metze at samba.org>
Date: Mon Mar 11 23:14:02 2019 +0100
blackbox/dbcheck-links.sh: add regression test for lost deleted object repair
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13816
Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
commit 598e38d2a5e0832429ba65b4e55bf7127618f894
Author: Stefan Metzmacher <metze at samba.org>
Date: Thu Feb 28 18:16:27 2019 +0100
dbcheck: add find_repl_attid() helper function
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13816
Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
commit e388e599495b6d7c38b8b6966332e27f8b958783
Author: Stefan Metzmacher <metze at samba.org>
Date: Mon Feb 25 15:35:22 2019 +0100
dbcheck: don't remove dangling one-way links on already deleted objects
This would typically happen when the garbage collection
removed a parent object before a child object (both with
the DISALLOW_MOVE_ON_DELETE bit set in systemFlags),
while dbcheck is running at the same time as the garbage collection.
In this case the lastKnownParent attributes points a non existing
object.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13816
Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
commit 6d50ee74920c39cdb18b427bfaaf200775bf2d73
Author: Stefan Metzmacher <metze at samba.org>
Date: Mon Feb 25 15:35:22 2019 +0100
dbcheck: don't move already deleted objects to LostAndFound
This would typically happen when the garbage collection
removed a parent object before a child object (both with
the DISALLOW_MOVE_ON_DELETE bit set in systemFlags),
while dbcheck is running at the same time as the garbage collection.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13816
Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
commit 9afcd5331ce567bd80d35175f8e4e21c506e9347
Author: Stefan Metzmacher <metze at samba.org>
Date: Mon Feb 25 15:09:36 2019 +0100
dbcheck: do isDeleted, systemFlags and replPropertyMetaData detection first
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13816
Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
commit 07a8326746f0c444eedf3860b178fc29d84e8d16
Author: Stefan Metzmacher <metze at samba.org>
Date: Mon Mar 11 22:45:46 2019 +0100
dbcheck: use DSDB_CONTROL_DBCHECK_FIX_LINK_DN_NAME when renaming deleted objects
We should never do originating updates on deleted objects.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13816
Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
commit 3e8a435d27da899d0e3dab7cbc0a1c738067eba3
Author: Stefan Metzmacher <metze at samba.org>
Date: Mon Mar 11 22:38:38 2019 +0100
dsdb:repl_meta_data: allow CONTROL_DBCHECK_FIX_LINK_DN_NAME to by pass rename
We need a way to rename an object without updating the replication meta
data.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13816
Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
commit 5357f591accffbf8c62335c308b985811b66f0b5
Author: Stefan Metzmacher <metze at samba.org>
Date: Mon Mar 11 14:52:57 2019 +0100
blackbox/dbcheck-links.sh: reproduce lost deleted object problem
When a parent object is removed during the tombstone garbage collection
before a child object and samba-tool dbcheck runs at the same time, the
following can happen:
- If the object child had DISALLOW_MOVE_ON_DELETE in systemFlags,
samba-tool dbcheck moves the object under the LostAndFound[Config]
object (as an originating update!)
- The lastKnownParent attribute is removed (as an originating update!)
These originating updates cause the object to have an extended time
as tombstone. And these changes are replicated to other DCs,
which very likely already removed the object completely!
This means the destination DC of replication has no chance to handle
the object it gets from the source DC with just 2 attributes (name, lastKnownParent).
The destination logs something like:
No objectClass found in replPropertyMetaData
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13816
Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
commit 8ba6f1c895ee9b6b592578f21e7f79ed36236bef
Author: Stefan Metzmacher <metze at samba.org>
Date: Tue Mar 12 10:36:49 2019 +0100
blackbox/*.sh: pass -u to 'diff'
This is what we work with every day...
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13816
Signed-off-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
commit 3075b85c1abf63d49613f6dc82b0cc54146628ba
Author: Tim Beale <timbeale at catalyst.net.nz>
Date: Wed Mar 13 10:30:25 2019 +1300
selftest: Remove unnecessary dns_hub hashmap entries
These are only used within the function, and there's already a local
variable that stores the same info.
Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
commit 50abab31f8ba37780c3e111819842ce42f4029c7
Author: Tim Beale <timbeale at catalyst.net.nz>
Date: Tue Mar 12 17:39:50 2019 +1300
selftest: Use consistent env variables for dns_hub
Setting up a testenv involves populating 2 different hashmaps - an
intermediary one (usually called 'ctx') and one that is used to populate
the testenv environment variables (usually called 'env_vars' or
'dcvars').
Because the dns_hub setup is very simple, it doesn't need two different
hashmaps. However, the variable names are still a mix of the two
hashmaps.
This patch updates dns_hub to use the second, more finalized hashmap
variable-names.
Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
commit 18ebdc31a5f396d4bc399539037daa8e713fe726
Author: Tim Beale <timbeale at catalyst.net.nz>
Date: Tue Mar 12 14:04:21 2019 +1300
selftest: Use new helper function for client's smb.conf IP
This has the side-effect of giving the client an IPv6 address, which it
hasn't had up until now. But it at least makes the client and server
interfaces settings consistent, and gets rid of a hard-coded IP address.
Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
commit 1fb01c5b11fe4acfe1178f42cccb24136ae938e5
Author: Tim Beale <timbeale at catalyst.net.nz>
Date: Wed Mar 13 10:18:41 2019 +1300
tests: Make IPv4 assumption explicit
This test is asserting the expected number of *IPv4* addresses, not any
interface address (including IPv6). It works currently because the
selftest client doesn't have an IPv6 address in its smb.conf.
This patch makes the IPv4 assumption explicit by importing
interface_ips_v4() from the provision code. We need to tweak this to
pass through an 'all_interfaces' flag, otherwise it filters out the
loopback IP addresses that the testenv is using.
Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
commit e9c01fdbb8eb1a185f4c45a5928e9d13d46e3aed
Author: Tim Beale <timbeale at catalyst.net.nz>
Date: Tue Mar 12 14:00:55 2019 +1300
selftest: Add helper function to get interfaces config
Add a helper function to return the IPv4/IPv6 addresses for the
smb.conf. This keeps the netmask assumptions in the same places as
the IP subnet assumptions.
This refactor means we no longer need to store $ctx->{interfaces}, as it
was only used in one place.
Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
commit a3da2f63be4bff6ca77b7e1e4d45f42dcf177eef
Author: Tim Beale <timbeale at catalyst.net.nz>
Date: Tue Feb 26 12:14:55 2019 +1300
selftest: Remove secondary client interfaces
I can't see anything in the tests that ever tries to use these other IP
addresses. While it makes sense that we might want the tests to simulate
multiple different clients (with different IPs), we don't appear to do
this currently.
Removing the spare client addresses minimizes the number of hard-coded
IP addresses in selftest.
Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
commit c69092e9bad79252000ea76d119b17dca552d8c8
Author: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Date: Wed Mar 13 10:29:23 2019 +1300
s4/scripting: remove obsolete w32err_code.py
This has been replaced by gen_werror.py which shares common code with other scripts
(e.g. gen_ntstatus) and is more likely to work with conteporary microsoft HTML.
Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
-----------------------------------------------------------------------
Summary of changes:
python/samba/dbchecker.py | 226 +++++++++++--
python/samba/netcmd/__init__.py | 1 +
python/samba/netcmd/dbcheck.py | 10 +-
python/samba/provision/__init__.py | 4 +-
python/samba/tests/join.py | 5 +-
selftest/selftest.pl | 7 +-
selftest/target/Samba.pm | 16 +-
selftest/target/Samba3.pm | 5 +-
selftest/target/Samba4.pm | 22 +-
source4/dsdb/samdb/ldb_modules/repl_meta_data.c | 7 +
source4/scripting/bin/w32err_code.py | 363 ---------------------
...cted-dbcheck-link-output-lost-deleted-user1.txt | 9 +
...cted-dbcheck-link-output-lost-deleted-user2.txt | 8 +
...cted-dbcheck-link-output-lost-deleted-user3.txt | 19 ++
testprogs/blackbox/dbcheck-links.sh | 356 +++++++++++++++++++-
testprogs/blackbox/dbcheck-oldrelease.sh | 42 +--
testprogs/blackbox/tombstones-expunge.sh | 14 +-
17 files changed, 661 insertions(+), 453 deletions(-)
delete mode 100755 source4/scripting/bin/w32err_code.py
create mode 100644 source4/selftest/provisions/release-4-5-0-pre1/expected-dbcheck-link-output-lost-deleted-user1.txt
create mode 100644 source4/selftest/provisions/release-4-5-0-pre1/expected-dbcheck-link-output-lost-deleted-user2.txt
create mode 100644 source4/selftest/provisions/release-4-5-0-pre1/expected-dbcheck-link-output-lost-deleted-user3.txt
Changeset truncated at 500 lines:
diff --git a/python/samba/dbchecker.py b/python/samba/dbchecker.py
index 705be388827..98508192c10 100644
--- a/python/samba/dbchecker.py
+++ b/python/samba/dbchecker.py
@@ -61,7 +61,8 @@ class dbcheck(object):
def __init__(self, samdb, samdb_schema=None, verbose=False, fix=False,
yes=False, quiet=False, in_transaction=False,
quick_membership_checks=False,
- reset_well_known_acls=False):
+ reset_well_known_acls=False,
+ check_expired_tombstones=False):
self.samdb = samdb
self.dict_oid_name = None
self.samdb_schema = (samdb_schema or samdb)
@@ -109,6 +110,8 @@ class dbcheck(object):
self.fix_sid_rid_set_conflict = False
self.quick_membership_checks = quick_membership_checks
self.reset_well_known_acls = reset_well_known_acls
+ self.check_expired_tombstones = check_expired_tombstones
+ self.expired_tombstones = 0
self.reset_all_well_known_acls = False
self.in_transaction = in_transaction
self.infrastructure_dn = ldb.Dn(samdb, "CN=Infrastructure," + samdb.domain_dn())
@@ -122,6 +125,7 @@ class dbcheck(object):
self.fix_missing_deleted_objects = False
self.fix_replica_locations = False
self.fix_missing_rid_set_master = False
+ self.fix_changes_after_deletion_bug = False
self.dn_set = set()
self.link_id_cache = {}
@@ -210,6 +214,14 @@ class dbcheck(object):
else:
self.rid_set_dn = None
+ ntds_service_dn = "CN=Directory Service,CN=Windows NT,CN=Services,%s" % \
+ self.samdb.get_config_basedn().get_linearized()
+ res = samdb.search(base=ntds_service_dn,
+ scope=ldb.SCOPE_BASE,
+ expression="(objectClass=nTDSService)",
+ attrs=["tombstoneLifetime"])
+ self.tombstoneLifetime = int(res[0]["tombstoneLifetime"][0])
+
self.compatibleFeatures = []
self.requiredFeatures = []
@@ -246,6 +258,13 @@ class dbcheck(object):
if DN is None:
error_count += self.check_rootdse()
+ if self.expired_tombstones > 0:
+ self.report("NOTICE: found %d expired tombstones, "
+ "'samba' will remove them daily, "
+ "'samba-tool domain tombstones expunge' "
+ "would do that immediately." % (
+ self.expired_tombstones))
+
if error_count != 0 and not self.fix:
self.report("Please use --fix to fix these errors")
@@ -571,6 +590,19 @@ newSuperior: %s""" % (str(from_dn), str(to_rdn), str(to_base)))
def err_missing_target_dn_or_GUID(self, dn, attrname, val, dsdb_dn):
"""handle a missing target DN (if specified, GUID form can't be found,
and otherwise DN string form can't be found)"""
+
+ # Don't change anything if the object itself is deleted
+ if str(dn).find('\\0ADEL') != -1:
+ # We don't bump the error count as Samba produces these
+ # in normal operation
+ self.report("WARNING: no target object found for GUID "
+ "component link %s in deleted object "
+ "%s - %s" % (attrname, dn, val))
+ self.report("Not removing dangling one-way "
+ "link on deleted object "
+ "(tombstone garbage collection in progress?)")
+ return 0
+
# check if its a backlink
linkID, _ = self.get_attr_linkID_and_reverse_name(attrname)
if (linkID & 1 == 0) and str(dsdb_dn).find('\\0ADEL') == -1:
@@ -880,7 +912,7 @@ newSuperior: %s""" % (str(from_dn), str(to_rdn), str(to_base)))
else:
self.samdb.transaction_cancel()
- def err_wrong_dn(self, obj, new_dn, rdn_attr, rdn_val, name_val):
+ def err_wrong_dn(self, obj, new_dn, rdn_attr, rdn_val, name_val, controls):
'''handle a wrong dn'''
new_rdn = ldb.Dn(self.samdb, str(new_dn))
@@ -897,7 +929,7 @@ newSuperior: %s""" % (str(from_dn), str(to_rdn), str(to_base)))
self.report("Not renaming %s to %s" % (obj.dn, new_dn))
return
- if self.do_rename(obj.dn, new_rdn, new_parent, ["show_recycled:1", "relax:0"],
+ if self.do_rename(obj.dn, new_rdn, new_parent, controls,
"Failed to rename object %s into %s" % (obj.dn, new_dn)):
self.report("Renamed %s into %s" % (obj.dn, new_dn))
@@ -1488,6 +1520,13 @@ newSuperior: %s""" % (str(from_dn), str(to_rdn), str(to_base)))
return error_count
+ def find_repl_attid(self, repl, attid):
+ for o in repl.ctr.array:
+ if o.attid == attid:
+ return o
+
+ return None
+
def get_originating_time(self, val, attid):
'''Read metadata properties and return the originating time for
a given attributeId.
@@ -1496,11 +1535,9 @@ newSuperior: %s""" % (str(from_dn), str(to_rdn), str(to_base)))
'''
repl = ndr_unpack(drsblobs.replPropertyMetaDataBlob, val)
-
- for o in repl.ctr.array:
- if o.attid == attid:
- return o.originating_change_time
-
+ o = self.find_repl_attid(repl, attid)
+ if o is not None:
+ return o.originating_change_time
return 0
def process_metadata(self, dn, val):
@@ -1750,6 +1787,132 @@ newSuperior: %s""" % (str(from_dn), str(to_rdn), str(to_base)))
self.report("Fixed attribute '%s' of '%s'\n" % (sd_attr, dn))
self.samdb.set_session_info(self.system_session_info)
+ def is_expired_tombstone(self, dn, repl_val):
+ if self.check_expired_tombstones:
+ # This is not the default, it's just
+ # used to keep dbcheck tests work with
+ # old static provision dumps
+ return False
+
+ repl = ndr_unpack(drsblobs.replPropertyMetaDataBlob, repl_val)
+
+ isDeleted = self.find_repl_attid(repl, drsuapi.DRSUAPI_ATTID_isDeleted)
+
+ delete_time = samba.nttime2unix(isDeleted.originating_change_time)
+ current_time = time.time()
+
+ tombstone_delta = self.tombstoneLifetime * (24 * 60 * 60)
+
+ delta = current_time - delete_time
+ if delta <= tombstone_delta:
+ return False
+
+ self.report("SKIPING: object %s is an expired tombstone" % dn)
+ self.report("isDeleted: attid=0x%08x version=%d invocation=%s usn=%s (local=%s) at %s" % (
+ isDeleted.attid,
+ isDeleted.version,
+ isDeleted.originating_invocation_id,
+ isDeleted.originating_usn,
+ isDeleted.local_usn,
+ time.ctime(samba.nttime2unix(isDeleted.originating_change_time))))
+ self.expired_tombstones += 1
+ return True
+
+ def find_changes_after_deletion(self, repl_val):
+ repl = ndr_unpack(drsblobs.replPropertyMetaDataBlob, repl_val)
+
+ isDeleted = self.find_repl_attid(repl, drsuapi.DRSUAPI_ATTID_isDeleted)
+
+ delete_time = samba.nttime2unix(isDeleted.originating_change_time)
+
+ tombstone_delta = self.tombstoneLifetime * (24 * 60 * 60)
+
+ found = []
+ for o in repl.ctr.array:
+ if o.attid == drsuapi.DRSUAPI_ATTID_isDeleted:
+ continue
+
+ if o.local_usn <= isDeleted.local_usn:
+ continue
+
+ if o.originating_change_time <= isDeleted.originating_change_time:
+ continue
+
+ change_time = samba.nttime2unix(o.originating_change_time)
+
+ delta = change_time - delete_time
+ if delta <= tombstone_delta:
+ continue
+
+ # If the modification happened after the tombstone lifetime
+ # has passed, we have a bug as the object might be deleted
+ # already on other DCs and won't be able to replicate
+ # back
+ found.append(o)
+
+ return found, isDeleted
+
+ def has_changes_after_deletion(self, dn, repl_val):
+ found, isDeleted = self.find_changes_after_deletion(repl_val)
+ if len(found) == 0:
+ return False
+
+ def report_attid(o):
+ try:
+ attname = self.samdb_schema.get_lDAPDisplayName_by_attid(o.attid)
+ except KeyError:
+ attname = "<unknown:0x%x08x>" % o.attid
+
+ self.report("%s: attid=0x%08x version=%d invocation=%s usn=%s (local=%s) at %s" % (
+ attname, o.attid, o.version,
+ o.originating_invocation_id,
+ o.originating_usn,
+ o.local_usn,
+ time.ctime(samba.nttime2unix(o.originating_change_time))))
+
+ self.report("ERROR: object %s, has changes after deletion" % dn)
+ report_attid(isDeleted)
+ for o in found:
+ report_attid(o)
+
+ return True
+
+ def err_changes_after_deletion(self, dn, repl_val):
+ found, isDeleted = self.find_changes_after_deletion(repl_val)
+
+ in_schema_nc = dn.is_child_of(self.schema_dn)
+ rdn_attr = dn.get_rdn_name()
+ rdn_attid = self.samdb_schema.get_attid_from_lDAPDisplayName(rdn_attr,
+ is_schema_nc=in_schema_nc)
+
+ unexpected = []
+ for o in found:
+ if o.attid == rdn_attid:
+ continue
+ if o.attid == drsuapi.DRSUAPI_ATTID_name:
+ continue
+ if o.attid == drsuapi.DRSUAPI_ATTID_lastKnownParent:
+ continue
+ try:
+ attname = self.samdb_schema.get_lDAPDisplayName_by_attid(o.attid)
+ except KeyError:
+ attname = "<unknown:0x%x08x>" % o.attid
+ unexpected.append(attname)
+
+ if len(unexpected) > 0:
+ self.report('Unexpeted attributes: %s' % ",".join(unexpected))
+ self.report('Not fixing changes after deletion bug')
+ return
+
+ if not self.confirm_all('Delete broken tombstone object %s deleted %s days ago?' % (
+ dn, self.tombstoneLifetime), 'fix_changes_after_deletion_bug'):
+ self.report('Not fixing changes after deletion bug')
+ return
+
+ if self.do_delete(dn, ["relax:0"],
+ "Failed to remove DN %s" % dn):
+ self.report("Removed DN %s" % dn)
+
def has_replmetadata_zero_invocationid(self, dn, repl_meta_data):
repl = ndr_unpack(drsblobs.replPropertyMetaDataBlob,
repl_meta_data)
@@ -2098,7 +2261,6 @@ newSuperior: %s""" % (str(from_dn), str(to_rdn), str(to_base)))
error_count = 0
set_attrs_from_md = set()
set_attrs_seen = set()
- got_repl_property_meta_data = False
got_objectclass = False
nc_dn = self.samdb.get_nc_root(obj.dn)
@@ -2115,6 +2277,26 @@ newSuperior: %s""" % (str(from_dn), str(to_rdn), str(to_base)))
name_val = None
isDeleted = False
systemFlags = 0
+ repl_meta_data_val = None
+
+ for attrname in obj:
+ if str(attrname).lower() == 'isdeleted':
+ if str(obj[attrname][0]) != "FALSE":
+ isDeleted = True
+
+ if str(attrname).lower() == 'systemflags':
+ systemFlags = int(obj[attrname][0])
+
+ if str(attrname).lower() == 'replpropertymetadata':
+ repl_meta_data_val = obj[attrname][0]
+
+ if isDeleted and repl_meta_data_val:
+ if self.has_changes_after_deletion(dn, repl_meta_data_val):
+ error_count += 1
+ self.err_changes_after_deletion(dn, repl_meta_data_val)
+ return error_count
+ if self.is_expired_tombstone(dn, repl_meta_data_val):
+ return error_count
for attrname in obj:
if attrname == 'dn' or attrname == "distinguishedName":
@@ -2140,13 +2322,6 @@ newSuperior: %s""" % (str(from_dn), str(to_rdn), str(to_base)))
else:
object_rdn_val = str(obj[attrname][0])
- if str(attrname).lower() == 'isdeleted':
- if str(obj[attrname][0]) != "FALSE":
- isDeleted = True
-
- if str(attrname).lower() == 'systemflags':
- systemFlags = int(obj[attrname][0])
-
if str(attrname).lower() == 'replpropertymetadata':
if self.has_replmetadata_zero_invocationid(dn, obj[attrname][0]):
error_count += 1
@@ -2176,7 +2351,6 @@ newSuperior: %s""" % (str(from_dn), str(to_rdn), str(to_base)))
self.report("ERROR: Not fixing incorrect initial attributeID in '%s' on '%s', it should be objectClass" %
(attrname, str(dn)))
- got_repl_property_meta_data = True
continue
if str(attrname).lower() == 'ntsecuritydescriptor':
@@ -2335,9 +2509,11 @@ newSuperior: %s""" % (str(from_dn), str(to_rdn), str(to_base)))
if name_val is not None:
parent_dn = None
+ controls = ["show_recycled:1", "relax:0"]
if isDeleted:
if not (systemFlags & samba.dsdb.SYSTEM_FLAG_DISALLOW_MOVE_ON_DELETE):
parent_dn = deleted_objects_dn
+ controls += ["local_oid:%s:1" % dsdb.DSDB_CONTROL_DBCHECK_FIX_LINK_DN_NAME]
if parent_dn is None:
parent_dn = obj.dn.parent()
expected_dn = ldb.Dn(self.samdb, "RDN=RDN,%s" % (parent_dn))
@@ -2348,19 +2524,20 @@ newSuperior: %s""" % (str(from_dn), str(to_rdn), str(to_base)))
if expected_dn != obj.dn:
error_count += 1
- self.err_wrong_dn(obj, expected_dn, object_rdn_attr, object_rdn_val, name_val)
+ self.err_wrong_dn(obj, expected_dn, object_rdn_attr,
+ object_rdn_val, name_val, controls)
elif obj.dn.get_rdn_value() != object_rdn_val:
error_count += 1
self.report("ERROR: Not fixing %s=%r on '%s'" % (object_rdn_attr, object_rdn_val, str(obj.dn)))
show_dn = True
- if got_repl_property_meta_data:
+ if repl_meta_data_val:
if obj.dn == deleted_objects_dn:
isDeletedAttId = 131120
# It's 29/12/9999 at 23:59:59 UTC as specified in MS-ADTS 7.1.1.4.2 Deleted Objects Container
expectedTimeDo = 2650466015990000000
- originating = self.get_originating_time(obj["replPropertyMetaData"][0], isDeletedAttId)
+ originating = self.get_originating_time(repl_meta_data_val, isDeletedAttId)
if originating != expectedTimeDo:
if self.confirm_all("Fix isDeleted originating_change_time on '%s'" % str(dn), 'fix_time_metadata'):
nmsg = ldb.Message()
@@ -2395,8 +2572,13 @@ newSuperior: %s""" % (str(from_dn), str(to_rdn), str(to_base)))
except ldb.LdbError as e11:
(enum, estr) = e11.args
if enum == ldb.ERR_NO_SUCH_OBJECT:
- self.err_missing_parent(obj)
- error_count += 1
+ if isDeleted:
+ self.report("WARNING: parent object not found for %s" % (obj.dn))
+ self.report("Not moving to LostAndFound "
+ "(tombstone garbage collection in progress?)")
+ else:
+ self.err_missing_parent(obj)
+ error_count += 1
else:
raise
diff --git a/python/samba/netcmd/__init__.py b/python/samba/netcmd/__init__.py
index cb22b5dc1b0..54e9107005a 100644
--- a/python/samba/netcmd/__init__.py
+++ b/python/samba/netcmd/__init__.py
@@ -27,6 +27,7 @@ import textwrap
class Option(optparse.Option):
+ SUPPRESS_HELP = optparse.SUPPRESS_HELP
pass
# This help formatter does text wrapping and preserves newlines
diff --git a/python/samba/netcmd/dbcheck.py b/python/samba/netcmd/dbcheck.py
index 9006233bd3d..f0dd85282e3 100644
--- a/python/samba/netcmd/dbcheck.py
+++ b/python/samba/netcmd/dbcheck.py
@@ -79,6 +79,9 @@ class cmd_dbcheck(Command):
default=False, action="store_true"),
Option("-H", "--URL", help="LDB URL for database or target server (defaults to local SAM database)",
type=str, metavar="URL", dest="H"),
+ Option("--selftest-check-expired-tombstones",
+ dest="selftest_check_expired_tombstones", default=False, action="store_true",
+ help=Option.SUPPRESS_HELP), # This is only used by tests
]
def run(self, DN=None, H=None, verbose=False, fix=False, yes=False,
@@ -86,7 +89,9 @@ class cmd_dbcheck(Command):
scope="SUB", credopts=None, sambaopts=None, versionopts=None,
attrs=None, reindex=False, force_modules=False,
quick_membership_checks=False,
- reset_well_known_acls=False, yes_rules=[]):
+ reset_well_known_acls=False,
+ selftest_check_expired_tombstones=False,
+ yes_rules=[]):
lp = sambaopts.get_loadparm()
@@ -139,7 +144,8 @@ class cmd_dbcheck(Command):
fix=fix, yes=yes, quiet=quiet,
in_transaction=started_transaction,
quick_membership_checks=quick_membership_checks,
- reset_well_known_acls=reset_well_known_acls)
+ reset_well_known_acls=reset_well_known_acls,
+ check_expired_tombstones=selftest_check_expired_tombstones)
for option in yes_rules:
if hasattr(chk, option):
diff --git a/python/samba/provision/__init__.py b/python/samba/provision/__init__.py
index 1b7762eb12b..0a3a7b89cb7 100644
--- a/python/samba/provision/__init__.py
+++ b/python/samba/provision/__init__.py
@@ -1853,9 +1853,9 @@ def checksysvolacl(samdb, netlogon, sysvol, domainsid, dnsdomain, domaindn,
direct_db_access)
-def interface_ips_v4(lp):
+def interface_ips_v4(lp, all_interfaces=False):
"""return only IPv4 IPs"""
- ips = samba.interface_ips(lp, False)
+ ips = samba.interface_ips(lp, all_interfaces)
ret = []
for i in ips:
if i.find(':') == -1:
diff --git a/python/samba/tests/join.py b/python/samba/tests/join.py
index 09a102e26af..e0ad34e1e74 100644
--- a/python/samba/tests/join.py
+++ b/python/samba/tests/join.py
@@ -24,6 +24,7 @@ from samba.tests.dns_base import DNSTKeyTest
from samba.join import DCJoinContext
from samba.dcerpc import drsuapi, misc, dns
from samba.credentials import Credentials
+from samba.provision import interface_ips_v4
def get_logger(name="subunit"):
@@ -92,7 +93,7 @@ class JoinTestCase(DNSTKeyTest):
questions.append(q)
# Get expected IPs
- IPs = samba.interface_ips(self.lp)
+ IPs = interface_ips_v4(self.lp, all_interfaces=True)
self.finish_name_packet(p, questions)
(response, response_packet) = self.dns_transaction_tcp(p, host=self.server_ip)
@@ -132,7 +133,7 @@ class JoinTestCase(DNSTKeyTest):
updates = []
# Delete the old expected IPs
- IPs = samba.interface_ips(self.lp)
+ IPs = interface_ips_v4(self.lp, all_interfaces=True)
for IP in IPs[1:]:
if ":" in IP:
r = dns.res_rec()
diff --git a/selftest/selftest.pl b/selftest/selftest.pl
index 773b28439d4..919a9d50177 100755
--- a/selftest/selftest.pl
+++ b/selftest/selftest.pl
@@ -511,12 +511,7 @@ foreach (@opt_include) {
push (@includes, read_test_regexes($_));
}
-my $interfaces = join(',', ("127.0.0.11/8",
- "127.0.0.12/8",
- "127.0.0.13/8",
- "127.0.0.14/8",
- "127.0.0.15/8",
- "127.0.0.16/8"));
+my $interfaces = Samba::get_interfaces_config("client");
my $clientdir = "$prefix_abs/client";
diff --git a/selftest/target/Samba.pm b/selftest/target/Samba.pm
index 3a2386202a6..f7332da893e 100644
--- a/selftest/target/Samba.pm
+++ b/selftest/target/Samba.pm
@@ -437,9 +437,12 @@ sub get_interface($)
localnt4dc9 => 9,
# 10 is spare
- # 11-16 used by selftest.pl for client interfaces
+ # 11 is used by selftest.pl for the client interface
client => 11,
+ # 12-16 have been historically reserved for the client, although
+ # aren't actually used
+
addc_no_nss => 17,
addc_no_ntlm => 18,
idmapadmember => 19,
@@ -502,6 +505,17 @@ sub get_ipv6_addr
return sprintf("fd00:0000:0000:0000:0000:0000:5357:5f%02x", $swiface);
--
Samba Shared Repository
More information about the samba-cvs
mailing list