[SCM] Samba Shared Repository - branch master updated
Garming Sam
garming at samba.org
Thu Sep 1 07:39:06 UTC 2016
The branch, master has been updated
via f479b1b gc_tombstones: Typo fix
via b1ed0f4 dsdb: Do not check isDeleted as a possible link
via 737756b lib/ldb-samba: Add test for DSDB_MATCH_FOR_EXPUNGE match rule
via d276735 dsdb: Add comments to dsdb_garbage_collect_tombstones()
via 5620616 samba-tool: Run samba-tool domain tombstones expunge in a transaction
via a999e8c selftest: Add test for 'samba-tool tombstones expunge'
via d66deaa dsdb: Use a date comparison in the search to avoid returning all deleted objects
via 2400389 dsdb: Expose ldb error string to dsdb_garbage_collect_tombstones() callers
via 55b9b9a samba-tool: Add command-line tool to trigger tombstone expunge
via 02d82af python: Add binding for dsdb_garbage_collect_tombstones()
via 1c63653 dsdb: Expand garbage_collect_tombstones to expunge links also
via 824220e dsdb: move tombstone lifetime calculation out of dsdb_garbage_collect_tombstones()
via d894f08 dsdb: Rework more KCC service-specific details out of dsdb_garbage_collect_tombstones()
via 4e0ac09 dsdb: Rework kcc_deleted() into dsdb_garbage_collect_tombstones()
via 9221ce3 kcc: Move kcc/kcc_deleted.c into kcc/garbage_collect_tombstones.c
via cf6bc40 ldb: Fix comment on ldb_register_extended_match_rule
via eb1cb17 ldb-samba: Add new extended match rule DSDB_MATCH_FOR_EXPUNGE
via 9142a01 pydsdb: Raise TypeError for type errors, rather than incorrectly raising an LdbError
via 599d3f7 selftest: Correct name of samba4.blackbox.dbcheck.release-4-5-0-pre1
from 7d33761 tevent: Fix some typos
https://git.samba.org/?p=samba.git;a=shortlog;h=master
- Log -----------------------------------------------------------------
commit f479b1b3fd787fd2cbedc6161a4e0eb0af1059e3
Author: Garming Sam <garming at catalyst.net.nz>
Date: Thu Sep 1 14:26:53 2016 +1200
gc_tombstones: Typo fix
Signed-off-by: Garming Sam <garming at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglasbagnall at catalyst.net.nz>
Autobuild-User(master): Garming Sam <garming at samba.org>
Autobuild-Date(master): Thu Sep 1 09:38:47 CEST 2016 on sn-devel-144
commit b1ed0f4597e5ebd059d33b0a33de4ded96c9386f
Author: Andrew Bartlett <abartlet at samba.org>
Date: Thu Sep 1 14:17:27 2016 +1200
dsdb: Do not check isDeleted as a possible link
Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Garming Sam <garming at catalyst.net.nz>
commit 737756b060364c09b62a500ef26eef044230ecac
Author: Andrew Bartlett <abartlet at samba.org>
Date: Thu Sep 1 13:34:12 2016 +1200
lib/ldb-samba: Add test for DSDB_MATCH_FOR_EXPUNGE match rule
Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Garming Sam <garming at catalyst.net.nz>
commit d27673587707f2dc59bd5161cd70ca96118d5ceb
Author: Andrew Bartlett <abartlet at samba.org>
Date: Thu Sep 1 13:23:11 2016 +1200
dsdb: Add comments to dsdb_garbage_collect_tombstones()
Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Garming Sam <garming at catalyst.net.nz>
commit 5620616968bd68b80d872079ad3bbb97ac1e7a8a
Author: Andrew Bartlett <abartlet at samba.org>
Date: Thu Sep 1 12:13:40 2016 +1200
samba-tool: Run samba-tool domain tombstones expunge in a transaction
Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Garming Sam <garming at catalyst.net.nz>
commit a999e8c0cbbfcb59caf4732c952e3e9856ea7577
Author: Andrew Bartlett <abartlet at samba.org>
Date: Wed Aug 31 17:07:29 2016 +1200
selftest: Add test for 'samba-tool tombstones expunge'
Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Garming Sam <garming at catalyst.net.nz>
commit d66deaabc6faca769e9e18098eef4729f3dad379
Author: Andrew Bartlett <abartlet at samba.org>
Date: Tue Aug 30 10:30:51 2016 +1200
dsdb: Use a date comparison in the search to avoid returning all deleted objects
Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Garming Sam <garming at catalyst.net.nz>
commit 240038979d748b830b788753b38c3cd576eafe30
Author: Andrew Bartlett <abartlet at samba.org>
Date: Tue Aug 30 10:22:47 2016 +1200
dsdb: Expose ldb error string to dsdb_garbage_collect_tombstones() callers
Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Garming Sam <garming at catalyst.net.nz>
commit 55b9b9a969b0e7ef6590710fda85265fc3146159
Author: Andrew Bartlett <abartlet at samba.org>
Date: Mon Aug 29 18:36:19 2016 +1200
samba-tool: Add command-line tool to trigger tombstone expunge
This allows us to carefully test the garbage collection of tombstoned objects
without running the full server and waiting for the timer to expire
Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Garming Sam <garming at catalyst.net.nz>
commit 02d82af06f3f13bdfd7497d168ba06f1092ede12
Author: Andrew Bartlett <abartlet at samba.org>
Date: Mon Jul 18 13:11:10 2016 +1200
python: Add binding for dsdb_garbage_collect_tombstones()
Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Garming Sam <garming at catalyst.net.nz>
commit 1c636532874da6cf998538027f088c1da019f15d
Author: Andrew Bartlett <abartlet at samba.org>
Date: Mon Jul 18 13:17:52 2016 +1200
dsdb: Expand garbage_collect_tombstones to expunge links also
This requires a significant rework, as we can no longer
do a one-level search and hope to find most of the deleted
objects. Therefore we fall back to a full scan, but less often.
Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Garming Sam <garming at catalyst.net.nz>
commit 824220e9bd7ffb9457ffc5a5bf0b5d279146f85c
Author: Andrew Bartlett <abartlet at samba.org>
Date: Mon Aug 29 19:02:39 2016 +1200
dsdb: move tombstone lifetime calculation out of dsdb_garbage_collect_tombstones()
This will allow it to be specified by the caller when we add python bindings
Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Garming Sam <garming at catalyst.net.nz>
commit d894f08ba95ae994dd1603af28bd74943bfdec9f
Author: Andrew Bartlett <abartlet at samba.org>
Date: Mon Aug 29 18:56:10 2016 +1200
dsdb: Rework more KCC service-specific details out of dsdb_garbage_collect_tombstones()
Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Garming Sam <garming at catalyst.net.nz>
commit 4e0ac09ef6c6fadd67718f7b5aca7283fb8086b1
Author: Andrew Bartlett <abartlet at samba.org>
Date: Mon Jul 18 13:10:23 2016 +1200
dsdb: Rework kcc_deleted() into dsdb_garbage_collect_tombstones()
This is so that in a future commit, we can wrap this in python and allow it to be called
from outside the samba server processs.
This requires that we rework the callers and internals to avoid reference to
private data structures of the KCC service.
Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Garming Sam <garming at catalyst.net.nz>
commit 9221ce3a3237a8ded78e371fef2b8e4f03722b63
Author: Andrew Bartlett <abartlet at samba.org>
Date: Mon Jul 18 11:53:50 2016 +1200
kcc: Move kcc/kcc_deleted.c into kcc/garbage_collect_tombstones.c
This is in preperation for a python binding for this function
Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Garming Sam <garming at catalyst.net.nz>
commit cf6bc40578c5de7e2eca0174d2be7aaa25e6b44e
Author: Andrew Bartlett <abartlet at samba.org>
Date: Mon Aug 29 18:20:45 2016 +1200
ldb: Fix comment on ldb_register_extended_match_rule
Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Garming Sam <garming at catalyst.net.nz>
commit eb1cb175980a87b8a7cbd765783af33e4a7d6017
Author: Andrew Bartlett <abartlet at samba.org>
Date: Mon Aug 29 18:20:18 2016 +1200
ldb-samba: Add new extended match rule DSDB_MATCH_FOR_EXPUNGE
This allows us to find links that need to be expunged
without passing the whole DB up in the search response.
While each message still needs to be examined, this code
only has to do memory allocation for entries with links
Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Garming Sam <garming at catalyst.net.nz>
commit 9142a01bb55a09e836c70d15fe420fb2599aec6f
Author: Andrew Bartlett <abartlet at samba.org>
Date: Thu Aug 25 11:28:32 2016 +1200
pydsdb: Raise TypeError for type errors, rather than incorrectly raising an LdbError
Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Garming Sam <garming at catalyst.net.nz>
commit 599d3f7f8358f6107e1d13ab0a92c3143f32435e
Author: Andrew Bartlett <abartlet at samba.org>
Date: Wed Aug 31 11:39:24 2016 +1200
selftest: Correct name of samba4.blackbox.dbcheck.release-4-5-0-pre1
Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Garming Sam <garming at catalyst.net.nz>
-----------------------------------------------------------------------
Summary of changes:
lib/ldb-samba/ldb_matching_rules.c | 133 +++++++++-
lib/ldb-samba/ldb_matching_rules.h | 1 +
lib/ldb/common/ldb_match.c | 4 +-
python/samba/netcmd/domain.py | 81 ++++++
python/samba/samdb.py | 14 +
selftest/tests.py | 9 +-
source4/dsdb/common/util.h | 10 +
source4/dsdb/kcc/garbage_collect_tombstones.c | 295 +++++++++++++++++++++
.../kcc/garbage_collect_tombstones.h} | 32 ++-
source4/dsdb/kcc/kcc_deleted.c | 146 ----------
source4/dsdb/kcc/kcc_periodic.c | 58 +++-
source4/dsdb/kcc/kcc_service.c | 5 +-
source4/dsdb/kcc/kcc_service.h | 13 +-
source4/dsdb/pydsdb.c | 108 +++++++-
source4/dsdb/wscript_build | 10 +-
.../release-4-5-0-pre1/add-four-more-links.ldif | 15 ++
.../release-4-5-0-pre1/add-two-more-users.ldif | 15 ++
.../expected-deleted-links-after-expunge.ldif | 23 ++
.../release-4-5-0-pre1/expected-expunge-output.txt | 1 +
.../expected-links-after-expunge.ldif | 22 ++
.../expected-match-rule-links.ldif | 47 ++++
.../expected-objects-after-expunge.ldif | 2 +
.../release-4-5-0-pre1/remove-one-more-link.ldif | 5 +
.../release-4-5-0-pre1/remove-one-more-user.ldif | 3 +
source4/setup/schema_samba4.ldif | 1 +
testprogs/blackbox/tombstones-expunge.sh | 194 ++++++++++++++
26 files changed, 1059 insertions(+), 188 deletions(-)
create mode 100644 source4/dsdb/kcc/garbage_collect_tombstones.c
copy source4/{param/pyparam.h => dsdb/kcc/garbage_collect_tombstones.h} (61%)
delete mode 100644 source4/dsdb/kcc/kcc_deleted.c
create mode 100644 source4/selftest/provisions/release-4-5-0-pre1/add-four-more-links.ldif
create mode 100644 source4/selftest/provisions/release-4-5-0-pre1/add-two-more-users.ldif
create mode 100644 source4/selftest/provisions/release-4-5-0-pre1/expected-deleted-links-after-expunge.ldif
create mode 100644 source4/selftest/provisions/release-4-5-0-pre1/expected-expunge-output.txt
create mode 100644 source4/selftest/provisions/release-4-5-0-pre1/expected-links-after-expunge.ldif
create mode 100644 source4/selftest/provisions/release-4-5-0-pre1/expected-match-rule-links.ldif
create mode 100644 source4/selftest/provisions/release-4-5-0-pre1/expected-objects-after-expunge.ldif
create mode 100644 source4/selftest/provisions/release-4-5-0-pre1/remove-one-more-link.ldif
create mode 100644 source4/selftest/provisions/release-4-5-0-pre1/remove-one-more-user.ldif
create mode 100755 testprogs/blackbox/tombstones-expunge.sh
Changeset truncated at 500 lines:
diff --git a/lib/ldb-samba/ldb_matching_rules.c b/lib/ldb-samba/ldb_matching_rules.c
index 637858f..aa86979 100644
--- a/lib/ldb-samba/ldb_matching_rules.c
+++ b/lib/ldb-samba/ldb_matching_rules.c
@@ -4,6 +4,7 @@
ldb database library - Extended match rules
Copyright (C) 2014 Samuel Cabrero <samuelcabrero at kernevil.me>
+ Copyright (C) Andrew Bartlett <abartlet at samba.org>
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
@@ -23,6 +24,8 @@
#include <ldb_module.h>
#include "dsdb/samdb/samdb.h"
#include "ldb_matching_rules.h"
+#include "libcli/security/security.h"
+#include "dsdb/common/util.h"
static int ldb_eval_transitive_filter_helper(TALLOC_CTX *mem_ctx,
struct ldb_context *ldb,
@@ -324,9 +327,128 @@ static int ldb_comparator_trans(struct ldb_context *ldb,
}
+/*
+ * This rule provides match of a link attribute against a 'should be expunged' criteria
+ *
+ * This allows a search filter such as:
+ *
+ * member:1.3.6.1.4.1.7165.4.5.2:=131139216000000000
+ *
+ * This searches the member attribute, but also any member attributes
+ * that are deleted and should be expunged after the specified NTTIME
+ * time.
+ *
+ */
+static int dsdb_match_for_expunge(struct ldb_context *ldb,
+ const char *oid,
+ const struct ldb_message *msg,
+ const char *attribute_to_match,
+ const struct ldb_val *value_to_match,
+ bool *matched)
+{
+ const struct dsdb_schema *schema;
+ const struct dsdb_attribute *schema_attr;
+ TALLOC_CTX *tmp_ctx;
+ unsigned int i;
+ struct ldb_message_element *el;
+ struct auth_session_info *session_info;
+ uint64_t tombstone_time;
+ *matched = false;
+
+ el = ldb_msg_find_element(msg, attribute_to_match);
+ if (el == NULL) {
+ return LDB_SUCCESS;
+ }
+
+ session_info
+ = talloc_get_type(ldb_get_opaque(ldb, "sessionInfo"),
+ struct auth_session_info);
+ if (security_session_user_level(session_info, NULL) != SECURITY_SYSTEM) {
+ return LDB_ERR_INSUFFICIENT_ACCESS_RIGHTS;
+ }
+
+ /*
+ * If the target attribute to match is not a linked attribute, then
+ * the filter evaluates to undefined
+ */
+ schema = dsdb_get_schema(ldb, NULL);
+ if (schema == NULL) {
+ return LDB_ERR_OPERATIONS_ERROR;
+ }
+
+ /* TODO this is O(log n) per attribute */
+ schema_attr = dsdb_attribute_by_lDAPDisplayName(schema, attribute_to_match);
+ if (schema_attr == NULL) {
+ return LDB_ERR_NO_SUCH_ATTRIBUTE;
+ }
+
+ /*
+ * This extended match filter is only valid for forward linked attributes.
+ */
+ if (schema_attr->linkID == 0 || (schema_attr->linkID & 1) == 1) {
+ return LDB_ERR_NO_SUCH_ATTRIBUTE;
+ }
+
+ /* Just check we don't allow the caller to fill our stack */
+ if (value_to_match->length >=64) {
+ return LDB_ERR_INVALID_ATTRIBUTE_SYNTAX;
+ } else {
+ char *p = NULL;
+ char s[value_to_match->length+1];
+ memcpy(s, value_to_match->data, value_to_match->length);
+ s[value_to_match->length] = 0;
+ if (s[0] == '\0' || s[0] == '-') {
+ return LDB_ERR_INVALID_ATTRIBUTE_SYNTAX;
+ }
+ tombstone_time = strtoull(s, &p, 10);
+ if (p == NULL || p == s || *p != '\0' || tombstone_time == ULLONG_MAX) {
+ return LDB_ERR_INVALID_ATTRIBUTE_SYNTAX;
+ }
+ }
+
+ tmp_ctx = talloc_new(ldb);
+ if (tmp_ctx == NULL) {
+ return LDB_ERR_OPERATIONS_ERROR;
+ }
+
+ for (i = 0; i < el->num_values; i++) {
+ NTSTATUS status;
+ struct dsdb_dn *dn;
+ uint64_t rmd_changetime;
+ if (dsdb_dn_is_deleted_val(&el->values[i]) == false) {
+ continue;
+ }
+
+ dn = dsdb_dn_parse(tmp_ctx, ldb, &el->values[i],
+ schema_attr->syntax->ldap_oid);
+ if (dn == NULL) {
+ DEBUG(1, ("Error: Failed to parse linked attribute blob of %s.\n", el->name));
+ continue;
+ }
+
+ status = dsdb_get_extended_dn_uint64(dn->dn, &rmd_changetime,
+ "RMD_CHANGETIME");
+ if (!NT_STATUS_IS_OK(status)) {
+ DEBUG(1, ("Error: RMD_CHANGETIME is missing on a forward link.\n"));
+ continue;
+ }
+
+ if (rmd_changetime > tombstone_time) {
+ continue;
+ }
+
+ *matched = true;
+ break;
+ }
+ talloc_free(tmp_ctx);
+ return LDB_SUCCESS;
+}
+
+
int ldb_register_samba_matching_rules(struct ldb_context *ldb)
{
- struct ldb_extended_match_rule *transitive_eval;
+ struct ldb_extended_match_rule *transitive_eval = NULL,
+ *match_for_expunge = NULL;
int ret;
transitive_eval = talloc_zero(ldb, struct ldb_extended_match_rule);
@@ -338,5 +460,14 @@ int ldb_register_samba_matching_rules(struct ldb_context *ldb)
return ret;
}
+ match_for_expunge = talloc_zero(ldb, struct ldb_extended_match_rule);
+ match_for_expunge->oid = DSDB_MATCH_FOR_EXPUNGE;
+ match_for_expunge->callback = dsdb_match_for_expunge;
+ ret = ldb_register_extended_match_rule(ldb, match_for_expunge);
+ if (ret != LDB_SUCCESS) {
+ talloc_free(match_for_expunge);
+ return ret;
+ }
+
return LDB_SUCCESS;
}
diff --git a/lib/ldb-samba/ldb_matching_rules.h b/lib/ldb-samba/ldb_matching_rules.h
index e969b3d..421e1ce 100644
--- a/lib/ldb-samba/ldb_matching_rules.h
+++ b/lib/ldb-samba/ldb_matching_rules.h
@@ -24,5 +24,6 @@
/* This rule provides recursive search of a link attribute */
#define SAMBA_LDAP_MATCH_RULE_TRANSITIVE_EVAL "1.2.840.113556.1.4.1941"
+#define DSDB_MATCH_FOR_EXPUNGE "1.3.6.1.4.1.7165.4.5.2"
#endif /* _LDB_MATCHING_RULES_H_ */
diff --git a/lib/ldb/common/ldb_match.c b/lib/ldb/common/ldb_match.c
index e83ad63..1415fac 100644
--- a/lib/ldb/common/ldb_match.c
+++ b/lib/ldb/common/ldb_match.c
@@ -693,9 +693,7 @@ _PRIVATE_ int ldb_register_extended_match_rules(struct ldb_context *ldb)
}
/*
- register a new ldb backend
-
- if override is true, then override any existing backend for this prefix
+ register a new ldb extended matching rule
*/
int ldb_register_extended_match_rule(struct ldb_context *ldb,
const struct ldb_extended_match_rule *rule)
diff --git a/python/samba/netcmd/domain.py b/python/samba/netcmd/domain.py
index d92820a..928caa3 100644
--- a/python/samba/netcmd/domain.py
+++ b/python/samba/netcmd/domain.py
@@ -32,6 +32,7 @@ import random
import tempfile
import logging
import subprocess
+import time
from getpass import getpass
from samba.net import Net, LIBNET_JOIN_AUTOMATIC
import samba.ntacls
@@ -3747,6 +3748,79 @@ class cmd_domain_trust_namespaces(DomainTrustCommand):
tln=local_tdo_info.domain_name.string)
return
+class cmd_domain_tombstones_expunge(Command):
+ """Expunge tombstones from the database.
+
+This command expunges tombstones from the database."""
+ synopsis = "%prog NC [NC [...]] [options]"
+
+ takes_options = [
+ Option("-H", "--URL", help="LDB URL for database or target server", type=str,
+ metavar="URL", dest="H"),
+ Option("--current-time",
+ help="The current time to evaluate the tombstone lifetime from, expressed as YYYY-MM-DD",
+ type=str),
+ Option("--tombstone-lifetime", help="Number of days a tombstone should be preserved for", type=int),
+ ]
+
+ takes_args = ["nc*"]
+
+ takes_optiongroups = {
+ "sambaopts": options.SambaOptions,
+ "credopts": options.CredentialsOptions,
+ "versionopts": options.VersionOptions,
+ }
+
+ def run(self, *ncs, **kwargs):
+ sambaopts = kwargs.get("sambaopts")
+ credopts = kwargs.get("credopts")
+ versionpts = kwargs.get("versionopts")
+ H = kwargs.get("H")
+ current_time_string = kwargs.get("current_time")
+ tombstone_lifetime = kwargs.get("tombstone_lifetime")
+ lp = sambaopts.get_loadparm()
+ creds = credopts.get_credentials(lp)
+ samdb = SamDB(url=H, session_info=system_session(),
+ credentials=creds, lp=lp)
+
+ if current_time_string is not None:
+ current_time_obj = time.strptime(current_time_string, "%Y-%m-%d")
+ current_time = long(time.mktime(current_time_obj))
+
+ else:
+ current_time = long(time.time())
+
+ if len(ncs) == 0:
+ res = samdb.search(expression="", base="", scope=ldb.SCOPE_BASE,
+ attrs=["namingContexts"])
+
+ ncs = []
+ for nc in res[0]["namingContexts"]:
+ ncs.append(str(nc))
+ else:
+ ncs = list(ncs)
+
+ started_transaction = False
+ try:
+ samdb.transaction_start()
+ started_transaction = True
+ (removed_objects,
+ removed_links) = samdb.garbage_collect_tombstones(ncs,
+ current_time=current_time,
+ tombstone_lifetime=tombstone_lifetime)
+
+ except Exception, err:
+ if started_transaction:
+ samdb.transaction_cancel()
+ raise CommandError("Failed to expunge / garbage collect tombstones", err)
+
+ samdb.transaction_commit()
+
+ self.outf.write("Removed %d objects and %d links successfully\n"
+ % (removed_objects, removed_links))
+
+
+
class cmd_domain_trust(SuperCommand):
"""Domain and forest trust management."""
@@ -3758,6 +3832,12 @@ class cmd_domain_trust(SuperCommand):
subcommands["validate"] = cmd_domain_trust_validate()
subcommands["namespaces"] = cmd_domain_trust_namespaces()
+class cmd_domain_tombstones(SuperCommand):
+ """Domain tombstone and recycled object management."""
+
+ subcommands = {}
+ subcommands["expunge"] = cmd_domain_tombstones_expunge()
+
class cmd_domain(SuperCommand):
"""Domain management."""
@@ -3774,3 +3854,4 @@ class cmd_domain(SuperCommand):
subcommands["classicupgrade"] = cmd_domain_classicupgrade()
subcommands["samba3upgrade"] = cmd_domain_samba3upgrade()
subcommands["trust"] = cmd_domain_trust()
+ subcommands["tombstones"] = cmd_domain_tombstones()
diff --git a/python/samba/samdb.py b/python/samba/samdb.py
index e12465d..3d7ea3e 100644
--- a/python/samba/samdb.py
+++ b/python/samba/samdb.py
@@ -949,3 +949,17 @@ accountExpires: %u
is removed, this routine will put a tombstone in the record.
'''
return dsdb_dns.replace_by_dn(self, dn, new_records)
+
+ def garbage_collect_tombstones(self, dn, current_time,
+ tombstone_lifetime=None):
+ '''garbage_collect_tombstones(lp, samdb, [dn], current_time, tombstone_lifetime)
+ -> (num_objects_expunged, num_links_expunged)'''
+
+
+ if tombstone_lifetime is None:
+ return dsdb._dsdb_garbage_collect_tombstones(self, dn,
+ current_time)
+ else:
+ return dsdb._dsdb_garbage_collect_tombstones(self, dn,
+ current_time,
+ tombstone_lifetime)
diff --git a/selftest/tests.py b/selftest/tests.py
index 62dadab..9f65f40 100644
--- a/selftest/tests.py
+++ b/selftest/tests.py
@@ -91,6 +91,11 @@ plantestsuite(
["PYTHON=%s" % python, os.path.join(bbdir, "dbcheck-oldrelease.sh"),
'$PREFIX_ABS/provision', 'release-4-1-6-partial-object', configuration])
plantestsuite(
+ "samba4.blackbox.dbcheck.release-4-5-0-pre1", "none",
+ ["PYTHON=%s" % python,
+ os.path.join(bbdir, "dbcheck-oldrelease.sh"),
+ '$PREFIX_ABS/provision', 'release-4-5-0-pre1', configuration])
+plantestsuite(
"samba4.blackbox.upgradeprovision.alpha13", "none",
["PYTHON=%s" % python,
os.path.join(bbdir, "upgradeprovision-oldrelease.sh"),
@@ -101,9 +106,9 @@ plantestsuite(
os.path.join(bbdir, "upgradeprovision-oldrelease.sh"),
'$PREFIX_ABS/provision', 'release-4-0-0', configuration])
plantestsuite(
- "samba4.blackbox.upgradeprovision.release-4-5-0-pre1", "none",
+ "samba4.blackbox.tombstones-expunge.release-4-5-0-pre1", "none",
["PYTHON=%s" % python,
- os.path.join(bbdir, "dbcheck-oldrelease.sh"),
+ os.path.join(bbdir, "tombstones-expunge.sh"),
'$PREFIX_ABS/provision', 'release-4-5-0-pre1', configuration])
planpythontestsuite("none", "samba.tests.upgradeprovision")
planpythontestsuite("none", "samba.tests.xattr")
diff --git a/source4/dsdb/common/util.h b/source4/dsdb/common/util.h
index f2867a2..ede6d8b 100644
--- a/source4/dsdb/common/util.h
+++ b/source4/dsdb/common/util.h
@@ -80,4 +80,14 @@ int dsdb_werror_at(struct ldb_context *ldb, int ldb_ecode, WERROR werr,
dsdb_werror_at(ldb_module_get_ctx(module), ldb_ecode, werr, \
__location__, __func__, reason)
+
+struct dsdb_ldb_dn_list_node {
+ struct dsdb_ldb_dn_list_node *prev, *next;
+
+ /* the dn of the partition */
+ struct ldb_dn *dn;
+};
+
+
+
#endif /* __DSDB_COMMON_UTIL_H__ */
diff --git a/source4/dsdb/kcc/garbage_collect_tombstones.c b/source4/dsdb/kcc/garbage_collect_tombstones.c
new file mode 100644
index 0000000..8d8a51f
--- /dev/null
+++ b/source4/dsdb/kcc/garbage_collect_tombstones.c
@@ -0,0 +1,295 @@
+/*
+ Unix SMB/CIFS implementation.
+
+ handle removal of deleted objects
+
+ Copyright (C) 2009 Andrew Tridgell
+
+ This program is free software; you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation; either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+*/
+
+#include "includes.h"
+#include <ldb_errors.h>
+#include "../lib/util/dlinklist.h"
+#include "librpc/gen_ndr/ndr_misc.h"
+#include "librpc/gen_ndr/ndr_drsuapi.h"
+#include "librpc/gen_ndr/ndr_drsblobs.h"
+#include "param/param.h"
+#include "lib/util/dlinklist.h"
+#include "ldb.h"
+#include "dsdb/kcc/garbage_collect_tombstones.h"
+#include "lib/ldb-samba/ldb_matching_rules.h"
+#include "lib/util/time.h"
+
+/*
+ * Per MS-ADTS 3.1.1.5.5 Delete Operation
+ *
+ * "Tombstones are a type of deleted object distinguished from
+ * existing-objects by the presence of the isDeleted attribute with the
+ * value true."
+ *
+ * "After a time period at least as large as a tombstone lifetime, the
+ * tombstone is removed from the directory."
+ *
+ * The purpose of this routine is to remove such objects. It is
+ * called from a timed event in the KCC, and from samba-tool domain
+ * expunge tombstones.
+ *
+ * Additionally, linked attributes have similar properties.
+ */
+NTSTATUS dsdb_garbage_collect_tombstones(TALLOC_CTX *mem_ctx,
+ struct ldb_context *samdb,
+ struct dsdb_ldb_dn_list_node *part,
+ time_t current_time,
+ uint32_t tombstoneLifetime,
+ unsigned int *num_objects_removed,
+ unsigned int *num_links_removed,
+ char **error_string)
+{
+ int ret;
+
+ const char **attrs = NULL;
+ char *filter = NULL;
+
+ unsigned int i;
+ struct dsdb_attribute *next_attr;
+ unsigned int num_link_attrs;
+ struct dsdb_schema *schema = dsdb_get_schema(samdb, mem_ctx);
+ unsigned long long expunge_time = current_time - tombstoneLifetime*60*60*24;
+ char *expunge_time_string = ldb_timestring_utc(mem_ctx, expunge_time);
+ NTTIME expunge_time_nttime;
+ unix_to_nt_time(&expunge_time_nttime, expunge_time);
+
+ *num_objects_removed = 0;
+ *num_links_removed = 0;
+ *error_string = NULL;
+ num_link_attrs = 0;
+
+ /*
+ * This filter is a bit strange, but the idea is to filter for
+ * objects that need to have tombstones expunged without
+ * bringing a potentially large databse all into memory. To
+ * do that, we could use callbacks, but instead we use a
+ * custom match rule to triage the objects during the search,
+ * and ideally avoid memory allocation for most of the
+ * un-matched objects.
+ *
+ * The parameter to DSDB_MATCH_FOR_EXPUNGE is the NTTIME, we
+ * return records with deleted links deleted before this time.
+ *
+ * We use a date comparison on whenChanged to avoid returning
+ * all isDeleted records
+ */
+
+ filter = talloc_asprintf(mem_ctx, "(|");
+ for (next_attr = schema->attributes; next_attr != NULL; next_attr = next_attr->next) {
+ if (next_attr->linkID != 0 && ((next_attr->linkID & 1) == 0)) {
+ num_link_attrs++;
+ filter = talloc_asprintf_append(filter,
+ "(%s:" DSDB_MATCH_FOR_EXPUNGE ":=%llu)",
+ next_attr->lDAPDisplayName,
+ (unsigned long long)expunge_time_nttime);
+ if (filter == NULL) {
+ return NT_STATUS_NO_MEMORY;
+ }
+ }
+ }
+
+ attrs = talloc_array(mem_ctx, const char *, num_link_attrs + 2);
+ i = 0;
+ for (next_attr = schema->attributes; next_attr != NULL; next_attr = next_attr->next) {
+ if (next_attr->linkID != 0 && ((next_attr->linkID & 1) == 0)) {
+ attrs[i++] = next_attr->lDAPDisplayName;
+ }
+ }
+ attrs[i] = "isDeleted";
+ attrs[i+1] = NULL;
+
+ filter = talloc_asprintf_append(filter, "(&(isDeleted=TRUE)(whenChanged<=%s)))", expunge_time_string);
+ if (filter == NULL) {
+ return NT_STATUS_NO_MEMORY;
+ }
--
Samba Shared Repository
More information about the samba-cvs
mailing list