[PATCH] Partial fix for DNS performance regression
Andrew Bartlett
abartlet at samba.org
Mon Dec 18 00:21:33 UTC 2017
The attached patches for master partially address the DNS performance
regression found in Samba 4.7.3.
In master, a DNS query against a large domain (7000 entries in the
zone) can take 18 seconds, much longer than the retry timeout (causing
back-off, retry, much pain).
Master (for 4.8) is already better than 4.7.3 in this area ('only'
10s), but these patches avoid the SCOPE_SUBTREE search until the
wildcard lookup is needed.
The patches for 4.7 include a backport of some already-made
improvements in Samba's master branch, part of the preparation patches
for the GUID index mode.
My rough notes on performance show that the LDB changes save about
another 20% for NXDOMAIN results, the DNS changes are the source of the
primary gains.
For the domain root, 'host domain.com' takes about 100ms in all cases
after the patch, returning to the baseline seen with 4.7.0.
However 'host foo.domain.com' takes:
(rough figures)
- master: 270ms
- master + patch: 150ms
- 4.7.3 (unpatched): 1000ms
- 4.7.3 + patch: 340ms
- 4.7.3 + DNS changes only: 500ms
- 4.7.0: 20ms
We are still an order of magnitude slower with wildcard DNS support
compared to before we added it, for NXDOMAIN responses. This isn't OK,
so we need to keep working on this, but the attached should help for
now.
Please review carefully and push the patches to master, then we will
backport, perhaps for 4.7.4 if we are lucky.
Thanks,
Andrew Bartlett
--
Andrew Bartlett
https://samba.org/~abartlet/
Authentication Developer, Samba Team https://samba.org
Samba Development and Support, Catalyst IT
https://catalyst.net.nz/services/samba
-------------- next part --------------
From 80512fdcdc6a4eed3af066c5dfd084ead1b19ff2 Mon Sep 17 00:00:00 2001
From: Andrew Bartlett <abartlet at samba.org>
Date: Fri, 15 Dec 2017 11:40:28 +1300
Subject: [PATCH 1/8] dns_server: Do not look for a wildcard for @
This query is made for every record returned via BIND9 DLZ.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13191
Signed-off-by: Andrew Bartlett <abartlet at samba.org>
---
source4/dns_server/dnsserver_common.c | 10 ++++++++++
1 file changed, 10 insertions(+)
diff --git a/source4/dns_server/dnsserver_common.c b/source4/dns_server/dnsserver_common.c
index 217e65b39f4..093898361d0 100644
--- a/source4/dns_server/dnsserver_common.c
+++ b/source4/dns_server/dnsserver_common.c
@@ -547,6 +547,16 @@ WERROR dns_common_wildcard_lookup(struct ldb_context *samdb,
return DNS_ERR(NAME_ERROR);
}
+ /* Don't look for a wildcard for @ */
+ if (name->length == 1 && name->data[0] == '@') {
+ return dns_common_lookup(samdb,
+ mem_ctx,
+ dn,
+ records,
+ num_records,
+ NULL);
+ }
+
werr = dns_name_check(
mem_ctx,
strlen((const char*)name->data),
--
2.11.0
From 461fbf21dd3d414464ea1ab22375f1b52a5f7a66 Mon Sep 17 00:00:00 2001
From: Andrew Bartlett <abartlet at samba.org>
Date: Fri, 15 Dec 2017 11:41:15 +1300
Subject: [PATCH 2/8] dns_server: Do not embed a trailing NUL in the query
The data to match on does not include the NUL, just the significant bytes
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13191
Signed-off-by: Andrew Bartlett <abartlet at samba.org>
---
source4/dns_server/dnsserver_common.c | 10 ++++++----
1 file changed, 6 insertions(+), 4 deletions(-)
diff --git a/source4/dns_server/dnsserver_common.c b/source4/dns_server/dnsserver_common.c
index 093898361d0..574d1510230 100644
--- a/source4/dns_server/dnsserver_common.c
+++ b/source4/dns_server/dnsserver_common.c
@@ -250,9 +250,11 @@ static struct ldb_parse_tree *build_equality_operation(
struct ldb_parse_tree *el = NULL; /* Equality node being built */
struct ldb_val *value = NULL; /* Value the attr will be compared
with */
- size_t length = 0; /* calculated length of the value
- including option '*' prefix and
- '\0' string terminator */
+ size_t length = 0; /* calculated length of the
+ value including option
+ '*' prefix but no '\0'
+ string terminator (this
+ is an ldb_val) */
el = talloc(mem_ctx, struct ldb_parse_tree);
if (el == NULL) {
@@ -263,7 +265,7 @@ static struct ldb_parse_tree *build_equality_operation(
el->operation = LDB_OP_EQUALITY;
el->u.equality.attr = talloc_strdup(mem_ctx, attr);
value = &el->u.equality.value;
- length = (add_asterix) ? size + 2 : size + 1;
+ length = (add_asterix) ? size + 1 : size;
value->data = talloc_zero_array(el, uint8_t, length);
if (el == NULL) {
DBG_ERR("Unable to allocate value->data\n");
--
2.11.0
From 82ad391fc83d8eee6c721fb8ff5160f030e66826 Mon Sep 17 00:00:00 2001
From: Andrew Bartlett <abartlet at samba.org>
Date: Fri, 15 Dec 2017 12:30:50 +1300
Subject: [PATCH 3/8] dns_server: Do the exact match query first, then do the
wildcard lookup
The wildcard lookup is SCOPE_ONELEVEL which in current ldb will load the full index for that zone.
This is inefficient as we do not currently intersect that index with the
index on DN=
Not that a not-found and wildcard response will still go via the ONELEVEL
index for now so this only helps for correct, directly matching queries.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13191
Signed-off-by: Andrew Bartlett <abartlet at samba.org>
---
source4/dns_server/dnsserver_common.c | 14 ++++++++++++++
1 file changed, 14 insertions(+)
diff --git a/source4/dns_server/dnsserver_common.c b/source4/dns_server/dnsserver_common.c
index 574d1510230..368d32326b0 100644
--- a/source4/dns_server/dnsserver_common.c
+++ b/source4/dns_server/dnsserver_common.c
@@ -567,6 +567,20 @@ WERROR dns_common_wildcard_lookup(struct ldb_context *samdb,
return werr;
}
+ /*
+ * Do a point search first, then fall back to a wildcard
+ * lookup if it does not exist
+ */
+ werr = dns_common_lookup(samdb,
+ mem_ctx,
+ dn,
+ records,
+ num_records,
+ NULL);
+ if (!W_ERROR_EQUAL(werr, WERR_DNS_ERROR_NAME_DOES_NOT_EXIST)) {
+ return werr;
+ }
+
ret = dns_wildcard_lookup(samdb, mem_ctx, dn, &msg);
if (ret == LDB_ERR_OPERATIONS_ERROR) {
return DNS_ERR(SERVER_FAILURE);
--
2.11.0
From 12d4153e09f7f93669d42967c3f1d4fad53fd424 Mon Sep 17 00:00:00 2001
From: Andrew Bartlett <abartlet at samba.org>
Date: Thu, 10 Aug 2017 16:23:33 +1200
Subject: [PATCH 4/8] ldb_tdb: Provide struct ltdb_private to index routines
This will make it easier to switch the GUID index mode on and off
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13191
Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Garming Sam <garming at catalyst.net.nz>
(cherry picked from commit 3d952157d72b3a4635f3942449c1727c438c97c6)
---
lib/ldb/ldb_tdb/ldb_index.c | 90 ++++++++++++++++++++++++++++++---------------
lib/ldb/ldb_tdb/ldb_tdb.c | 38 +++++++++++++------
lib/ldb/ldb_tdb/ldb_tdb.h | 16 ++++++--
3 files changed, 99 insertions(+), 45 deletions(-)
diff --git a/lib/ldb/ldb_tdb/ldb_index.c b/lib/ldb/ldb_tdb/ldb_index.c
index 3510dd9b1bc..ddcf0124dab 100644
--- a/lib/ldb/ldb_tdb/ldb_index.c
+++ b/lib/ldb/ldb_tdb/ldb_index.c
@@ -79,7 +79,9 @@ static int dn_list_cmp(const struct ldb_val *v1, const struct ldb_val *v2)
find a entry in a dn_list, using a ldb_val. Uses a case sensitive
comparison with the dn returns -1 if not found
*/
-static int ltdb_dn_list_find_val(const struct dn_list *list, const struct ldb_val *v)
+static int ltdb_dn_list_find_val(struct ltdb_private *ltdb,
+ const struct dn_list *list,
+ const struct ldb_val *v)
{
unsigned int i;
for (i=0; i<list->count; i++) {
@@ -94,12 +96,14 @@ static int ltdb_dn_list_find_val(const struct dn_list *list, const struct ldb_va
find a entry in a dn_list. Uses a case sensitive comparison with the dn
returns -1 if not found
*/
-static int ltdb_dn_list_find_str(struct dn_list *list, const char *dn)
+static int ltdb_dn_list_find_str(struct ltdb_private *ltdb,
+ struct dn_list *list,
+ const char *dn)
{
struct ldb_val v;
v.data = discard_const_p(unsigned char, dn);
v.length = strlen(dn);
- return ltdb_dn_list_find_val(list, &v);
+ return ltdb_dn_list_find_val(ltdb, list, &v);
}
/*
@@ -219,7 +223,9 @@ normal_index:
/*
save a dn_list into a full @IDX style record
*/
-static int ltdb_dn_list_store_full(struct ldb_module *module, struct ldb_dn *dn,
+static int ltdb_dn_list_store_full(struct ldb_module *module,
+ struct ltdb_private *ltdb,
+ struct ldb_dn *dn,
struct dn_list *list)
{
struct ldb_message *msg;
@@ -274,7 +280,8 @@ static int ltdb_dn_list_store(struct ldb_module *module, struct ldb_dn *dn,
struct dn_list *list2;
if (ltdb->idxptr == NULL) {
- return ltdb_dn_list_store_full(module, dn, list);
+ return ltdb_dn_list_store_full(module, ltdb,
+ dn, list);
}
if (ltdb->idxptr->itdb == NULL) {
@@ -345,7 +352,8 @@ static int ltdb_index_traverse_store(struct tdb_context *tdb, TDB_DATA key, TDB_
return -1;
}
- ltdb->idxptr->error = ltdb_dn_list_store_full(module, dn, list);
+ ltdb->idxptr->error = ltdb_dn_list_store_full(module, ltdb,
+ dn, list);
talloc_free(dn);
if (ltdb->idxptr->error != 0) {
return -1;
@@ -575,6 +583,7 @@ static int ltdb_index_dn_leaf(struct ldb_module *module,
list = list & list2
*/
static bool list_intersect(struct ldb_context *ldb,
+ struct ltdb_private *ltdb,
struct dn_list *list, const struct dn_list *list2)
{
struct dn_list *list3;
@@ -622,7 +631,8 @@ static bool list_intersect(struct ldb_context *ldb,
list3->count = 0;
for (i=0;i<list->count;i++) {
- if (ltdb_dn_list_find_val(list2, &list->dn[i]) != -1) {
+ if (ltdb_dn_list_find_val(ltdb, list2,
+ &list->dn[i]) != -1) {
list3->dn[list3->count] = list->dn[i];
list3->count++;
}
@@ -846,7 +856,8 @@ static int ltdb_index_dn_and(struct ldb_module *module,
list->dn = list2->dn;
list->count = list2->count;
found = true;
- } else if (!list_intersect(ldb, list, list2)) {
+ } else if (!list_intersect(ldb, ltdb,
+ list, list2)) {
talloc_free(list2);
return LDB_ERR_OPERATIONS_ERROR;
}
@@ -952,7 +963,8 @@ static int ltdb_index_dn(struct ldb_module *module,
filter a candidate dn_list from an indexed search into a set of results
extracting just the given attributes
*/
-static int ltdb_index_filter(const struct dn_list *dn_list,
+static int ltdb_index_filter(struct ltdb_private *ltdb,
+ const struct dn_list *dn_list,
struct ltdb_context *ac,
uint32_t *match_count)
{
@@ -1123,7 +1135,7 @@ int ltdb_search_indexed(struct ltdb_context *ac, uint32_t *match_count)
break;
}
- ret = ltdb_index_filter(dn_list, ac, match_count);
+ ret = ltdb_index_filter(ltdb, dn_list, ac, match_count);
talloc_free(dn_list);
return ret;
}
@@ -1148,7 +1160,9 @@ int ltdb_search_indexed(struct ltdb_context *ac, uint32_t *match_count)
*
* @return An ldb error code
*/
-static int ltdb_index_add1(struct ldb_module *module, const char *dn,
+static int ltdb_index_add1(struct ldb_module *module,
+ struct ltdb_private *ltdb,
+ const char *dn,
struct ldb_message_element *el, int v_idx)
{
struct ldb_context *ldb;
@@ -1227,12 +1241,15 @@ static int ltdb_index_add1(struct ldb_module *module, const char *dn,
/*
add index entries for one elements in a message
*/
-static int ltdb_index_add_el(struct ldb_module *module, const char *dn,
+static int ltdb_index_add_el(struct ldb_module *module,
+ struct ltdb_private *ltdb,
+ const char *dn,
struct ldb_message_element *el)
{
unsigned int i;
for (i = 0; i < el->num_values; i++) {
- int ret = ltdb_index_add1(module, dn, el, i);
+ int ret = ltdb_index_add1(module, ltdb,
+ dn, el, i);
if (ret != LDB_SUCCESS) {
return ret;
}
@@ -1244,11 +1261,12 @@ static int ltdb_index_add_el(struct ldb_module *module, const char *dn,
/*
add index entries for all elements in a message
*/
-static int ltdb_index_add_all(struct ldb_module *module, const char *dn,
+static int ltdb_index_add_all(struct ldb_module *module,
+ struct ltdb_private *ltdb,
+ const char *dn,
struct ldb_message_element *elements,
unsigned int num_el)
{
- struct ltdb_private *ltdb = talloc_get_type(ldb_module_get_private(module), struct ltdb_private);
unsigned int i;
if (dn[0] == '@') {
@@ -1265,7 +1283,7 @@ static int ltdb_index_add_all(struct ldb_module *module, const char *dn,
if (!ltdb_is_indexed(module, ltdb, elements[i].name)) {
continue;
}
- ret = ltdb_index_add_el(module, dn, &elements[i]);
+ ret = ltdb_index_add_el(module, ltdb, dn, &elements[i]);
if (ret != LDB_SUCCESS) {
struct ldb_context *ldb = ldb_module_get_ctx(module);
ldb_asprintf_errstring(ldb,
@@ -1321,9 +1339,9 @@ static int ltdb_index_onelevel(struct ldb_module *module,
el.num_values = 1;
if (add) {
- ret = ltdb_index_add1(module, dn, &el, 0);
+ ret = ltdb_index_add1(module, ltdb, dn, &el, 0);
} else { /* delete */
- ret = ltdb_index_del_value(module, msg->dn, &el, 0);
+ ret = ltdb_index_del_value(module, ltdb, msg->dn, &el, 0);
}
talloc_free(pdn);
@@ -1335,23 +1353,27 @@ static int ltdb_index_onelevel(struct ldb_module *module,
add the index entries for a new element in a record
The caller guarantees that these element values are not yet indexed
*/
-int ltdb_index_add_element(struct ldb_module *module, struct ldb_dn *dn,
+int ltdb_index_add_element(struct ldb_module *module,
+ struct ltdb_private *ltdb,
+ struct ldb_dn *dn,
struct ldb_message_element *el)
{
- struct ltdb_private *ltdb = talloc_get_type(ldb_module_get_private(module), struct ltdb_private);
if (ldb_dn_is_special(dn)) {
return LDB_SUCCESS;
}
if (!ltdb_is_indexed(module, ltdb, el->name)) {
return LDB_SUCCESS;
}
- return ltdb_index_add_el(module, ldb_dn_get_linearized(dn), el);
+ return ltdb_index_add_el(module, ltdb,
+ ldb_dn_get_linearized(dn), el);
}
/*
add the index entries for a new record
*/
-int ltdb_index_add_new(struct ldb_module *module, const struct ldb_message *msg)
+int ltdb_index_add_new(struct ldb_module *module,
+ struct ltdb_private *ltdb,
+ const struct ldb_message *msg)
{
const char *dn;
int ret;
@@ -1365,7 +1387,8 @@ int ltdb_index_add_new(struct ldb_module *module, const struct ldb_message *msg)
return LDB_ERR_OPERATIONS_ERROR;
}
- ret = ltdb_index_add_all(module, dn, msg->elements, msg->num_elements);
+ ret = ltdb_index_add_all(module, ltdb, dn, msg->elements,
+ msg->num_elements);
if (ret != LDB_SUCCESS) {
return ret;
}
@@ -1377,7 +1400,9 @@ int ltdb_index_add_new(struct ldb_module *module, const struct ldb_message *msg)
/*
delete an index entry for one message element
*/
-int ltdb_index_del_value(struct ldb_module *module, struct ldb_dn *dn,
+int ltdb_index_del_value(struct ldb_module *module,
+ struct ltdb_private *ltdb,
+ struct ldb_dn *dn,
struct ldb_message_element *el, unsigned int v_idx)
{
struct ldb_context *ldb;
@@ -1422,7 +1447,7 @@ int ltdb_index_del_value(struct ldb_module *module, struct ldb_dn *dn,
return ret;
}
- i = ltdb_dn_list_find_str(list, dn_str);
+ i = ltdb_dn_list_find_str(ltdb, list, dn_str);
if (i == -1) {
/* nothing to delete */
talloc_free(dn_key);
@@ -1452,10 +1477,11 @@ int ltdb_index_del_value(struct ldb_module *module, struct ldb_dn *dn,
delete the index entries for a element
return -1 on failure
*/
-int ltdb_index_del_element(struct ldb_module *module, struct ldb_dn *dn,
+int ltdb_index_del_element(struct ldb_module *module,
+ struct ltdb_private *ltdb,
+ struct ldb_dn *dn,
struct ldb_message_element *el)
{
- struct ltdb_private *ltdb = talloc_get_type(ldb_module_get_private(module), struct ltdb_private);
const char *dn_str;
int ret;
unsigned int i;
@@ -1478,7 +1504,7 @@ int ltdb_index_del_element(struct ldb_module *module, struct ldb_dn *dn,
return LDB_SUCCESS;
}
for (i = 0; i < el->num_values; i++) {
- ret = ltdb_index_del_value(module, dn, el, i);
+ ret = ltdb_index_del_value(module, ltdb, dn, el, i);
if (ret != LDB_SUCCESS) {
return ret;
}
@@ -1512,7 +1538,8 @@ int ltdb_index_delete(struct ldb_module *module, const struct ldb_message *msg)
}
for (i = 0; i < msg->num_elements; i++) {
- ret = ltdb_index_del_element(module, msg->dn, &msg->elements[i]);
+ ret = ltdb_index_del_element(module, ltdb,
+ msg->dn, &msg->elements[i]);
if (ret != LDB_SUCCESS) {
return ret;
}
@@ -1677,6 +1704,8 @@ static int re_index(struct tdb_context *tdb, TDB_DATA key, TDB_DATA data, void *
struct ldb_context *ldb;
struct ltdb_reindex_context *ctx = (struct ltdb_reindex_context *)state;
struct ldb_module *module = ctx->module;
+ struct ltdb_private *ltdb = talloc_get_type(ldb_module_get_private(module),
+ struct ltdb_private);
struct ldb_message *msg;
const char *dn = NULL;
unsigned int nb_elements_in_db;
@@ -1738,7 +1767,8 @@ static int re_index(struct tdb_context *tdb, TDB_DATA key, TDB_DATA data, void *
return -1;
}
- ret = ltdb_index_add_all(module, dn, msg->elements, msg->num_elements);
+ ret = ltdb_index_add_all(module, ltdb, dn,
+ msg->elements, msg->num_elements);
if (ret != LDB_SUCCESS) {
ctx->error = ret;
diff --git a/lib/ldb/ldb_tdb/ldb_tdb.c b/lib/ldb/ldb_tdb/ldb_tdb.c
index ccad8168a6e..afa0f9e47c3 100644
--- a/lib/ldb/ldb_tdb/ldb_tdb.c
+++ b/lib/ldb/ldb_tdb/ldb_tdb.c
@@ -355,6 +355,7 @@ static bool ldb_tdb_single_valued(const struct ldb_schema_attribute *a,
}
static int ltdb_add_internal(struct ldb_module *module,
+ struct ltdb_private *ltdb,
const struct ldb_message *msg,
bool check_single_value)
{
@@ -419,7 +420,7 @@ static int ltdb_add_internal(struct ldb_module *module,
return ret;
}
- ret = ltdb_index_add_new(module, msg);
+ ret = ltdb_index_add_new(module, ltdb, msg);
if (ret != LDB_SUCCESS) {
return ret;
}
@@ -436,6 +437,8 @@ static int ltdb_add(struct ltdb_context *ctx)
{
struct ldb_module *module = ctx->module;
struct ldb_request *req = ctx->req;
+ void *data = ldb_module_get_private(module);
+ struct ltdb_private *ltdb = talloc_get_type(data, struct ltdb_private);
int ret = LDB_SUCCESS;
ret = ltdb_check_special_dn(module, req->op.add.message);
@@ -449,7 +452,8 @@ static int ltdb_add(struct ltdb_context *ctx)
return LDB_ERR_OPERATIONS_ERROR;
}
- ret = ltdb_add_internal(module, req->op.add.message, true);
+ ret = ltdb_add_internal(module, ltdb,
+ req->op.add.message, true);
return ret;
}
@@ -609,6 +613,7 @@ static int ltdb_msg_add_element(struct ldb_message *msg,
delete all elements having a specified attribute name
*/
static int msg_delete_attribute(struct ldb_module *module,
+ struct ltdb_private *ltdb,
struct ldb_message *msg, const char *name)
{
unsigned int i;
@@ -621,7 +626,7 @@ static int msg_delete_attribute(struct ldb_module *module,
}
i = el - msg->elements;
- ret = ltdb_index_del_element(module, msg->dn, el);
+ ret = ltdb_index_del_element(module, ltdb, msg->dn, el);
if (ret != LDB_SUCCESS) {
return ret;
}
@@ -643,6 +648,7 @@ static int msg_delete_attribute(struct ldb_module *module,
return LDB Error on failure
*/
static int msg_delete_element(struct ldb_module *module,
+ struct ltdb_private *ltdb,
struct ldb_message *msg,
const char *name,
const struct ldb_val *val)
@@ -675,10 +681,11 @@ static int msg_delete_element(struct ldb_module *module,
}
if (matched) {
if (el->num_values == 1) {
- return msg_delete_attribute(module, msg, name);
+ return msg_delete_attribute(module,
+ ltdb, msg, name);
}
- ret = ltdb_index_del_value(module, msg->dn, el, i);
+ ret = ltdb_index_del_value(module, ltdb, msg->dn, el, i);
if (ret != LDB_SUCCESS) {
return ret;
}
@@ -715,6 +722,8 @@ int ltdb_modify_internal(struct ldb_module *module,
struct ldb_request *req)
{
struct ldb_context *ldb = ldb_module_get_ctx(module);
+ void *data = ldb_module_get_private(module);
+ struct ltdb_private *ltdb = talloc_get_type(data, struct ltdb_private);
struct ldb_message *msg2;
unsigned int i, j;
int ret = LDB_SUCCESS, idx;
@@ -800,7 +809,8 @@ int ltdb_modify_internal(struct ldb_module *module,
ret = LDB_ERR_OTHER;
goto done;
}
- ret = ltdb_index_add_element(module, msg2->dn,
+ ret = ltdb_index_add_element(module, ltdb,
+ msg2->dn,
el);
if (ret != LDB_SUCCESS) {
goto done;
@@ -881,7 +891,8 @@ int ltdb_modify_internal(struct ldb_module *module,
el2->values = vals;
el2->num_values += el->num_values;
- ret = ltdb_index_add_element(module, msg2->dn, el);
+ ret = ltdb_index_add_element(module, ltdb,
+ msg2->dn, el);
if (ret != LDB_SUCCESS) {
goto done;
}
@@ -945,7 +956,8 @@ int ltdb_modify_internal(struct ldb_module *module,
}
/* Delete the attribute if it exists in the DB */
- if (msg_delete_attribute(module, msg2,
+ if (msg_delete_attribute(module, ltdb,
+ msg2,
el->name) != 0) {
ret = LDB_ERR_OTHER;
goto done;
@@ -958,7 +970,8 @@ int ltdb_modify_internal(struct ldb_module *module,
goto done;
}
- ret = ltdb_index_add_element(module, msg2->dn, el);
+ ret = ltdb_index_add_element(module, ltdb,
+ msg2->dn, el);
if (ret != LDB_SUCCESS) {
goto done;
}
@@ -974,7 +987,9 @@ int ltdb_modify_internal(struct ldb_module *module,
if (msg->elements[i].num_values == 0) {
/* Delete the whole attribute */
- ret = msg_delete_attribute(module, msg2,
+ ret = msg_delete_attribute(module,
+ ltdb,
+ msg2,
msg->elements[i].name);
if (ret == LDB_ERR_NO_SUCH_ATTRIBUTE &&
control_permissive) {
@@ -991,6 +1006,7 @@ int ltdb_modify_internal(struct ldb_module *module,
/* Delete specified values from an attribute */
for (j=0; j < msg->elements[i].num_values; j++) {
ret = msg_delete_element(module,
+ ltdb,
msg2,
msg->elements[i].name,
&msg->elements[i].values[j]);
@@ -1142,7 +1158,7 @@ static int ltdb_rename(struct ltdb_context *ctx)
* deleted attributes. We could go through all elements but that's
* maybe not the most efficient way
*/
- ret = ltdb_add_internal(module, msg, false);
+ ret = ltdb_add_internal(module, ltdb, msg, false);
talloc_free(msg);
diff --git a/lib/ldb/ldb_tdb/ldb_tdb.h b/lib/ldb/ldb_tdb/ldb_tdb.h
index a391606dd16..7871cbcb6d6 100644
--- a/lib/ldb/ldb_tdb/ldb_tdb.h
+++ b/lib/ldb/ldb_tdb/ldb_tdb.h
@@ -82,13 +82,21 @@ int ltdb_check_at_attributes_values(const struct ldb_val *value);
struct ldb_parse_tree;
int ltdb_search_indexed(struct ltdb_context *ctx, uint32_t *);
-int ltdb_index_add_new(struct ldb_module *module, const struct ldb_message *msg);
+int ltdb_index_add_new(struct ldb_module *module,
+ struct ltdb_private *ltdb,
+ const struct ldb_message *msg);
int ltdb_index_delete(struct ldb_module *module, const struct ldb_message *msg);
-int ltdb_index_del_element(struct ldb_module *module, struct ldb_dn *dn,
+int ltdb_index_del_element(struct ldb_module *module,
+ struct ltdb_private *ltdb,
+ struct ldb_dn *dn,
struct ldb_message_element *el);
-int ltdb_index_add_element(struct ldb_module *module, struct ldb_dn *dn,
+int ltdb_index_add_element(struct ldb_module *module,
+ struct ltdb_private *ltdb,
+ struct ldb_dn *dn,
struct ldb_message_element *el);
-int ltdb_index_del_value(struct ldb_module *module, struct ldb_dn *dn,
+int ltdb_index_del_value(struct ldb_module *module,
+ struct ltdb_private *ltdb,
+ struct ldb_dn *dn,
struct ldb_message_element *el, unsigned int v_idx);
int ltdb_reindex(struct ldb_module *module);
int ltdb_index_transaction_start(struct ldb_module *module);
--
2.11.0
From 0b16aae45d065a30485a0c8b2137948f4b3a9745 Mon Sep 17 00:00:00 2001
From: Andrew Bartlett <abartlet at samba.org>
Date: Tue, 15 Aug 2017 17:21:28 +1200
Subject: [PATCH 5/8] ldb_tdb: Add helper function
ltdb_search_and_return_base()
This avoids an extra DB lookup for the base, when that is the only
record we will return, and avoids going into the index code for
a base search, as that won't work for special DNs once the GUID
index mode is enabled.
Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Garming Sam <garming at catalyst.net.nz>
(cherry picked from commit cd8ecb3332b1939a07c3455ec3a66ca3d23bb32d)
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13191
---
lib/ldb/ldb_tdb/ldb_search.c | 100 ++++++++++++++++++++++++++++++++++++++++---
1 file changed, 95 insertions(+), 5 deletions(-)
diff --git a/lib/ldb/ldb_tdb/ldb_search.c b/lib/ldb/ldb_tdb/ldb_search.c
index a6c408a53ce..117f6858a17 100644
--- a/lib/ldb/ldb_tdb/ldb_search.c
+++ b/lib/ldb/ldb_tdb/ldb_search.c
@@ -501,6 +501,81 @@ static int ltdb_search_full(struct ltdb_context *ctx)
return ctx->error;
}
+static int ltdb_search_and_return_base(struct ltdb_private *ltdb,
+ struct ltdb_context *ctx)
+{
+ struct ldb_message *msg, *filtered_msg;
+ struct ldb_context *ldb = ldb_module_get_ctx(ctx->module);
+ int ret;
+ bool matched;
+
+ msg = ldb_msg_new(ctx);
+ if (!msg) {
+ return LDB_ERR_OPERATIONS_ERROR;
+ }
+ ret = ltdb_search_dn1(ctx->module, ctx->base, msg,
+ LDB_UNPACK_DATA_FLAG_NO_DATA_ALLOC|
+ LDB_UNPACK_DATA_FLAG_NO_VALUES_ALLOC);
+
+ if (ret == LDB_ERR_NO_SUCH_OBJECT) {
+ if (ltdb->check_base == false) {
+ /*
+ * In this case, we are done, as no base
+ * checking is allowed in this DB
+ */
+ talloc_free(msg);
+ return LDB_SUCCESS;
+ }
+ ldb_asprintf_errstring(ldb,
+ "No such Base DN: %s",
+ ldb_dn_get_linearized(ctx->base));
+ }
+ if (ret != LDB_SUCCESS) {
+ talloc_free(msg);
+ return ret;
+ }
+
+
+ /*
+ * We use this, not ldb_match_msg_error() as we know
+ * we matched on the scope BASE, as we just fetched
+ * the base DN
+ */
+
+ ret = ldb_match_message(ldb, msg,
+ ctx->tree,
+ ctx->scope,
+ &matched);
+ if (ret != LDB_SUCCESS) {
+ talloc_free(msg);
+ return ret;
+ }
+ if (!matched) {
+ talloc_free(msg);
+ return LDB_SUCCESS;
+ }
+
+ /* filter the attributes that the user wants */
+ ret = ltdb_filter_attrs(ctx, msg, ctx->attrs, &filtered_msg);
+
+ talloc_free(msg);
+
+ if (ret == -1) {
+ return LDB_ERR_OPERATIONS_ERROR;
+ }
+
+ ret = ldb_module_send_entry(ctx->req, filtered_msg, NULL);
+ if (ret != LDB_SUCCESS) {
+ /* Regardless of success or failure, the msg
+ * is the callbacks responsiblity, and should
+ * not be talloc_free()'ed */
+ ctx->request_terminated = true;
+ return ret;
+ }
+
+ return LDB_SUCCESS;
+}
+
/*
search the database with a LDAP-like expression.
choses a search method
@@ -532,6 +607,11 @@ int ltdb_search(struct ltdb_context *ctx)
return LDB_ERR_OPERATIONS_ERROR;
}
+ ctx->tree = req->op.search.tree;
+ ctx->scope = req->op.search.scope;
+ ctx->base = req->op.search.base;
+ ctx->attrs = req->op.search.attrs;
+
if ((req->op.search.base == NULL) || (ldb_dn_is_null(req->op.search.base) == true)) {
/* Check what we should do with a NULL dn */
@@ -559,6 +639,21 @@ int ltdb_search(struct ltdb_context *ctx)
ldb_dn_get_linearized(req->op.search.base));
ret = LDB_ERR_INVALID_DN_SYNTAX;
+ } else if (req->op.search.scope == LDB_SCOPE_BASE) {
+
+ /*
+ * If we are LDB_SCOPE_BASE, do just one search and
+ * return early. This is critical to ensure we do not
+ * go into the index code for special DNs, as that
+ * will try to look up an index record for a special
+ * record (which doesn't exist).
+ */
+ ret = ltdb_search_and_return_base(ltdb, ctx);
+
+ ltdb_unlock_read(module);
+
+ return ret;
+
} else if (ltdb->check_base) {
/* This database has been marked as 'checkBaseOnSearch', so do a spot check of the base dn */
ret = ltdb_search_base(module, req->op.search.base);
@@ -574,11 +669,6 @@ int ltdb_search(struct ltdb_context *ctx)
ret = LDB_SUCCESS;
}
- ctx->tree = req->op.search.tree;
- ctx->scope = req->op.search.scope;
- ctx->base = req->op.search.base;
- ctx->attrs = req->op.search.attrs;
-
if (ret == LDB_SUCCESS) {
uint32_t match_count = 0;
--
2.11.0
From 7c6e2557e91257cfa42a2b0b3a91136436d5fc11 Mon Sep 17 00:00:00 2001
From: Andrew Bartlett <abartlet at samba.org>
Date: Mon, 28 Aug 2017 14:55:00 +1200
Subject: [PATCH 6/8] ldb_tdb: Make ldb_match_message() available to ldb_tdb
Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Garming Sam <garming at catalyst.net.nz>
(cherry picked from commit d8a22884c1089f35e6df111c1f3f65b0848a1438)
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13191
---
lib/ldb/common/ldb_match.c | 22 +++++++++++++++-------
lib/ldb/include/ldb_private.h | 19 +++++++++++++++++++
2 files changed, 34 insertions(+), 7 deletions(-)
diff --git a/lib/ldb/common/ldb_match.c b/lib/ldb/common/ldb_match.c
index 1415fac59f9..25fe3f9c21b 100644
--- a/lib/ldb/common/ldb_match.c
+++ b/lib/ldb/common/ldb_match.c
@@ -515,17 +515,18 @@ static int ldb_match_extended(struct ldb_context *ldb,
}
/*
- return 0 if the given parse tree matches the given message. Assumes
- the message is in sorted order
+ Check if a particular message will match the given filter
- return 1 if it matches, and 0 if it doesn't match
+ set *matched to true if it matches, false otherwise
+
+ returns LDB_SUCCESS or an error
this is a recursive function, and does short-circuit evaluation
*/
-static int ldb_match_message(struct ldb_context *ldb,
- const struct ldb_message *msg,
- const struct ldb_parse_tree *tree,
- enum ldb_scope scope, bool *matched)
+int ldb_match_message(struct ldb_context *ldb,
+ const struct ldb_message *msg,
+ const struct ldb_parse_tree *tree,
+ enum ldb_scope scope, bool *matched)
{
unsigned int i;
int ret;
@@ -587,6 +588,13 @@ static int ldb_match_message(struct ldb_context *ldb,
return LDB_ERR_INAPPROPRIATE_MATCHING;
}
+/*
+ return 0 if the given parse tree matches the given message. Assumes
+ the message is in sorted order
+
+ return 1 if it matches, and 0 if it doesn't match
+*/
+
int ldb_match_msg(struct ldb_context *ldb,
const struct ldb_message *msg,
const struct ldb_parse_tree *tree,
diff --git a/lib/ldb/include/ldb_private.h b/lib/ldb/include/ldb_private.h
index ae7ec3cfd0d..ab215411a9a 100644
--- a/lib/ldb/include/ldb_private.h
+++ b/lib/ldb/include/ldb_private.h
@@ -287,5 +287,24 @@ int ldb_msg_find_duplicate_val(struct ldb_context *ldb,
const struct ldb_message_element *el,
struct ldb_val **duplicate,
uint32_t options);
+/**
+ Check if a particular message will match the given filter
+
+ \param ldb an ldb context
+ \param msg the message to be checked
+ \param tree the filter tree to check against
+ \param scope the scope to match against
+ (to avoid matching special DNs except on a base search)
+ \param matched a pointer to a boolean set true if it matches,
+ false otherwise
+
+ returns LDB_SUCCESS or an error
+
+ \note this is a recursive function, and does short-circuit evaluation
+ */
+int ldb_match_message(struct ldb_context *ldb,
+ const struct ldb_message *msg,
+ const struct ldb_parse_tree *tree,
+ enum ldb_scope scope, bool *matched);
#endif
--
2.11.0
From 55b4404fbb1969bbfc665a803d9c6c9bab12d4fe Mon Sep 17 00:00:00 2001
From: Andrew Bartlett <abartlet at samba.org>
Date: Tue, 29 Aug 2017 09:59:54 +1200
Subject: [PATCH 7/8] ldb_tdb: Trust the BASE and ONELEVEL index
This avoids re-checking the fetched DN against the scope
Signed-off-by: Andrew Bartlett <abartlet at samba.org>
(re-worked from commit 93b18984bd4603ad14a3ed71f4e10b43443803e0)
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13191
---
lib/ldb/ldb_tdb/ldb_index.c | 23 +++++++++++++++++++++--
1 file changed, 21 insertions(+), 2 deletions(-)
diff --git a/lib/ldb/ldb_tdb/ldb_index.c b/lib/ldb/ldb_tdb/ldb_index.c
index ddcf0124dab..dc74f7b9184 100644
--- a/lib/ldb/ldb_tdb/ldb_index.c
+++ b/lib/ldb/ldb_tdb/ldb_index.c
@@ -1007,8 +1007,18 @@ static int ltdb_index_filter(struct ltdb_private *ltdb,
return LDB_ERR_OPERATIONS_ERROR;
}
- ret = ldb_match_msg_error(ldb, msg,
- ac->tree, ac->base, ac->scope, &matched);
+ /* We trust the index for SCOPE_ONELEVEL and SCOPE_BASE */
+ if ((ac->scope == LDB_SCOPE_ONELEVEL
+ && ltdb->cache->one_level_indexes)
+ || ac->scope == LDB_SCOPE_BASE) {
+ ret = ldb_match_message(ldb, msg, ac->tree,
+ ac->scope, &matched);
+ } else {
+ ret = ldb_match_msg_error(ldb, msg,
+ ac->tree, ac->base,
+ ac->scope, &matched);
+ }
+
if (ret != LDB_SUCCESS) {
talloc_free(msg);
return ret;
@@ -1113,6 +1123,11 @@ int ltdb_search_indexed(struct ltdb_context *ac, uint32_t *match_count)
talloc_free(dn_list);
return LDB_ERR_OPERATIONS_ERROR;
}
+ /*
+ * If we ever start to also load the index values for
+ * the tree, we must ensure we strictly intersect with
+ * this list, as we trust the ONELEVEL index
+ */
ret = ltdb_index_dn_one(ac->module, ac->base, dn_list);
if (ret != LDB_SUCCESS) {
talloc_free(dn_list);
@@ -1126,6 +1141,10 @@ int ltdb_search_indexed(struct ltdb_context *ac, uint32_t *match_count)
talloc_free(dn_list);
return LDB_ERR_OPERATIONS_ERROR;
}
+ /*
+ * Here we load the index for the tree. We have no
+ * index for the subtree.
+ */
ret = ltdb_index_dn(ac->module, ltdb, ac->tree, dn_list);
if (ret != LDB_SUCCESS) {
talloc_free(dn_list);
--
2.11.0
From f81baa40523795f8c5fee346abe614617a40461a Mon Sep 17 00:00:00 2001
From: Andrew Bartlett <abartlet at samba.org>
Date: Mon, 18 Dec 2017 10:14:31 +1300
Subject: [PATCH 8/8] ldb: Release 1.2.3
- backported performance fixes for SCOPE_ONELEVEL
Signed-off-by: Andrew Bartlett <abartlet at samba.org>
---
lib/ldb/ABI/ldb-1.2.3.sigs | 278 ++++++++++++++++++++++++++++++++++
lib/ldb/ABI/pyldb-util-1.2.3.sigs | 2 +
lib/ldb/ABI/pyldb-util.py3-1.2.3.sigs | 2 +
lib/ldb/wscript | 2 +-
4 files changed, 283 insertions(+), 1 deletion(-)
create mode 100644 lib/ldb/ABI/ldb-1.2.3.sigs
create mode 100644 lib/ldb/ABI/pyldb-util-1.2.3.sigs
create mode 100644 lib/ldb/ABI/pyldb-util.py3-1.2.3.sigs
diff --git a/lib/ldb/ABI/ldb-1.2.3.sigs b/lib/ldb/ABI/ldb-1.2.3.sigs
new file mode 100644
index 00000000000..88c285e357a
--- /dev/null
+++ b/lib/ldb/ABI/ldb-1.2.3.sigs
@@ -0,0 +1,278 @@
+ldb_add: int (struct ldb_context *, const struct ldb_message *)
+ldb_any_comparison: int (struct ldb_context *, void *, ldb_attr_handler_t, const struct ldb_val *, const struct ldb_val *)
+ldb_asprintf_errstring: void (struct ldb_context *, const char *, ...)
+ldb_attr_casefold: char *(TALLOC_CTX *, const char *)
+ldb_attr_dn: int (const char *)
+ldb_attr_in_list: int (const char * const *, const char *)
+ldb_attr_list_copy: const char **(TALLOC_CTX *, const char * const *)
+ldb_attr_list_copy_add: const char **(TALLOC_CTX *, const char * const *, const char *)
+ldb_base64_decode: int (char *)
+ldb_base64_encode: char *(TALLOC_CTX *, const char *, int)
+ldb_binary_decode: struct ldb_val (TALLOC_CTX *, const char *)
+ldb_binary_encode: char *(TALLOC_CTX *, struct ldb_val)
+ldb_binary_encode_string: char *(TALLOC_CTX *, const char *)
+ldb_build_add_req: int (struct ldb_request **, struct ldb_context *, TALLOC_CTX *, const struct ldb_message *, struct ldb_control **, void *, ldb_request_callback_t, struct ldb_request *)
+ldb_build_del_req: int (struct ldb_request **, struct ldb_context *, TALLOC_CTX *, struct ldb_dn *, struct ldb_control **, void *, ldb_request_callback_t, struct ldb_request *)
+ldb_build_extended_req: int (struct ldb_request **, struct ldb_context *, TALLOC_CTX *, const char *, void *, struct ldb_control **, void *, ldb_request_callback_t, struct ldb_request *)
+ldb_build_mod_req: int (struct ldb_request **, struct ldb_context *, TALLOC_CTX *, const struct ldb_message *, struct ldb_control **, void *, ldb_request_callback_t, struct ldb_request *)
+ldb_build_rename_req: int (struct ldb_request **, struct ldb_context *, TALLOC_CTX *, struct ldb_dn *, struct ldb_dn *, struct ldb_control **, void *, ldb_request_callback_t, struct ldb_request *)
+ldb_build_search_req: int (struct ldb_request **, struct ldb_context *, TALLOC_CTX *, struct ldb_dn *, enum ldb_scope, const char *, const char * const *, struct ldb_control **, void *, ldb_request_callback_t, struct ldb_request *)
+ldb_build_search_req_ex: int (struct ldb_request **, struct ldb_context *, TALLOC_CTX *, struct ldb_dn *, enum ldb_scope, struct ldb_parse_tree *, const char * const *, struct ldb_control **, void *, ldb_request_callback_t, struct ldb_request *)
+ldb_casefold: char *(struct ldb_context *, TALLOC_CTX *, const char *, size_t)
+ldb_casefold_default: char *(void *, TALLOC_CTX *, const char *, size_t)
+ldb_check_critical_controls: int (struct ldb_control **)
+ldb_comparison_binary: int (struct ldb_context *, void *, const struct ldb_val *, const struct ldb_val *)
+ldb_comparison_fold: int (struct ldb_context *, void *, const struct ldb_val *, const struct ldb_val *)
+ldb_connect: int (struct ldb_context *, const char *, unsigned int, const char **)
+ldb_control_to_string: char *(TALLOC_CTX *, const struct ldb_control *)
+ldb_controls_except_specified: struct ldb_control **(struct ldb_control **, TALLOC_CTX *, struct ldb_control *)
+ldb_debug: void (struct ldb_context *, enum ldb_debug_level, const char *, ...)
+ldb_debug_add: void (struct ldb_context *, const char *, ...)
+ldb_debug_end: void (struct ldb_context *, enum ldb_debug_level)
+ldb_debug_set: void (struct ldb_context *, enum ldb_debug_level, const char *, ...)
+ldb_delete: int (struct ldb_context *, struct ldb_dn *)
+ldb_dn_add_base: bool (struct ldb_dn *, struct ldb_dn *)
+ldb_dn_add_base_fmt: bool (struct ldb_dn *, const char *, ...)
+ldb_dn_add_child: bool (struct ldb_dn *, struct ldb_dn *)
+ldb_dn_add_child_fmt: bool (struct ldb_dn *, const char *, ...)
+ldb_dn_alloc_casefold: char *(TALLOC_CTX *, struct ldb_dn *)
+ldb_dn_alloc_linearized: char *(TALLOC_CTX *, struct ldb_dn *)
+ldb_dn_canonical_ex_string: char *(TALLOC_CTX *, struct ldb_dn *)
+ldb_dn_canonical_string: char *(TALLOC_CTX *, struct ldb_dn *)
+ldb_dn_check_local: bool (struct ldb_module *, struct ldb_dn *)
+ldb_dn_check_special: bool (struct ldb_dn *, const char *)
+ldb_dn_compare: int (struct ldb_dn *, struct ldb_dn *)
+ldb_dn_compare_base: int (struct ldb_dn *, struct ldb_dn *)
+ldb_dn_copy: struct ldb_dn *(TALLOC_CTX *, struct ldb_dn *)
+ldb_dn_escape_value: char *(TALLOC_CTX *, struct ldb_val)
+ldb_dn_extended_add_syntax: int (struct ldb_context *, unsigned int, const struct ldb_dn_extended_syntax *)
+ldb_dn_extended_filter: void (struct ldb_dn *, const char * const *)
+ldb_dn_extended_syntax_by_name: const struct ldb_dn_extended_syntax *(struct ldb_context *, const char *)
+ldb_dn_from_ldb_val: struct ldb_dn *(TALLOC_CTX *, struct ldb_context *, const struct ldb_val *)
+ldb_dn_get_casefold: const char *(struct ldb_dn *)
+ldb_dn_get_comp_num: int (struct ldb_dn *)
+ldb_dn_get_component_name: const char *(struct ldb_dn *, unsigned int)
+ldb_dn_get_component_val: const struct ldb_val *(struct ldb_dn *, unsigned int)
+ldb_dn_get_extended_comp_num: int (struct ldb_dn *)
+ldb_dn_get_extended_component: const struct ldb_val *(struct ldb_dn *, const char *)
+ldb_dn_get_extended_linearized: char *(TALLOC_CTX *, struct ldb_dn *, int)
+ldb_dn_get_ldb_context: struct ldb_context *(struct ldb_dn *)
+ldb_dn_get_linearized: const char *(struct ldb_dn *)
+ldb_dn_get_parent: struct ldb_dn *(TALLOC_CTX *, struct ldb_dn *)
+ldb_dn_get_rdn_name: const char *(struct ldb_dn *)
+ldb_dn_get_rdn_val: const struct ldb_val *(struct ldb_dn *)
+ldb_dn_has_extended: bool (struct ldb_dn *)
+ldb_dn_is_null: bool (struct ldb_dn *)
+ldb_dn_is_special: bool (struct ldb_dn *)
+ldb_dn_is_valid: bool (struct ldb_dn *)
+ldb_dn_map_local: struct ldb_dn *(struct ldb_module *, void *, struct ldb_dn *)
+ldb_dn_map_rebase_remote: struct ldb_dn *(struct ldb_module *, void *, struct ldb_dn *)
+ldb_dn_map_remote: struct ldb_dn *(struct ldb_module *, void *, struct ldb_dn *)
+ldb_dn_minimise: bool (struct ldb_dn *)
+ldb_dn_new: struct ldb_dn *(TALLOC_CTX *, struct ldb_context *, const char *)
+ldb_dn_new_fmt: struct ldb_dn *(TALLOC_CTX *, struct ldb_context *, const char *, ...)
+ldb_dn_remove_base_components: bool (struct ldb_dn *, unsigned int)
+ldb_dn_remove_child_components: bool (struct ldb_dn *, unsigned int)
+ldb_dn_remove_extended_components: void (struct ldb_dn *)
+ldb_dn_replace_components: bool (struct ldb_dn *, struct ldb_dn *)
+ldb_dn_set_component: int (struct ldb_dn *, int, const char *, const struct ldb_val)
+ldb_dn_set_extended_component: int (struct ldb_dn *, const char *, const struct ldb_val *)
+ldb_dn_update_components: int (struct ldb_dn *, const struct ldb_dn *)
+ldb_dn_validate: bool (struct ldb_dn *)
+ldb_dump_results: void (struct ldb_context *, struct ldb_result *, FILE *)
+ldb_error_at: int (struct ldb_context *, int, const char *, const char *, int)
+ldb_errstring: const char *(struct ldb_context *)
+ldb_extended: int (struct ldb_context *, const char *, void *, struct ldb_result **)
+ldb_extended_default_callback: int (struct ldb_request *, struct ldb_reply *)
+ldb_filter_from_tree: char *(TALLOC_CTX *, const struct ldb_parse_tree *)
+ldb_get_config_basedn: struct ldb_dn *(struct ldb_context *)
+ldb_get_create_perms: unsigned int (struct ldb_context *)
+ldb_get_default_basedn: struct ldb_dn *(struct ldb_context *)
+ldb_get_event_context: struct tevent_context *(struct ldb_context *)
+ldb_get_flags: unsigned int (struct ldb_context *)
+ldb_get_opaque: void *(struct ldb_context *, const char *)
+ldb_get_root_basedn: struct ldb_dn *(struct ldb_context *)
+ldb_get_schema_basedn: struct ldb_dn *(struct ldb_context *)
+ldb_global_init: int (void)
+ldb_handle_get_event_context: struct tevent_context *(struct ldb_handle *)
+ldb_handle_new: struct ldb_handle *(TALLOC_CTX *, struct ldb_context *)
+ldb_handle_use_global_event_context: void (struct ldb_handle *)
+ldb_handler_copy: int (struct ldb_context *, void *, const struct ldb_val *, struct ldb_val *)
+ldb_handler_fold: int (struct ldb_context *, void *, const struct ldb_val *, struct ldb_val *)
+ldb_init: struct ldb_context *(TALLOC_CTX *, struct tevent_context *)
+ldb_ldif_message_redacted_string: char *(struct ldb_context *, TALLOC_CTX *, enum ldb_changetype, const struct ldb_message *)
+ldb_ldif_message_string: char *(struct ldb_context *, TALLOC_CTX *, enum ldb_changetype, const struct ldb_message *)
+ldb_ldif_parse_modrdn: int (struct ldb_context *, const struct ldb_ldif *, TALLOC_CTX *, struct ldb_dn **, struct ldb_dn **, bool *, struct ldb_dn **, struct ldb_dn **)
+ldb_ldif_read: struct ldb_ldif *(struct ldb_context *, int (*)(void *), void *)
+ldb_ldif_read_file: struct ldb_ldif *(struct ldb_context *, FILE *)
+ldb_ldif_read_file_state: struct ldb_ldif *(struct ldb_context *, struct ldif_read_file_state *)
+ldb_ldif_read_free: void (struct ldb_context *, struct ldb_ldif *)
+ldb_ldif_read_string: struct ldb_ldif *(struct ldb_context *, const char **)
+ldb_ldif_write: int (struct ldb_context *, int (*)(void *, const char *, ...), void *, const struct ldb_ldif *)
+ldb_ldif_write_file: int (struct ldb_context *, FILE *, const struct ldb_ldif *)
+ldb_ldif_write_redacted_trace_string: char *(struct ldb_context *, TALLOC_CTX *, const struct ldb_ldif *)
+ldb_ldif_write_string: char *(struct ldb_context *, TALLOC_CTX *, const struct ldb_ldif *)
+ldb_load_modules: int (struct ldb_context *, const char **)
+ldb_map_add: int (struct ldb_module *, struct ldb_request *)
+ldb_map_delete: int (struct ldb_module *, struct ldb_request *)
+ldb_map_init: int (struct ldb_module *, const struct ldb_map_attribute *, const struct ldb_map_objectclass *, const char * const *, const char *, const char *)
+ldb_map_modify: int (struct ldb_module *, struct ldb_request *)
+ldb_map_rename: int (struct ldb_module *, struct ldb_request *)
+ldb_map_search: int (struct ldb_module *, struct ldb_request *)
+ldb_match_message: int (struct ldb_context *, const struct ldb_message *, const struct ldb_parse_tree *, enum ldb_scope, bool *)
+ldb_match_msg: int (struct ldb_context *, const struct ldb_message *, const struct ldb_parse_tree *, struct ldb_dn *, enum ldb_scope)
+ldb_match_msg_error: int (struct ldb_context *, const struct ldb_message *, const struct ldb_parse_tree *, struct ldb_dn *, enum ldb_scope, bool *)
+ldb_match_msg_objectclass: int (const struct ldb_message *, const char *)
+ldb_mod_register_control: int (struct ldb_module *, const char *)
+ldb_modify: int (struct ldb_context *, const struct ldb_message *)
+ldb_modify_default_callback: int (struct ldb_request *, struct ldb_reply *)
+ldb_module_call_chain: char *(struct ldb_request *, TALLOC_CTX *)
+ldb_module_connect_backend: int (struct ldb_context *, const char *, const char **, struct ldb_module **)
+ldb_module_done: int (struct ldb_request *, struct ldb_control **, struct ldb_extended *, int)
+ldb_module_flags: uint32_t (struct ldb_context *)
+ldb_module_get_ctx: struct ldb_context *(struct ldb_module *)
+ldb_module_get_name: const char *(struct ldb_module *)
+ldb_module_get_ops: const struct ldb_module_ops *(struct ldb_module *)
+ldb_module_get_private: void *(struct ldb_module *)
+ldb_module_init_chain: int (struct ldb_context *, struct ldb_module *)
+ldb_module_load_list: int (struct ldb_context *, const char **, struct ldb_module *, struct ldb_module **)
+ldb_module_new: struct ldb_module *(TALLOC_CTX *, struct ldb_context *, const char *, const struct ldb_module_ops *)
+ldb_module_next: struct ldb_module *(struct ldb_module *)
+ldb_module_popt_options: struct poptOption **(struct ldb_context *)
+ldb_module_send_entry: int (struct ldb_request *, struct ldb_message *, struct ldb_control **)
+ldb_module_send_referral: int (struct ldb_request *, char *)
+ldb_module_set_next: void (struct ldb_module *, struct ldb_module *)
+ldb_module_set_private: void (struct ldb_module *, void *)
+ldb_modules_hook: int (struct ldb_context *, enum ldb_module_hook_type)
+ldb_modules_list_from_string: const char **(struct ldb_context *, TALLOC_CTX *, const char *)
+ldb_modules_load: int (const char *, const char *)
+ldb_msg_add: int (struct ldb_message *, const struct ldb_message_element *, int)
+ldb_msg_add_empty: int (struct ldb_message *, const char *, int, struct ldb_message_element **)
+ldb_msg_add_fmt: int (struct ldb_message *, const char *, const char *, ...)
+ldb_msg_add_linearized_dn: int (struct ldb_message *, const char *, struct ldb_dn *)
+ldb_msg_add_steal_string: int (struct ldb_message *, const char *, char *)
+ldb_msg_add_steal_value: int (struct ldb_message *, const char *, struct ldb_val *)
+ldb_msg_add_string: int (struct ldb_message *, const char *, const char *)
+ldb_msg_add_value: int (struct ldb_message *, const char *, const struct ldb_val *, struct ldb_message_element **)
+ldb_msg_canonicalize: struct ldb_message *(struct ldb_context *, const struct ldb_message *)
+ldb_msg_check_string_attribute: int (const struct ldb_message *, const char *, const char *)
+ldb_msg_copy: struct ldb_message *(TALLOC_CTX *, const struct ldb_message *)
+ldb_msg_copy_attr: int (struct ldb_message *, const char *, const char *)
+ldb_msg_copy_shallow: struct ldb_message *(TALLOC_CTX *, const struct ldb_message *)
+ldb_msg_diff: struct ldb_message *(struct ldb_context *, struct ldb_message *, struct ldb_message *)
+ldb_msg_difference: int (struct ldb_context *, TALLOC_CTX *, struct ldb_message *, struct ldb_message *, struct ldb_message **)
+ldb_msg_element_compare: int (struct ldb_message_element *, struct ldb_message_element *)
+ldb_msg_element_compare_name: int (struct ldb_message_element *, struct ldb_message_element *)
+ldb_msg_element_equal_ordered: bool (const struct ldb_message_element *, const struct ldb_message_element *)
+ldb_msg_find_attr_as_bool: int (const struct ldb_message *, const char *, int)
+ldb_msg_find_attr_as_dn: struct ldb_dn *(struct ldb_context *, TALLOC_CTX *, const struct ldb_message *, const char *)
+ldb_msg_find_attr_as_double: double (const struct ldb_message *, const char *, double)
+ldb_msg_find_attr_as_int: int (const struct ldb_message *, const char *, int)
+ldb_msg_find_attr_as_int64: int64_t (const struct ldb_message *, const char *, int64_t)
+ldb_msg_find_attr_as_string: const char *(const struct ldb_message *, const char *, const char *)
+ldb_msg_find_attr_as_uint: unsigned int (const struct ldb_message *, const char *, unsigned int)
+ldb_msg_find_attr_as_uint64: uint64_t (const struct ldb_message *, const char *, uint64_t)
+ldb_msg_find_common_values: int (struct ldb_context *, TALLOC_CTX *, struct ldb_message_element *, struct ldb_message_element *, uint32_t)
+ldb_msg_find_duplicate_val: int (struct ldb_context *, TALLOC_CTX *, const struct ldb_message_element *, struct ldb_val **, uint32_t)
+ldb_msg_find_element: struct ldb_message_element *(const struct ldb_message *, const char *)
+ldb_msg_find_ldb_val: const struct ldb_val *(const struct ldb_message *, const char *)
+ldb_msg_find_val: struct ldb_val *(const struct ldb_message_element *, struct ldb_val *)
+ldb_msg_new: struct ldb_message *(TALLOC_CTX *)
+ldb_msg_normalize: int (struct ldb_context *, TALLOC_CTX *, const struct ldb_message *, struct ldb_message **)
+ldb_msg_remove_attr: void (struct ldb_message *, const char *)
+ldb_msg_remove_element: void (struct ldb_message *, struct ldb_message_element *)
+ldb_msg_rename_attr: int (struct ldb_message *, const char *, const char *)
+ldb_msg_sanity_check: int (struct ldb_context *, const struct ldb_message *)
+ldb_msg_sort_elements: void (struct ldb_message *)
+ldb_next_del_trans: int (struct ldb_module *)
+ldb_next_end_trans: int (struct ldb_module *)
+ldb_next_init: int (struct ldb_module *)
+ldb_next_prepare_commit: int (struct ldb_module *)
+ldb_next_read_lock: int (struct ldb_module *)
+ldb_next_read_unlock: int (struct ldb_module *)
+ldb_next_remote_request: int (struct ldb_module *, struct ldb_request *)
+ldb_next_request: int (struct ldb_module *, struct ldb_request *)
+ldb_next_start_trans: int (struct ldb_module *)
+ldb_op_default_callback: int (struct ldb_request *, struct ldb_reply *)
+ldb_options_find: const char *(struct ldb_context *, const char **, const char *)
+ldb_pack_data: int (struct ldb_context *, const struct ldb_message *, struct ldb_val *)
+ldb_parse_control_from_string: struct ldb_control *(struct ldb_context *, TALLOC_CTX *, const char *)
+ldb_parse_control_strings: struct ldb_control **(struct ldb_context *, TALLOC_CTX *, const char **)
+ldb_parse_tree: struct ldb_parse_tree *(TALLOC_CTX *, const char *)
+ldb_parse_tree_attr_replace: void (struct ldb_parse_tree *, const char *, const char *)
+ldb_parse_tree_copy_shallow: struct ldb_parse_tree *(TALLOC_CTX *, const struct ldb_parse_tree *)
+ldb_parse_tree_walk: int (struct ldb_parse_tree *, int (*)(struct ldb_parse_tree *, void *), void *)
+ldb_qsort: void (void * const, size_t, size_t, void *, ldb_qsort_cmp_fn_t)
+ldb_register_backend: int (const char *, ldb_connect_fn, bool)
+ldb_register_extended_match_rule: int (struct ldb_context *, const struct ldb_extended_match_rule *)
+ldb_register_hook: int (ldb_hook_fn)
+ldb_register_module: int (const struct ldb_module_ops *)
+ldb_rename: int (struct ldb_context *, struct ldb_dn *, struct ldb_dn *)
+ldb_reply_add_control: int (struct ldb_reply *, const char *, bool, void *)
+ldb_reply_get_control: struct ldb_control *(struct ldb_reply *, const char *)
+ldb_req_get_custom_flags: uint32_t (struct ldb_request *)
+ldb_req_is_untrusted: bool (struct ldb_request *)
+ldb_req_location: const char *(struct ldb_request *)
+ldb_req_mark_trusted: void (struct ldb_request *)
+ldb_req_mark_untrusted: void (struct ldb_request *)
+ldb_req_set_custom_flags: void (struct ldb_request *, uint32_t)
+ldb_req_set_location: void (struct ldb_request *, const char *)
+ldb_request: int (struct ldb_context *, struct ldb_request *)
+ldb_request_add_control: int (struct ldb_request *, const char *, bool, void *)
+ldb_request_done: int (struct ldb_request *, int)
+ldb_request_get_control: struct ldb_control *(struct ldb_request *, const char *)
+ldb_request_get_status: int (struct ldb_request *)
+ldb_request_replace_control: int (struct ldb_request *, const char *, bool, void *)
+ldb_request_set_state: void (struct ldb_request *, int)
+ldb_reset_err_string: void (struct ldb_context *)
+ldb_save_controls: int (struct ldb_control *, struct ldb_request *, struct ldb_control ***)
+ldb_schema_attribute_add: int (struct ldb_context *, const char *, unsigned int, const char *)
+ldb_schema_attribute_add_with_syntax: int (struct ldb_context *, const char *, unsigned int, const struct ldb_schema_syntax *)
+ldb_schema_attribute_by_name: const struct ldb_schema_attribute *(struct ldb_context *, const char *)
+ldb_schema_attribute_fill_with_syntax: int (struct ldb_context *, TALLOC_CTX *, const char *, unsigned int, const struct ldb_schema_syntax *, struct ldb_schema_attribute *)
+ldb_schema_attribute_remove: void (struct ldb_context *, const char *)
+ldb_schema_attribute_remove_flagged: void (struct ldb_context *, unsigned int)
+ldb_schema_attribute_set_override_handler: void (struct ldb_context *, ldb_attribute_handler_override_fn_t, void *)
+ldb_schema_set_override_indexlist: void (struct ldb_context *, bool)
+ldb_search: int (struct ldb_context *, TALLOC_CTX *, struct ldb_result **, struct ldb_dn *, enum ldb_scope, const char * const *, const char *, ...)
+ldb_search_default_callback: int (struct ldb_request *, struct ldb_reply *)
+ldb_sequence_number: int (struct ldb_context *, enum ldb_sequence_type, uint64_t *)
+ldb_set_create_perms: void (struct ldb_context *, unsigned int)
+ldb_set_debug: int (struct ldb_context *, void (*)(void *, enum ldb_debug_level, const char *, va_list), void *)
+ldb_set_debug_stderr: int (struct ldb_context *)
+ldb_set_default_dns: void (struct ldb_context *)
+ldb_set_errstring: void (struct ldb_context *, const char *)
+ldb_set_event_context: void (struct ldb_context *, struct tevent_context *)
+ldb_set_flags: void (struct ldb_context *, unsigned int)
+ldb_set_modules_dir: void (struct ldb_context *, const char *)
+ldb_set_opaque: int (struct ldb_context *, const char *, void *)
+ldb_set_require_private_event_context: void (struct ldb_context *)
+ldb_set_timeout: int (struct ldb_context *, struct ldb_request *, int)
+ldb_set_timeout_from_prev_req: int (struct ldb_context *, struct ldb_request *, struct ldb_request *)
+ldb_set_utf8_default: void (struct ldb_context *)
+ldb_set_utf8_fns: void (struct ldb_context *, void *, char *(*)(void *, void *, const char *, size_t))
+ldb_setup_wellknown_attributes: int (struct ldb_context *)
+ldb_should_b64_encode: int (struct ldb_context *, const struct ldb_val *)
+ldb_standard_syntax_by_name: const struct ldb_schema_syntax *(struct ldb_context *, const char *)
+ldb_strerror: const char *(int)
+ldb_string_to_time: time_t (const char *)
+ldb_string_utc_to_time: time_t (const char *)
+ldb_timestring: char *(TALLOC_CTX *, time_t)
+ldb_timestring_utc: char *(TALLOC_CTX *, time_t)
+ldb_transaction_cancel: int (struct ldb_context *)
+ldb_transaction_cancel_noerr: int (struct ldb_context *)
+ldb_transaction_commit: int (struct ldb_context *)
+ldb_transaction_prepare_commit: int (struct ldb_context *)
+ldb_transaction_start: int (struct ldb_context *)
+ldb_unpack_data: int (struct ldb_context *, const struct ldb_val *, struct ldb_message *)
+ldb_unpack_data_only_attr_list: int (struct ldb_context *, const struct ldb_val *, struct ldb_message *, const char * const *, unsigned int, unsigned int *)
+ldb_unpack_data_only_attr_list_flags: int (struct ldb_context *, const struct ldb_val *, struct ldb_message *, const char * const *, unsigned int, unsigned int, unsigned int *)
+ldb_val_dup: struct ldb_val (TALLOC_CTX *, const struct ldb_val *)
+ldb_val_equal_exact: int (const struct ldb_val *, const struct ldb_val *)
+ldb_val_map_local: struct ldb_val (struct ldb_module *, void *, const struct ldb_map_attribute *, const struct ldb_val *)
+ldb_val_map_remote: struct ldb_val (struct ldb_module *, void *, const struct ldb_map_attribute *, const struct ldb_val *)
+ldb_val_string_cmp: int (const struct ldb_val *, const char *)
+ldb_val_to_time: int (const struct ldb_val *, time_t *)
+ldb_valid_attr_name: int (const char *)
+ldb_vdebug: void (struct ldb_context *, enum ldb_debug_level, const char *, va_list)
+ldb_wait: int (struct ldb_handle *, enum ldb_wait_type)
diff --git a/lib/ldb/ABI/pyldb-util-1.2.3.sigs b/lib/ldb/ABI/pyldb-util-1.2.3.sigs
new file mode 100644
index 00000000000..74d6719d2bc
--- /dev/null
+++ b/lib/ldb/ABI/pyldb-util-1.2.3.sigs
@@ -0,0 +1,2 @@
+pyldb_Dn_FromDn: PyObject *(struct ldb_dn *)
+pyldb_Object_AsDn: bool (TALLOC_CTX *, PyObject *, struct ldb_context *, struct ldb_dn **)
diff --git a/lib/ldb/ABI/pyldb-util.py3-1.2.3.sigs b/lib/ldb/ABI/pyldb-util.py3-1.2.3.sigs
new file mode 100644
index 00000000000..74d6719d2bc
--- /dev/null
+++ b/lib/ldb/ABI/pyldb-util.py3-1.2.3.sigs
@@ -0,0 +1,2 @@
+pyldb_Dn_FromDn: PyObject *(struct ldb_dn *)
+pyldb_Object_AsDn: bool (TALLOC_CTX *, PyObject *, struct ldb_context *, struct ldb_dn **)
diff --git a/lib/ldb/wscript b/lib/ldb/wscript
index bd17b7b9edc..5ea52317d4d 100644
--- a/lib/ldb/wscript
+++ b/lib/ldb/wscript
@@ -1,7 +1,7 @@
#!/usr/bin/env python
APPNAME = 'ldb'
-VERSION = '1.2.2'
+VERSION = '1.2.3'
blddir = 'bin'
--
2.11.0
-------------- next part --------------
From 15686577239dbcc8d7e70330ff6dc91f517d7ec1 Mon Sep 17 00:00:00 2001
From: Andrew Bartlett <abartlet at samba.org>
Date: Fri, 15 Dec 2017 11:40:28 +1300
Subject: [PATCH 1/3] dns_server: Do not look for a wildcard for @
This query is made for every record returned via BIND9 DLZ.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13191
Signed-off-by: Andrew Bartlett <abartlet at samba.org>
---
source4/dns_server/dnsserver_common.c | 10 ++++++++++
1 file changed, 10 insertions(+)
diff --git a/source4/dns_server/dnsserver_common.c b/source4/dns_server/dnsserver_common.c
index 217e65b39f4..093898361d0 100644
--- a/source4/dns_server/dnsserver_common.c
+++ b/source4/dns_server/dnsserver_common.c
@@ -547,6 +547,16 @@ WERROR dns_common_wildcard_lookup(struct ldb_context *samdb,
return DNS_ERR(NAME_ERROR);
}
+ /* Don't look for a wildcard for @ */
+ if (name->length == 1 && name->data[0] == '@') {
+ return dns_common_lookup(samdb,
+ mem_ctx,
+ dn,
+ records,
+ num_records,
+ NULL);
+ }
+
werr = dns_name_check(
mem_ctx,
strlen((const char*)name->data),
--
2.11.0
From 99e1b1c784c171415b7e44003032fbb87884597b Mon Sep 17 00:00:00 2001
From: Andrew Bartlett <abartlet at samba.org>
Date: Fri, 15 Dec 2017 11:41:15 +1300
Subject: [PATCH 2/3] dns_server: Do not embed a trailing NUL in the query
The data to match on does not include the NUL, just the significant bytes
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13191
Signed-off-by: Andrew Bartlett <abartlet at samba.org>
---
source4/dns_server/dnsserver_common.c | 10 ++++++----
1 file changed, 6 insertions(+), 4 deletions(-)
diff --git a/source4/dns_server/dnsserver_common.c b/source4/dns_server/dnsserver_common.c
index 093898361d0..574d1510230 100644
--- a/source4/dns_server/dnsserver_common.c
+++ b/source4/dns_server/dnsserver_common.c
@@ -250,9 +250,11 @@ static struct ldb_parse_tree *build_equality_operation(
struct ldb_parse_tree *el = NULL; /* Equality node being built */
struct ldb_val *value = NULL; /* Value the attr will be compared
with */
- size_t length = 0; /* calculated length of the value
- including option '*' prefix and
- '\0' string terminator */
+ size_t length = 0; /* calculated length of the
+ value including option
+ '*' prefix but no '\0'
+ string terminator (this
+ is an ldb_val) */
el = talloc(mem_ctx, struct ldb_parse_tree);
if (el == NULL) {
@@ -263,7 +265,7 @@ static struct ldb_parse_tree *build_equality_operation(
el->operation = LDB_OP_EQUALITY;
el->u.equality.attr = talloc_strdup(mem_ctx, attr);
value = &el->u.equality.value;
- length = (add_asterix) ? size + 2 : size + 1;
+ length = (add_asterix) ? size + 1 : size;
value->data = talloc_zero_array(el, uint8_t, length);
if (el == NULL) {
DBG_ERR("Unable to allocate value->data\n");
--
2.11.0
From 1d991af0cc71a19def27daa4ce86314594209568 Mon Sep 17 00:00:00 2001
From: Andrew Bartlett <abartlet at samba.org>
Date: Fri, 15 Dec 2017 12:30:50 +1300
Subject: [PATCH 3/3] dns_server: Do the exact match query first, then do the
wildcard lookup
The wildcard lookup is SCOPE_ONELEVEL which in current ldb will load the
full index for that zone.
This is inefficient as we do not currently intersect that index with the
index on DN=
Not that a not-found and wildcard response will still go via the ONELEVEL
index for now so this only helps for correct, directly matching queries.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13191
Signed-off-by: Andrew Bartlett <abartlet at samba.org>
---
source4/dns_server/dnsserver_common.c | 14 ++++++++++++++
1 file changed, 14 insertions(+)
diff --git a/source4/dns_server/dnsserver_common.c b/source4/dns_server/dnsserver_common.c
index 574d1510230..368d32326b0 100644
--- a/source4/dns_server/dnsserver_common.c
+++ b/source4/dns_server/dnsserver_common.c
@@ -567,6 +567,20 @@ WERROR dns_common_wildcard_lookup(struct ldb_context *samdb,
return werr;
}
+ /*
+ * Do a point search first, then fall back to a wildcard
+ * lookup if it does not exist
+ */
+ werr = dns_common_lookup(samdb,
+ mem_ctx,
+ dn,
+ records,
+ num_records,
+ NULL);
+ if (!W_ERROR_EQUAL(werr, WERR_DNS_ERROR_NAME_DOES_NOT_EXIST)) {
+ return werr;
+ }
+
ret = dns_wildcard_lookup(samdb, mem_ctx, dn, &msg);
if (ret == LDB_ERR_OPERATIONS_ERROR) {
return DNS_ERR(SERVER_FAILURE);
--
2.11.0
More information about the samba-technical
mailing list