[SCM] Samba Shared Repository - branch master updated
Andrew Bartlett
abartlet at samba.org
Wed May 1 06:47:02 UTC 2019
The branch, master has been updated
via bd53819b28b script/attr_count_read: load and correlate all data
via 60620273dba dsdb/modules: a module to count attribute searches and results
via a047e71bc7d pytests: slightly better errors in Testcase.insta_creds()
via e5a099482d9 pytests: try ldap.modify_order with normal user
via c73888ff6f2 dsdb pytests: test the effect of reordering modify requests
via 865e464d5e5 s4/tests.py: shorten lines with common path
via fd9859d407f dsdb/pytest/ldap: use idiomatic 'e' for exceptions
via 5a0df7aec6b dsdb/pytest/ldap: revive commented out test for attr size range
via 0a5c5e2f5ef .gitlab-ci.yml: keep samba-ci-private tag only for private jobs
from 2a5bf72b00b s4:samdb: Make sure value is initialized with 0
https://git.samba.org/?p=samba.git;a=shortlog;h=master
- Log -----------------------------------------------------------------
commit bd53819b28bab04408fc7fd7cfecc04a9aff9baf
Author: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Date: Sun Mar 31 16:07:57 2019 +1300
script/attr_count_read: load and correlate all data
This changes script/attr_count_read to take the samba private directory
as an argument and load all the databases at once, printing them as
one big table. It isn't extremely clear what it all means, but it
*tries* to tell you.
With --plot, it will attempt to load matplotlib and plot the number of
requested attributes against the number returned, with colour
of each point indicating its relative frequency. It is a scatterplot
that wants to be a heatmap.
With --no-casefold, you can get an extra confusing table where,
for instance, something repeatedly asks for "attributeId" which is not
accounted for, while in a completely different row an unrequested
"attributeID" is found many times over.
Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Wed May 1 06:46:36 UTC 2019 on sn-devel-184
commit 60620273dba1d7f7ff25710c5dd8fd6d32f2d149
Author: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Date: Thu Mar 28 16:07:48 2019 +1300
dsdb/modules: a module to count attribute searches and results
The dsdb module stack can turn a simple search request into a
complicated tree of sub-queries that include attributes not originally
asked for and excluding those that were. The corresponding replies
might contain unrequested attributes or (for good reasons, according
to some module) hide requested ones. The entire stack is there to
meddle and that is what is does. Except *this* module. It just counts.
To understand dsdb performance it helps to have some idea what
requests and replies are flying too and fro. This module, when
inserted anywhere in the stack, counts the requests and replies
passing through and the attributes they contain. This data is stored
in on-disk tdbs in the private/debug directory.
The module is not loaded by default. To load it you need to patch the
source4/dsdb/samdb/ldb_modules/samba_dsdb.c and put "count_attrs"
somewhere in the module lists in the samba_dsdb_init() function. For
example, to examine the traffic between repl_meta_data and
group_audit_log, you would do something like this around line 316:
"subtree_delete",
"repl_meta_data",
+ "count_attrs",
"group_audit_log",
"encrypted_secrets",
and recompile. Samba will then write to a number of tdb files in the
debug directory as requests and replies pass through. A simple script
is included to read these files. Doing this:
./script/attr_count_read st/ad_dc/private/debug/debug/attr_counts_not_found.tdb
will print a table showing how often various attritbutes were
requested but not found (from the point of view of the module).
A more sophisticated version of the script is coming in the next
commit, but this one is included first because in its simplicity it
documents the storage format reasonably well. The tdb keys are
attribute names, and the values are uint32_t in machine native order.
When the module is included in the stack there will be a very small
decrease in performance.
Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
commit a047e71bc7daea3855fc786c0e1c9d09878bba5f
Author: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Date: Thu Apr 18 13:37:27 2019 +1200
pytests: slightly better errors in Testcase.insta_creds()
Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
commit e5a099482d9ad17913065351f8c3d02a2b48c03d
Author: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Date: Thu Apr 18 13:39:58 2019 +1200
pytests: try ldap.modify_order with normal user
We run the tests again, trying to modify as a normal user rather than
Administrator.
It turns out that we do not always return the same error code as
Windows, but in all these tests both Windows and Samba always return
some kind of error (as you might hope).
Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
commit c73888ff6f22de86f72ff254d332b44848cb9959
Author: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Date: Sat Apr 13 00:23:26 2019 +1200
dsdb pytests: test the effect of reordering modify requests
Do we interpret these the same way as Windows? In many cases, no.
Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
commit 865e464d5e5ab7d2aae297cce7a20674162a5e06
Author: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Date: Sat Apr 13 00:21:29 2019 +1200
s4/tests.py: shorten lines with common path
A small step
Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
commit fd9859d407f4e38d159081713b5959ba7ba23aed
Author: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Date: Fri Apr 12 15:51:05 2019 +1200
dsdb/pytest/ldap: use idiomatic 'e' for exceptions
Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
commit 5a0df7aec6bd37c7f9ce0b0ed5758da7a4977ee2
Author: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Date: Fri Apr 12 16:09:55 2019 +1200
dsdb/pytest/ldap: revive commented out test for attr size range
The test was presumably commented out because we fail it, and
known-failing it would have hidden the attr-too-short tests that it
was bundled with. If we disentangle them we can knwn-fail it, which
serves as a TODO list.
(passes against WIN2012R2).
Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
commit 0a5c5e2f5ef09b8cee55d65242d54febc84f7855
Author: Joe Guo <joeg at catalyst.net.nz>
Date: Wed May 1 15:18:31 2019 +1200
.gitlab-ci.yml: keep samba-ci-private tag only for private jobs
This will help us give the legacy 'private' tag, used in branches
under maintenance, more resources without those jobs running on the
normal production runners (therefore avoiding the additional cost for
the 90% of builds that are for master).
Signed-off-by: Joe Guo <joeg at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
-----------------------------------------------------------------------
Summary of changes:
.gitlab-ci.yml | 2 -
python/samba/tests/__init__.py | 7 +-
script/attr_count_read | 199 ++++++
selftest/knownfail.d/ldap | 2 +
selftest/knownfail.d/modify-order | 8 +
source4/dsdb/samdb/ldb_modules/count_attrs.c | 640 ++++++++++++++++++
.../dsdb/samdb/ldb_modules/wscript_build_server | 10 +
source4/dsdb/tests/python/ldap.py | 489 +++++++-------
source4/dsdb/tests/python/ldap_modify_order.py | 372 ++++++++++
...rder_account_locality_device-non-admin.expected | 31 +
.../modify_order_account_locality_device.expected | 34 +
...modify_order_container_flags-non-admin.expected | 129 ++++
.../testdata/modify_order_container_flags.expected | 134 ++++
...r_container_flags_multivalue-non-admin.expected | 127 ++++
...odify_order_container_flags_multivalue.expected | 138 ++++
.../modify_order_inapplicable-non-admin.expected | 31 +
.../testdata/modify_order_inapplicable.expected | 34 +
.../modify_order_member-non-admin.expected | 127 ++++
.../python/testdata/modify_order_member.expected | 190 ++++++
.../testdata/modify_order_mixed-non-admin.expected | 128 ++++
.../python/testdata/modify_order_mixed.expected | 143 ++++
.../modify_order_mixed2-non-admin.expected | 128 ++++
.../python/testdata/modify_order_mixed2.expected | 143 ++++
.../modify_order_objectclass-non-admin.expected | 31 +
.../testdata/modify_order_objectclass.expected | 35 +
.../modify_order_objectclass2-non-admin.expected | 726 ++++++++++++++++++++
.../testdata/modify_order_objectclass2.expected | 735 ++++++++++++++++++++
.../modify_order_singlevalue-non-admin.expected | 727 ++++++++++++++++++++
.../testdata/modify_order_singlevalue.expected | 740 ++++++++++++++++++++
...order_sometimes_inapplicable-non-admin.expected | 127 ++++
.../modify_order_sometimes_inapplicable.expected | 127 ++++
.../modify_order_telephone-non-admin.expected | 727 ++++++++++++++++++++
.../testdata/modify_order_telephone.expected | 752 +++++++++++++++++++++
...rder_telephone_delete_delete-non-admin.expected | 727 ++++++++++++++++++++
.../modify_order_telephone_delete_delete.expected | 736 ++++++++++++++++++++
source4/selftest/tests.py | 72 +-
36 files changed, 9242 insertions(+), 266 deletions(-)
create mode 100755 script/attr_count_read
create mode 100644 selftest/knownfail.d/ldap
create mode 100644 selftest/knownfail.d/modify-order
create mode 100644 source4/dsdb/samdb/ldb_modules/count_attrs.c
create mode 100644 source4/dsdb/tests/python/ldap_modify_order.py
create mode 100644 source4/dsdb/tests/python/testdata/modify_order_account_locality_device-non-admin.expected
create mode 100644 source4/dsdb/tests/python/testdata/modify_order_account_locality_device.expected
create mode 100644 source4/dsdb/tests/python/testdata/modify_order_container_flags-non-admin.expected
create mode 100644 source4/dsdb/tests/python/testdata/modify_order_container_flags.expected
create mode 100644 source4/dsdb/tests/python/testdata/modify_order_container_flags_multivalue-non-admin.expected
create mode 100644 source4/dsdb/tests/python/testdata/modify_order_container_flags_multivalue.expected
create mode 100644 source4/dsdb/tests/python/testdata/modify_order_inapplicable-non-admin.expected
create mode 100644 source4/dsdb/tests/python/testdata/modify_order_inapplicable.expected
create mode 100644 source4/dsdb/tests/python/testdata/modify_order_member-non-admin.expected
create mode 100644 source4/dsdb/tests/python/testdata/modify_order_member.expected
create mode 100644 source4/dsdb/tests/python/testdata/modify_order_mixed-non-admin.expected
create mode 100644 source4/dsdb/tests/python/testdata/modify_order_mixed.expected
create mode 100644 source4/dsdb/tests/python/testdata/modify_order_mixed2-non-admin.expected
create mode 100644 source4/dsdb/tests/python/testdata/modify_order_mixed2.expected
create mode 100644 source4/dsdb/tests/python/testdata/modify_order_objectclass-non-admin.expected
create mode 100644 source4/dsdb/tests/python/testdata/modify_order_objectclass.expected
create mode 100644 source4/dsdb/tests/python/testdata/modify_order_objectclass2-non-admin.expected
create mode 100644 source4/dsdb/tests/python/testdata/modify_order_objectclass2.expected
create mode 100644 source4/dsdb/tests/python/testdata/modify_order_singlevalue-non-admin.expected
create mode 100644 source4/dsdb/tests/python/testdata/modify_order_singlevalue.expected
create mode 100644 source4/dsdb/tests/python/testdata/modify_order_sometimes_inapplicable-non-admin.expected
create mode 100644 source4/dsdb/tests/python/testdata/modify_order_sometimes_inapplicable.expected
create mode 100644 source4/dsdb/tests/python/testdata/modify_order_telephone-non-admin.expected
create mode 100644 source4/dsdb/tests/python/testdata/modify_order_telephone.expected
create mode 100644 source4/dsdb/tests/python/testdata/modify_order_telephone_delete_delete-non-admin.expected
create mode 100644 source4/dsdb/tests/python/testdata/modify_order_telephone_delete_delete.expected
Changeset truncated at 500 lines:
diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 5401a9e27b9..bff0e089058 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -154,8 +154,6 @@ samba-ad-dc-ntvfs:
.private_template:
extends: .shared_template
tags:
- - docker
- - private
- samba-ci-private
only:
variables:
diff --git a/python/samba/tests/__init__.py b/python/samba/tests/__init__.py
index f904499b90b..cbd164de1f9 100644
--- a/python/samba/tests/__init__.py
+++ b/python/samba/tests/__init__.py
@@ -113,10 +113,11 @@ class TestCase(unittest.TestCase):
def insta_creds(self, template=None, username=None, userpass=None, kerberos_state=None):
if template is None:
- assert template is not None
+ raise ValueError("you need to supply a Credentials template")
- if username is not None:
- assert userpass is not None
+ if username is not None and userpass is None:
+ raise ValueError(
+ "you cannot set creds username without setting a password")
if username is None:
assert userpass is None
diff --git a/script/attr_count_read b/script/attr_count_read
new file mode 100755
index 00000000000..4338b6bee10
--- /dev/null
+++ b/script/attr_count_read
@@ -0,0 +1,199 @@
+#!/usr/bin/env python3
+#
+# Copyright (C) Catalyst IT Ltd. 2019
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+#
+
+from __future__ import print_function
+import sys
+import argparse
+import struct
+import os
+from collections import OrderedDict, Counter
+from pprint import pprint
+
+sys.path.insert(0, "bin/python")
+import tdb
+
+
+def unpack_uint(filename, casefold=True):
+ db = tdb.Tdb(filename)
+ d = {}
+ for k in db:
+ v = struct.unpack("I", db[k])[0]
+ k2 = k.decode('utf-8')
+ if casefold:
+ k2 = k2.lower()
+ if k2 in d: # because casefold
+ d[k2] += v
+ else:
+ d[k2] = v
+ return d
+
+
+def unpack_ssize_t_pair(filename, casefold):
+ db = tdb.Tdb(filename)
+ pairs = []
+ for k in db:
+ key = struct.unpack("nn", k)
+ v = struct.unpack("I", db[k])[0]
+ pairs.append((v, key))
+
+ pairs.sort(reverse=True)
+ #print(pairs)
+ return [(k, v) for (v, k) in pairs]
+
+
+DATABASES = [
+ ('requested', "debug/attr_counts_requested.tdb", unpack_uint,
+ "The attribute was specifically requested."),
+ ('duplicates', "debug/attr_counts_duplicates.tdb", unpack_uint,
+ "Requested more than once in the same request."),
+ ('empty request', "debug/attr_counts_empty_req.tdb", unpack_uint,
+ "No attributes were requested, but these were returned"),
+ ('null request', "debug/attr_counts_null_req.tdb", unpack_uint,
+ "The attribute list was NULL and these were returned."),
+ ('found', "debug/attr_counts_found.tdb", unpack_uint,
+ "The attribute was specifically requested and it was found."),
+ ('not found', "debug/attr_counts_not_found.tdb", unpack_uint,
+ "The attribute was specifically requested but was not found."),
+ ('unwanted', "debug/attr_counts_unwanted.tdb", unpack_uint,
+ "The attribute was not requested and it was found."),
+ ('star match', "debug/attr_counts_star_match.tdb", unpack_uint,
+ 'The attribute was not specifically requested but "*" was.'),
+ ('req vs found', "debug/attr_counts_req_vs_found.tdb", unpack_ssize_t_pair,
+ "How many attributes were requested versus how many were returned."),
+]
+
+
+def plot_pair_data(name, data, doc, lim=90):
+ # Note we keep the matplotlib import internal to this function for
+ # two reasons:
+ # 1. Some people won't have matplotlib, but might want to run the
+ # script.
+ # 2. The import takes hundreds of milliseconds, which is a
+ # nuisance if you don't wat graphs.
+ #
+ # This plot could be improved!
+ import matplotlib.pylab as plt
+ fig, ax = plt.subplots()
+ if lim:
+ data2 = []
+ for p, c in data:
+ if p[0] > lim or p[1] > lim:
+ print("not plotting %s: %s" % (p, c))
+ continue
+ data2.append((p, c))
+ skipped = len(data) - len(data2)
+ if skipped:
+ name += " (excluding %d out of range values)" % skipped
+ data = data2
+ xy, counts = zip(*data)
+ x, y = zip(*xy)
+ bins_x = max(x) + 4
+ bins_y = max(y)
+ ax.set_title(name)
+ ax.scatter(x, y, c=counts)
+ plt.show()
+
+
+def print_pair_data(name, data, doc):
+ print(name)
+ print(doc)
+ t = "%14s | %14s | %14s"
+ print(t % ("requested", "returned", "count"))
+ print(t % (('-' * 14,) * 3))
+
+ for xy, count in data:
+ x, y = xy
+ if x == -2:
+ x = 'NULL'
+ elif x == -4:
+ x = '*'
+ print(t % (x, y, count))
+
+
+def print_counts(count_data):
+ all_attrs = Counter()
+ for c in count_data:
+ all_attrs.update(c[1])
+
+ print("found %d attrs" % len(all_attrs))
+ longest = max(len(x) for x in all_attrs)
+
+ #pprint(all_attrs)
+ rows = OrderedDict()
+ for a, _ in all_attrs.most_common():
+ rows[a] = [a]
+
+ for col_name, counts, doc in count_data:
+ for attr, row in rows.items():
+ d = counts.get(attr, '')
+ row.append(d)
+
+ print("%15s: %s" % (col_name, doc))
+ print()
+
+ t = "%{}s".format(longest)
+ for c in count_data:
+ t += " | %{}s".format(max(len(c[0]), 7))
+
+ h = t % (("attribute",) + tuple(c[0] for c in count_data))
+ print(h)
+ print("-" * len(h))
+
+ for attr, row in rows.items():
+ print(t % tuple(row))
+ pass
+
+
+def main():
+ parser = argparse.ArgumentParser()
+ parser.add_argument('LDB_PRIVATE_DIR',
+ help="read attr counts in this directory")
+ parser.add_argument('--plot', action="store_true",
+ help='attempt to draw graphs')
+ parser.add_argument('--no-casefold', action="store_false",
+ default=True, dest="casefold",
+ help='See all the encountered case varients')
+ args = parser.parse_args()
+
+ if not os.path.isdir(args.LDB_PRIVATE_DIR):
+ parser.print_usage()
+ sys.exit(1)
+
+ count_data = []
+ pair_data = []
+ for k, filename, unpacker, doc in DATABASES:
+ filename = os.path.join(args.LDB_PRIVATE_DIR, filename)
+ try:
+ d = unpacker(filename, casefold=args.casefold)
+ except (RuntimeError, IOError) as e:
+ print("could not parse %s: %s" % (filename, e))
+ continue
+ if unpacker is unpack_ssize_t_pair:
+ pair_data.append((k, d, doc))
+ else:
+ count_data.append((k, d, doc))
+
+ for k, v, doc in pair_data:
+ if args.plot:
+ plot_pair_data(k, v, doc)
+ print_pair_data(k, v, doc)
+
+ print()
+ print_counts(count_data)
+
+main()
diff --git a/selftest/knownfail.d/ldap b/selftest/knownfail.d/ldap
new file mode 100644
index 00000000000..5bb01bc38bf
--- /dev/null
+++ b/selftest/knownfail.d/ldap
@@ -0,0 +1,2 @@
+# the attributes too long test returns the wrong error
+^samba4.ldap.python.+test_attribute_ranges_too_long
\ No newline at end of file
diff --git a/selftest/knownfail.d/modify-order b/selftest/knownfail.d/modify-order
new file mode 100644
index 00000000000..e14cd1eb356
--- /dev/null
+++ b/selftest/knownfail.d/modify-order
@@ -0,0 +1,8 @@
+samba4.ldap_modify_order.python.+ModifyOrderTests.test_modify_order_account_locality_device
+samba4.ldap_modify_order.python.+ModifyOrderTests.test_modify_order_container_flags_multivalue
+samba4.ldap_modify_order.python.+ModifyOrderTests.test_modify_order_objectclass
+samba4.ldap_modify_order.python.+ModifyOrderTests.test_modify_order_objectclass2
+samba4.ldap_modify_order.python.+ModifyOrderTests.test_modify_order_singlevalue
+samba4.ldap_modify_order.normal_user.+ModifyOrderTests.test_modify_order_account_locality_device
+samba4.ldap_modify_order.normal_user.+ModifyOrderTests.test_modify_order_container_flags[^_]
+samba4.ldap_modify_order.normal_user.+ModifyOrderTests.test_modify_order_objectclass2
diff --git a/source4/dsdb/samdb/ldb_modules/count_attrs.c b/source4/dsdb/samdb/ldb_modules/count_attrs.c
new file mode 100644
index 00000000000..2518492e813
--- /dev/null
+++ b/source4/dsdb/samdb/ldb_modules/count_attrs.c
@@ -0,0 +1,640 @@
+/*
+ ldb database library
+
+ Copyright (C) Andrew Bartlett <abartlet at samba.org> 2019
+
+ This program is free software; you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation; either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>.
+*/
+
+/*
+ * Count how often different attributes are searched for, for performance
+ * analysis. The counts are stored in tdb files in the 'debug' subdirectory of
+ * Samba installation's private directory, and can be read using
+ * script/attr_count_read.
+ */
+
+#include "includes.h"
+#include "ldb_module.h"
+#include "param/param.h"
+#include "lib/tdb_wrap/tdb_wrap.h"
+#include "system/filesys.h"
+
+#define NULL_ATTRS "__null_attrs__"
+#define EMPTY_ATTRS "__empty_attrs__"
+#define UNKNOWN_ATTR "__unknown_attribute__"
+#define STAR_ATTR "*"
+
+#define NULL_REQ_PSEUDO_N -2LL;
+#define STAR_REQ_PSEUDO_N -4LL;
+
+struct count_attrs_private {
+ struct tdb_wrap *requested;
+ struct tdb_wrap *duplicates;
+ struct tdb_wrap *found;
+ struct tdb_wrap *not_found;
+ struct tdb_wrap *unwanted;
+ struct tdb_wrap *star_match;
+ struct tdb_wrap *null_req;
+ struct tdb_wrap *empty_req;
+ struct tdb_wrap *req_vs_found;
+};
+
+
+struct count_attrs_context {
+ struct ldb_module *module;
+ struct ldb_request *req;
+ bool has_star;
+ bool is_null;
+ const char **requested_attrs;
+ size_t n_attrs;
+};
+
+
+static int add_key(struct tdb_context *tdb,
+ struct TDB_DATA key)
+{
+ int ret;
+ uint32_t one = 1;
+ struct TDB_DATA value = {
+ .dptr = (uint8_t *)&one,
+ .dsize = sizeof(one)
+ };
+ ret = tdb_store(tdb,
+ key,
+ value,
+ 0);
+ return ret;
+}
+
+static int increment_attr_count(struct tdb_context *tdb,
+ const char *attr)
+{
+ /*
+ * Note that as we don't lock the database, there is a small window
+ * between the fetch and store in which identical updates from
+ * separate processes can race to clobber each other. If this happens
+ * the stored count will be one less than it should be.
+ *
+ * We don't worry about that because it should be quite rare and
+ * agnostic as to which counts are affected, meaning the overall
+ * statistical truth is preserved.
+ */
+ int ret;
+ uint32_t *val;
+ TDB_DATA key = {
+ .dptr = discard_const(attr),
+ .dsize = strlen(attr)
+ };
+
+ TDB_DATA data = tdb_fetch(tdb, key);
+ if (data.dptr == NULL) {
+ ret = tdb_error(tdb);
+ if (ret != TDB_ERR_NOEXIST) {
+ const char *errstr = tdb_errorstr(tdb);
+ DBG_ERR("tdb fetch error: %s\n", errstr);
+ return LDB_ERR_OPERATIONS_ERROR;
+ }
+ /* this key is unknown. We'll add it and get out of here. */
+ ret = add_key(tdb, key);
+ if (ret != 0) {
+ DBG_ERR("could not add %s: %d\n", attr, ret);
+ }
+ return ret;
+ }
+
+ val = (uint32_t *)data.dptr;
+ (*val)++;
+
+ ret = tdb_store(tdb,
+ key,
+ data,
+ 0);
+
+ if (ret != 0) {
+ const char *errstr = tdb_errorstr(tdb);
+ DBG_ERR("tdb store error: %s\n", errstr);
+ free(data.dptr);
+ return LDB_ERR_OPERATIONS_ERROR;
+ }
+ free(data.dptr);
+ return LDB_SUCCESS;
+}
+
+
+static int increment_req_vs_found(struct tdb_context *tdb,
+ struct count_attrs_context *ac,
+ size_t n_found)
+{
+ /*
+ * Here we record the number of elements in each reply along with the
+ * number of attributes in the corresponding request. Requests for
+ * NULL and "*" are arbitrarily given the attribute counts -2 and -4
+ * respectively. This leads them to be plotted as two stacks on the
+ * left hand side of the scatter plot.
+ */
+ int ret;
+ ssize_t k[2];
+ uint32_t *val = NULL;
+ TDB_DATA key = {
+ .dptr = (unsigned char *)k,
+ .dsize = sizeof(k)
+ };
+ TDB_DATA data = {NULL};
+ ssize_t n_req = ac->n_attrs;
+ if (ac->is_null) {
+ n_req = NULL_REQ_PSEUDO_N;
+ } else if (ac->has_star) {
+ n_req = STAR_REQ_PSEUDO_N;
+ }
+ k[0] = n_req;
+ k[1] = n_found;
+
+ data = tdb_fetch(tdb, key);
+ if (data.dptr == NULL) {
+ ret = tdb_error(tdb);
+ if (ret != TDB_ERR_NOEXIST) {
+ const char *errstr = tdb_errorstr(tdb);
+ DBG_ERR("req vs found fetch error: %s\n", errstr);
+ return LDB_ERR_OPERATIONS_ERROR;
+ }
+ /* unknown key */
+ ret = add_key(tdb, key);
+ if (ret != 0) {
+ DBG_ERR("could not add req vs found %zu:%zu: %d\n",
+ n_req, n_found, ret);
+ }
+ return ret;
+ }
+
+ val = (uint32_t *)data.dptr;
+ (*val)++;
+
+ ret = tdb_store(tdb, key, data, 0);
+ if (ret != 0) {
+ const char *errstr = tdb_errorstr(tdb);
+ DBG_ERR("req vs found store error: %s\n", errstr);
+ free(data.dptr);
+ return LDB_ERR_OPERATIONS_ERROR;
+ }
+ free(data.dptr);
+ return LDB_SUCCESS;
+}
+
+
+static int strcasecmp_ptr(const char **a, const char **b)
+{
+ return strcasecmp(*a, *b);
+}
+
+
+static const char **get_sorted_attrs(TALLOC_CTX *mem_ctx,
+ const char * const *unsorted_attrs,
+ size_t n_attrs)
+{
+ size_t i;
+ const char **attrs = talloc_array(mem_ctx,
+ const char *,
+ n_attrs);
+
+ if (attrs == NULL) {
+ return NULL;
+ }
+ for (i = 0; i < n_attrs; i++) {
+ const char *a = unsorted_attrs[i];
+ if (a == NULL) {
+ DBG_ERR("attrs have disappeared! "
+ "wanted %zu; got %zu\n",
+ n_attrs, i);
+ talloc_free(attrs);
+ return NULL;
+ }
+ attrs[i] = a;
+ }
+
+ qsort(attrs, n_attrs, sizeof(char *), (__compar_fn_t)strcasecmp_ptr);
+ return attrs;
+}
+
+
+
+static int count_attrs_search_callback(struct ldb_request *req,
+ struct ldb_reply *ares)
+{
+ struct count_attrs_private *priv = NULL;
--
Samba Shared Repository
More information about the samba-cvs
mailing list