[PATCH] Modify samba-tool group commands to view summary of group memberships
Tim Beale
timbeale at catalyst.net.nz
Sun Oct 28 20:46:42 UTC 2018
Tweaked the command output after some feedback from Douglas. Also added
a test for --verbose.
CI pass: https://gitlab.com/catalyst-samba/samba/pipelines/34238919
On 25/10/18 11:48 AM, Tim Beale wrote:
> Currently we don't have a great idea of what group memberships look like
> in a typical AD network, e.g. the max number of group members. Obviously
> this will vary from network to network. But for a large database, the
> number of members in a group could potentially have a big impact on
> whether a particular bug occurs or not, e.g. 1,000 members vs 10,000
> members...
>
> This patch modifies the 'samba-tool group' commands to display more
> information about group membership. Specifically:
> * Extend 'samba-tool group list --verbose' to include how many members
> are in each group.
> * Add a new 'samba-tool group stats' to display summary information
> about the group memberships for the domain.
>
> Example output from the commands is attached.
>
> CI link: https://gitlab.com/catalyst-samba/samba/pipelines/34087257
>
> Review appreciated. Thanks.
>
-------------- next part --------------
From 0d14b76677f4312360f28cba83daddfb206c8e2e Mon Sep 17 00:00:00 2001
From: Tim Beale <timbeale at catalyst.net.nz>
Date: Thu, 18 Oct 2018 16:59:24 +1300
Subject: [PATCH 1/2] netcmd: Include num-members in 'samba-tool group list
--verbose'
This adds an easy way for users to see (via samba-tool) how many members
are in various groups, without querying the members for each individual
group.
For example, you could pipe this output to grep to check for groups with
zero or one members (i.e. historic groups that may no longer make
sense).
Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
---
python/samba/netcmd/group.py | 27 +++++++++++++---------
python/samba/tests/samba_tool/group.py | 41 ++++++++++++++++++++++++++++++++++
2 files changed, 57 insertions(+), 11 deletions(-)
diff --git a/python/samba/netcmd/group.py b/python/samba/netcmd/group.py
index 5158357..6bab0d9 100644
--- a/python/samba/netcmd/group.py
+++ b/python/samba/netcmd/group.py
@@ -324,37 +324,42 @@ class cmd_group_list(Command):
samdb = SamDB(url=H, session_info=system_session(),
credentials=creds, lp=lp)
+ attrs=["samaccountname"]
+ if verbose:
+ attrs += ["grouptype", "member"]
domain_dn = samdb.domain_dn()
res = samdb.search(domain_dn, scope=ldb.SCOPE_SUBTREE,
expression=("(objectClass=group)"),
- attrs=["samaccountname", "grouptype"])
+ attrs=attrs)
if (len(res) == 0):
return
if verbose:
- self.outf.write("Group Name Group Type Group Scope\n")
- self.outf.write("-----------------------------------------------------------------------------\n")
+ self.outf.write("Group Name Group Type Group Scope Members\n")
+ self.outf.write("--------------------------------------------------------------------------------\n")
for msg in res:
self.outf.write("%-44s" % msg.get("samaccountname", idx=0))
hgtype = hex(int("%s" % msg["grouptype"]) & 0x00000000FFFFFFFF)
if (hgtype == hex(int(security_group.get("Builtin")))):
- self.outf.write("Security Builtin\n")
+ self.outf.write("Security Builtin ")
elif (hgtype == hex(int(security_group.get("Domain")))):
- self.outf.write("Security Domain\n")
+ self.outf.write("Security Domain ")
elif (hgtype == hex(int(security_group.get("Global")))):
- self.outf.write("Security Global\n")
+ self.outf.write("Security Global ")
elif (hgtype == hex(int(security_group.get("Universal")))):
- self.outf.write("Security Universal\n")
+ self.outf.write("Security Universal")
elif (hgtype == hex(int(distribution_group.get("Global")))):
- self.outf.write("Distribution Global\n")
+ self.outf.write("Distribution Global ")
elif (hgtype == hex(int(distribution_group.get("Domain")))):
- self.outf.write("Distribution Domain\n")
+ self.outf.write("Distribution Domain ")
elif (hgtype == hex(int(distribution_group.get("Universal")))):
- self.outf.write("Distribution Universal\n")
+ self.outf.write("Distribution Universal")
else:
- self.outf.write("\n")
+ self.outf.write(" ")
+ num_members = len(msg.get("member", default=[]))
+ self.outf.write(" %6u\n" % num_members)
else:
for msg in res:
self.outf.write("%s\n" % msg.get("samaccountname", idx=0))
diff --git a/python/samba/tests/samba_tool/group.py b/python/samba/tests/samba_tool/group.py
index 7a5fd96..ee58347 100644
--- a/python/samba/tests/samba_tool/group.py
+++ b/python/samba/tests/samba_tool/group.py
@@ -117,6 +117,47 @@ class GroupCmdTestCase(SambaToolCmdTest):
found = self.assertMatch(out, name,
"group '%s' not found" % name)
+ def test_list_verbose(self):
+ (result, out, err) = self.runsubcmd("group", "list", "--verbose",
+ "-H", "ldap://%s" % os.environ["DC_SERVER"],
+ "-U%s%%%s" % (os.environ["DC_USERNAME"],
+ os.environ["DC_PASSWORD"]))
+ self.assertCmdSuccess(result, out, err, "Error running list --verbose")
+
+ # use the output to build a dictionary, where key=group-name,
+ # value=num-members
+ output_memberships = {}
+
+ # split the output by line, skipping the first 2 header lines
+ group_lines = out.split('\n')[2:-1]
+ for line in group_lines:
+ # split line by column whitespace (but keep the group name together
+ # if it contains spaces)
+ values = line.split(" ")
+ name = values[0]
+ num_members = int(values[-1])
+ output_memberships[name] = num_members
+
+ # build up a similar dict using an LDAP search
+ search_filter = "(objectClass=group)"
+ grouplist = self.samdb.search(base=self.samdb.domain_dn(),
+ scope=ldb.SCOPE_SUBTREE,
+ expression=search_filter,
+ attrs=["samaccountname", "member"])
+ self.assertTrue(len(grouplist) > 0, "no groups found in samdb")
+
+ ldap_memberships = {}
+ for groupobj in grouplist:
+ name = str(groupobj.get("samaccountname", idx=0))
+ num_members = len(groupobj.get("member", default=[]))
+ ldap_memberships[name] = num_members
+
+ # check the command output matches LDAP
+ self.assertTrue(output_memberships == ldap_memberships,
+ "Command output doesn't match LDAP results.\n" +
+ "Command='%s'\nLDAP='%s'" %(output_memberships,
+ ldap_memberships))
+
def test_listmembers(self):
(result, out, err) = self.runsubcmd("group", "listmembers", "Domain Users",
"-H", "ldap://%s" % os.environ["DC_SERVER"],
--
2.7.4
From 94970b3465ab9bcc9dfda08c0e84732d22be2af1 Mon Sep 17 00:00:00 2001
From: Tim Beale <timbeale at catalyst.net.nz>
Date: Thu, 18 Oct 2018 17:08:32 +1300
Subject: [PATCH 2/2] netcmd: Add 'samba-tool group stats' command
With large domains it's hard to get an idea of how many groups there
are, and how many users are in each group, on average. However, this
could have a big impact on whether a problem can be reproduced or not.
This patch dumps out some summary information so that you can get a
quick idea of how big the groups are.
Signed-off-by: Tim Beale <timbeale at catalyst.net.nz>
---
docs-xml/manpages/samba-tool.8.xml | 5 ++
python/samba/netcmd/group.py | 104 +++++++++++++++++++++++++++++++++
python/samba/tests/samba_tool/group.py | 18 ++++++
3 files changed, 127 insertions(+)
diff --git a/docs-xml/manpages/samba-tool.8.xml b/docs-xml/manpages/samba-tool.8.xml
index 2c043b9..01f5313 100644
--- a/docs-xml/manpages/samba-tool.8.xml
+++ b/docs-xml/manpages/samba-tool.8.xml
@@ -644,6 +644,11 @@
<para>Show group object and it's attributes.</para>
</refsect3>
+<refsect3>
+ <title>group stats [options]</title>
+ <para>Show statistics for overall groups and group memberships.</para>
+</refsect3>
+
<refsect2>
<title>ldapcmp <replaceable>URL1</replaceable> <replaceable>URL2</replaceable> <replaceable>domain|configuration|schema|dnsdomain|dnsforest</replaceable> [options] </title>
<para>Compare two LDAP databases.</para>
diff --git a/python/samba/netcmd/group.py b/python/samba/netcmd/group.py
index 6bab0d9..ba60267 100644
--- a/python/samba/netcmd/group.py
+++ b/python/samba/netcmd/group.py
@@ -35,6 +35,7 @@ from samba.dsdb import (
GTYPE_DISTRIBUTION_GLOBAL_GROUP,
GTYPE_DISTRIBUTION_UNIVERSAL_GROUP,
)
+from collections import defaultdict
security_group = dict({"Builtin": GTYPE_SECURITY_BUILTIN_LOCAL_GROUP,
"Domain": GTYPE_SECURITY_DOMAIN_LOCAL_GROUP,
@@ -589,6 +590,108 @@ Example3 shows how to display a users objectGUID and member attributes.
self.outf.write(user_ldif)
+class cmd_group_stats(Command):
+ """Summary statistics about group memberships."""
+
+ synopsis = "%prog [options]"
+
+ takes_options = [
+ Option("-H", "--URL", help="LDB URL for database or target server", type=str,
+ metavar="URL", dest="H"),
+ ]
+
+ takes_optiongroups = {
+ "sambaopts": options.SambaOptions,
+ "credopts": options.CredentialsOptions,
+ "versionopts": options.VersionOptions,
+ }
+
+ def num_in_range(self, range_min, range_max, group_freqs):
+ total_count = 0
+ for members, count in group_freqs.items():
+ if range_min <= members and members <= range_max:
+ total_count += count
+
+ return total_count
+
+ def run(self, sambaopts=None, credopts=None, versionopts=None, H=None):
+ lp = sambaopts.get_loadparm()
+ creds = credopts.get_credentials(lp, fallback_machine=True)
+
+ samdb = SamDB(url=H, session_info=system_session(),
+ credentials=creds, lp=lp)
+
+ domain_dn = samdb.domain_dn()
+ res = samdb.search(domain_dn, scope=ldb.SCOPE_SUBTREE,
+ expression=("(objectClass=group)"),
+ attrs=["samaccountname", "member"])
+
+ # first count up how many members each group has
+ group_assignments = {}
+ total_memberships = 0
+
+ for msg in res:
+ name = str(msg.get("samaccountname"))
+ num_members = len(msg.get("member", default=[]))
+ group_assignments[name] = num_members
+ total_memberships += num_members
+
+ num_groups = res.count
+ self.outf.write("Group membership statistics*\n")
+ self.outf.write("-------------------------------------------------\n")
+ self.outf.write("Total groups: {0}\n".format(num_groups))
+ self.outf.write("Total memberships: {0}\n".format(total_memberships))
+ average = total_memberships / float(num_groups)
+ self.outf.write("Average members per group: %.2f\n" % average)
+
+ # find the max and median memberships (note that some default groups
+ # always have zero members, so displaying the min is not very helpful)
+ group_names = list(group_assignments.keys())
+ group_members = list(group_assignments.values())
+ idx = group_members.index(max(group_members))
+ max_members = group_members[idx]
+ self.outf.write("Max members: {0} ({1})\n".format(max_members,
+ group_names[idx]))
+ group_members.sort()
+ midpoint = num_groups // 2
+ median = group_members[midpoint]
+ if num_groups % 2 == 0:
+ median = (median + group_members[midpoint - 1]) / 2
+ self.outf.write("Median members per group: {0}\n\n".format(median))
+
+ # convert this to the frequency of group membership, i.e. how many
+ # groups have 5 members, how many have 6 members, etc
+ group_freqs = defaultdict(int)
+ for group, num_members in group_assignments.items():
+ group_freqs[num_members] += 1
+
+ # now squash this down even further, so that we just display the number
+ # of groups that fall into one of the following membership bands
+ bands = [(0, 1), (2, 4), (5, 9), (10, 14), (15, 19), (20, 24),
+ (25, 29), (30, 39), (40, 49), (50, 59), (60, 69), (70, 79),
+ (80, 89), (90, 99), (100, 149), (150, 199), (200, 249),
+ (250, 299), (300, 399), (400, 499), (500, 999), (1000, 1999),
+ (2000, 2999), (3000, 3999), (4000, 4999), (5000, 9999),
+ (10000, max_members)]
+
+ self.outf.write("Members Number of Groups\n")
+ self.outf.write("-------------------------------------------------\n")
+
+ for band in bands:
+ band_start = band[0]
+ band_end = band[1]
+ if band_start > max_members:
+ break
+
+ num_groups = self.num_in_range(band_start, band_end, group_freqs)
+
+ if num_groups != 0:
+ band_str = "{0}-{1}".format(band_start, band_end)
+ self.outf.write("%13s %u\n" % (band_str, num_groups))
+
+ self.outf.write("\n* Note this does not include nested group memberships\n")
+
+
class cmd_group(SuperCommand):
"""Group management."""
@@ -601,3 +704,4 @@ class cmd_group(SuperCommand):
subcommands["listmembers"] = cmd_group_list_members()
subcommands["move"] = cmd_group_move()
subcommands["show"] = cmd_group_show()
+ subcommands["stats"] = cmd_group_stats()
diff --git a/python/samba/tests/samba_tool/group.py b/python/samba/tests/samba_tool/group.py
index ee58347..9862251 100644
--- a/python/samba/tests/samba_tool/group.py
+++ b/python/samba/tests/samba_tool/group.py
@@ -249,3 +249,21 @@ class GroupCmdTestCase(SambaToolCmdTest):
return grouplist[0]
else:
return None
+
+ def test_stats(self):
+ (result, out, err) = self.runsubcmd("group", "stats",
+ "-H", "ldap://%s" % os.environ["DC_SERVER"],
+ "-U%s%%%s" % (os.environ["DC_USERNAME"],
+ os.environ["DC_PASSWORD"]))
+ self.assertCmdSuccess(result, out, err, "Error running stats")
+
+ # sanity-check the command reports 'total groups' correctly
+ search_filter = "(objectClass=group)"
+ grouplist = self.samdb.search(base=self.samdb.domain_dn(),
+ scope=ldb.SCOPE_SUBTREE,
+ expression=search_filter,
+ attrs=[])
+
+ total_groups = len(grouplist)
+ self.assertTrue("Total groups: {0}".format(total_groups) in out,
+ "Total groups not reported correctly")
--
2.7.4
-------------- next part --------------
bin/samba-tool group stats -H ldap://$SERVER -U$USERNAME%$PASSWORD
Group membership statistics*
-------------------------------------------------
Total groups: 72
Total memberships: 2024
Average members per group: 28.11
Max members: 87 (STGG-0-2)
Median members per group: 5
Members Number of Groups
-------------------------------------------------
0-1 33
2-4 3
5-9 1
30-39 9
40-49 6
60-69 8
70-79 10
80-89 2
* Note this does not include nested group memberships
bin/samba-tool group list --verbose -H ldap://$SERVER -U$USERNAME%$PASSWORD
Group Name Group Type Group Scope Members
--------------------------------------------------------------------------------
STGG-0-9 Security Global 70
DnsAdmins Security Domain 0
STGG-0-7 Security Global 71
Performance Log Users Security Builtin 0
Windows Authorization Access Group Security Builtin 1
Read-only Domain Controllers Security Global 0
STGG-0-3 Security Global 68
STGG-0-15 Security Global 67
STGG-0-14 Security Global 70
STGG-0-26 Security Global 35
STGG-0-24 Security Global 35
DnsUpdateProxy Security Global 0
STGG-0-33 Security Global 32
Domain Users Security Global 0
STGG-0-16 Security Global 73
Cert Publishers Security Domain 0
STGG-0-25 Security Global 45
STGG-0-32 Security Global 45
Users Security Builtin 3
STGG-0-23 Security Global 31
Cryptographic Operators Security Builtin 0
Enterprise Read-only Domain Controllers Security Universal 0
STGG-0-20 Security Global 82
STGG-0-12 Security Global 63
Terminal Server License Servers Security Builtin 0
Domain Controllers Security Global 0
Replicator Security Builtin 0
STGG-0-21 Security Global 32
Incoming Forest Trust Builders Security Builtin 0
STGG-0-17 Security Global 77
STGG-0-1 Security Global 75
Distributed COM Users Security Builtin 0
STGG-0-34 Security Global 48
STGG-0-11 Security Global 72
Performance Monitor Users Security Builtin 0
Server Operators Security Builtin 0
STGG-0-28 Security Global 49
STGG-0-10 Security Global 67
Print Operators Security Builtin 0
Domain Admins Security Global 1
STGG-0-4 Security Global 73
STGG-0-30 Security Global 45
STGG-0-5 Security Global 71
STGG-0-31 Security Global 34
STGG-0-29 Security Global 36
STGG-0-35 Security Global 35
Domain Guests Security Global 0
Network Configuration Operators Security Builtin 0
Denied RODC Password Replication Group Security Domain 8
Group Policy Creator Owners Security Global 1
STGG-0-2 Security Global 87
Allowed RODC Password Replication Group Security Domain 1
Domain Computers Security Global 0
Administrators Security Builtin 3
Remote Desktop Users Security Builtin 0
IIS_IUSRS Security Builtin 1
Pre-Windows 2000 Compatible Access Security Builtin 1
STGG-0-8 Security Global 68
Certificate Service DCOM Access Security Builtin 0
Schema Admins Security Universal 1
STGG-0-13 Security Global 66
STGG-0-27 Security Global 36
STGG-0-18 Security Global 61
Backup Operators Security Builtin 0
Account Operators Security Builtin 0
STGG-0-6 Security Global 72
STGG-0-22 Security Global 44
RAS and IAS Servers Security Domain 0
Guests Security Builtin 2
Enterprise Admins Security Universal 1
STGG-0-19 Security Global 65
Event Log Readers Security Builtin 0
More information about the samba-technical
mailing list