[Samba] Slow ldap membership query in large active directory

[Andrew Bartlett]

On Sat, 2024-02-03 at 18:26 +0100, Sören Busse via samba wrote:
> Hey there,
> we've been using Samba AD DC successfully for about 4 years in our
> school with about 1000 people. 4 years ago we decided to create a
> group for each class + subject combination, so we have about 1400
> groups with ~30 members each (some are much bigger up to 800 people
> and others have only a few members). One of our systems, which uses
> LDAP, needs to retrieve the gidNumber of all the groups a user is a
> member of. This request is sent about 3 to 4 times per second (yes,
> this is a design flaw, but we cannot easily change it or enable
> caching):
> We noticed that the query to get all the gidNumbers of the courses
> the user is a member of takes about 370ms, while a simple query takes
> 47ms (including bind/unbind). See the test results below.
> Why is a query on the member attribute so expensive? I would have
> assumed that this very common query would be optimised like an index
> user => [groups], so that you only need to get the gidNumber
> attribute of the remaining groups. Or maybe there's a faster way to
> do the query / optimise the ldap database?
> Thank you very much in advance!

I'm not totally shocked, DN based searches are much more expensive, as
we have to confirm the DN exists and isn't deleted, not just match like
an integer. 
You can work out where we spend the time if you run the same search,
locally using ldbsearch -H /path/so/sam.ldb on a Samba build with debug
symbols and use Brendan Greg's FlameGraph utility:
https://www.brendangregg.com/FlameGraphs/cpuflamegraphs.html#Instructions
You can confirm that an index is in use by setting the environment
variable LDB_WARN_UNINDEXED=1, this will warn on unindexed searches,
but won't warn on poorly indexed searches. 


> ---
> When doing a very simple LDAP lookup using ldapsearch we get around
> 47ms of execution time (incl. bind and unbind):
> # time ldapsearch -H ldaps://10.12.100.1:636 -D "CN=Auth-
> User,CN=Users,DC=subdomain,DC=example,DC=de" -w xxxx -b
> "OU=myou,DC=subdomain,DC=example,DC=de" "(cn=user.name)"real   
> 0m0.047suser    0m0.026ssys    0m0.009s
> When trying to get the gidNumber of all groups the user is member of
> this request takes around 378ms (- 45ms roughly bind/unbind
> overhead):
> # time ldapsearch -H ldaps://10.12.100.1:636 -D "CN=Auth-
> User,CN=Users,DC=subdomain,DC=example,DC=de" -w xxxx -b
> "OU=courses,OU=myou,DC=subdomain,DC=example,DC=de"
> "(&(objectclass=group)(member=CN=user.name,OU=Employees,OU=Users,OU=m
> you,DC=subdomain,DC=example,DC=de))" gidNumber

Do try with the terms in the other order.  We match left-to-right, so
the term that matches the least objects should be first.  I don't think
this is the problem however. 
Try a search with just
'(member=CN=user.name,OU=Employees,OU=Users,OU=myou,DC=subdomain,DC=exa
mple,DC=de)' and LDB_WARN_UNINDEXED=1 to confirm that this term is
indexed for you.
> # numResponses: 68# numEntries: 67
> real    0m0.378suser    0m0.029ssys    0m0.012s
> When trying to get the gidNumber of all groups (courses) it only
> takes around 249ms (-45ms bind/unbind overhead). So querying the
> gidNumber of 1280 groups is faster then querying the gidNumber of
> groups where the user is a member:
> # time ldapsearch -H ldaps://10.12.100.1:636 -D "CN=Auth-
> User,CN=Users,DC=subdomain,DC=example,DC=de" -w xxxx -b
> "OU=courses,OU=myou,DC=subdomain,DC=example,DC=de"
> "(&(objectclass=group))" gidNumber
> # numResponses: 1281# numEntries: 1280
> real    0m0.249suser    0m0.051ssys    0m0.047s
> ---
> 
> 
-- 
Andrew Bartlett (he/him)       https://samba.org/~abartlet/Samba Team Member (since 2001) https://samba.orgSamba Team Lead                https://catalyst.net.nz/services/sambaCatalyst.Net Ltd
Proudly developing Samba for Catalyst.Net Ltd - a Catalyst IT group
company
Samba Development and Support: https://catalyst.net.nz/services/samba
Catalyst IT - Expert Open Source Solutions