[Samba] Slow ldap membership query in large active directory

[Sören Busse]

Hey there,

we've been using Samba AD DC successfully for about 4 years in our 
school with about 1000 people. 4 years ago we decided to create a group 
for each class + subject combination, so we have about 1400 groups with 
~30 members each (some are much bigger up to 800 people and others have 
only a few members). One of our systems, which uses LDAP, needs to 
retrieve the gidNumber of all the groups a user is a member of. This 
request is sent about 3 to 4 times per second (yes, this is a design 
flaw, but we cannot easily change it or enable caching):

We noticed that the query to get all the gidNumbers of the courses the 
user is a member of takes about 370ms, while a simple query takes 47ms 
(including bind/unbind). See the test results below.

Why is a query on the member attribute so expensive? I would have 
assumed that this very common query would be optimised like an index 
user => [groups], so that you only need to get the gidNumber attribute 
of the remaining groups. Or maybe there's a faster way to do the query / 
optimise the ldap database?

Thank you very much in advance!

---

When doing a very simple LDAP lookup using ldapsearch we get around 47ms 
of execution time (incl. bind and unbind):

# time ldapsearch -H ldaps://10.12.100.1:636 -D 
"CN=Auth-User,CN=Users,DC=subdomain,DC=example,DC=de" -w xxxx -b 
"OU=myou,DC=subdomain,DC=example,DC=de" "(cn=user.name)"
real    0m0.047s
user    0m0.026s
sys    0m0.009s

When trying to get the gidNumber of all groups the user is member of 
this request takes around 378ms (- 45ms roughly bind/unbind overhead):

# time ldapsearch -H ldaps://10.12.100.1:636 -D 
"CN=Auth-User,CN=Users,DC=subdomain,DC=example,DC=de" -w xxxx -b 
"OU=courses,OU=myou,DC=subdomain,DC=example,DC=de" 
"(&(objectclass=group)(member=CN=user.name,OU=Employees,OU=Users,OU=myou,DC=subdomain,DC=example,DC=de))" 
gidNumber

# numResponses: 68
# numEntries: 67

real    0m0.378s
user    0m0.029s
sys    0m0.012s

When trying to get the gidNumber of all groups (courses) it only takes 
around 249ms (-45ms bind/unbind overhead). So querying the gidNumber of 
1280 groups is faster then querying the gidNumber of groups where the 
user is a member:

# time ldapsearch -H ldaps://10.12.100.1:636 -D 
"CN=Auth-User,CN=Users,DC=subdomain,DC=example,DC=de" -w xxxx -b 
"OU=courses,OU=myou,DC=subdomain,DC=example,DC=de" 
"(&(objectclass=group))" gidNumber

# numResponses: 1281
# numEntries: 1280

real    0m0.249s
user    0m0.051s
sys    0m0.047s

---