[Samba] High cpu load on LDAP
Heinz Hölzl
heinz.hoelzl at gvcc.net
Mon Nov 18 13:29:46 UTC 2024
hi,
I have to activate the thread again ...
we keep having preformance problems on the DC, especially on Monday
morning when the PCs are switched on and the users log in.
some ldap-searches take a very long time, sometimes even over 15
seconds
e.g:
ldapsrv_SearchRequest: LDAP Query: Duration was 15.74s, SearchRequest
by S-1-5-21-xxxxxxxxxx-xxxxxxxxxxxxxx-8585 from
ipv4:192.168.35.117:49240 filter: [(objectClass=user)] basedn:
[DC=example,DC=net] scope: [SUB] result: Success
The load of the ldap processes reaches 100% of a CPU. The ldapserver is
then no longer responsive.
It seems that the ldapsearches are blocking each other.
The result is very long response times for login and other
authentications.
we have 6 DC, approx. 10,000 active users and approx. 5000 PC, 10 samba
fileservers, all with current samba version
is there a way to increase the performance here?
regards,
Heinz
Here are more logs (at a very quiet time) :
ldapsrv_SearchRequest: LDAP Query: Duration was 1.74s, SearchRequest
by S-1-5-21-xxxxxxxxxxxxx-xxxxxxxxxxxxxx-xxxxxxxxxxxxxxx-3084 from
ipv4:192.168.48.87:33768 filter:
[(|(objectClass=user)(objectClass=group))] basedn: [dc=GVCC,dc=NET]
scope: [SUB] result: Success
ldapsrv_SearchRequest: LDAP Query: Duration was 1.79s, SearchRequest
by S-1-5-21-xxxxxxxxxxxxx-xxxxxxxxxxxxxx-xxxxxxxxxxxxxxx-3084 from
ipv4:192.168.48.87:33768 filter:
[(|(objectClass=user)(objectClass=group))] basedn: [dc=GVCC,dc=NET]
scope: [SUB] result: Success
ldapsrv_SearchRequest: LDAP Query: Duration was 0.63s, SearchRequest
by S-1-5-18 from unix: filter:
[(&(objectClass=user)(userAccountControl:1.2.840.113556.1.4.803:=512)(!
(sAMAccountName=krbtgt*)))] basedn: [DC=example,DC=net] scope: [SUB]
result: Success
ldapsrv_SearchRequest: LDAP Query: Duration was 1.80s, SearchRequest
by S-1-5-21-xxxxxxxxxxxxx-xxxxxxxxxxxxxx-xxxxxxxxxxxxxxx-3048 from
ipv4:192.168.19.22:54708 filter:
[(|(objectClass=user)(objectClass=group))] basedn: [dc=GVCC,dc=NET]
scope: [SUB] result: Success
ldapsrv_SearchRequest: LDAP Query: Duration was 1.74s, SearchRequest
by S-1-5-21-xxxxxxxxxxxxx-xxxxxxxxxxxxxx-xxxxxxxxxxxxxxx-3048 from
ipv4:192.168.19.22:54708 filter:
[(|(objectClass=user)(objectClass=group))] basedn: [dc=GVCC,dc=NET]
scope: [SUB] result: Success
ldapsrv_SearchRequest: LDAP Query: Duration was 0.67s, SearchRequest
by S-1-5-18 from unix: filter:
[(&(objectClass=user)(userAccountControl:1.2.840.113556.1.4.803:=512)(!
(sAMAccountName=krbtgt*)))] basedn: [DC=example,DC=net] scope: [SUB]
result: Success
ldapsrv_SearchRequest: LDAP Query: Duration was 0.64s, SearchRequest
by S-1-5-18 from unix: filter:
[(&(objectClass=user)(userAccountControl:1.2.840.113556.1.4.803:=512)(!
(sAMAccountName=krbtgt*)))] basedn: [DC=example,DC=net] scope: [SUB]
result: Success
ldapsrv_SearchRequest: LDAP Query: Duration was 1.93s, SearchRequest
by S-1-5-21-xxxxxxxxxxxxx-xxxxxxxxxxxxxx-xxxxxxxxxxxxxxx-3085 from
ipv4:192.168.44.65:59148 filter:
[(|(objectClass=user)(objectClass=group))] basedn: [dc=GVCC,dc=NET]
scope: [SUB] result: Success
ldapsrv_SearchRequest: LDAP Query: Duration was 1.88s, SearchRequest
by S-1-5-21-xxxxxxxxxxxxx-xxxxxxxxxxxxxx-xxxxxxxxxxxxxxx-3085 from
ipv4:192.168.44.65:59148 filter:
[(|(objectClass=user)(objectClass=group))] basedn: [dc=GVCC,dc=NET]
scope: [SUB] result: Success
ldapsrv_SearchRequest: LDAP Query: Duration was 0.61s, SearchRequest
by S-1-5-18 from unix: filter:
[(&(objectClass=user)(userAccountControl:1.2.840.113556.1.4.803:=512)(!
(sAMAccountName=krbtgt*)))] basedn: [DC=example,DC=net] scope: [SUB]
result: Success
ldapsrv_SearchRequest: LDAP Query: Duration was 1.83s, SearchRequest
by S-1-5-21-xxxxxxxxxxxxx-xxxxxxxxxxxxxx-xxxxxxxxxxxxxxx-3085 from
ipv4:192.168.44.65:59148 filter:
[(|(objectClass=user)(objectClass=group))] basedn: [dc=GVCC,dc=NET]
scope: [SUB] result: Success
ldapsrv_SearchRequest: LDAP Query: Duration was 0.62s, SearchRequest
by S-1-5-18 from unix: filter:
[(&(objectClass=user)(userAccountControl:1.2.840.113556.1.4.803:=512)(!
(sAMAccountName=krbtgt*)))] basedn: [DC=example,DC=net] scope: [SUB]
result: Success
ldapsrv_SearchRequest: LDAP Query: Duration was 0.69s, SearchRequest
by S-1-5-18 from unix: filter:
[(&(objectClass=user)(userAccountControl:1.2.840.113556.1.4.803:=512)(!
(sAMAccountName=krbtgt*)))] basedn: [DC=example,DC=net] scope: [SUB]
result: Success
Am Freitag, dem 30.09.2022 um 09:14 +1300 schrieb Andrew Bartlett:
> [Sie erhalten nicht häufig E-Mails von abartlet at samba.org. Weitere
> Informationen, warum dies wichtig ist, finden Sie unter
> https://aka.ms/LearnAboutSenderIdentification ]
>
> On Mon, 2022-09-19 at 14:04 +0000, Heinz Hölzl via samba wrote:
> > hello,
> > I often have the problem of high load on the LDAP processes.
> > 1-3 LDAP processes cause 100% cpu load for approx. 10 sec. This
> > happens
> > regularly in intervals of 2-3 minutes.
> > How can I find out which client is causing this load and why?
> > How can I configure the logging to see who/what is causing the LDAP
> > process?
> > We have about 5000 users, 4000 clients in our AD with 4 DCs with
> > Samba
> > 4.16.4.
> >
> > thanks
>
>
> I would also note the similarity with this other post on the mailing
> list.
>
> If this is the same issue, the sockets to the client would be stuck
> in
> CLOSE_WAIT.
>
> https://lists.samba.org/archive/samba/2022-September/241873.html
>
> Can you, on the instance with the high-cpu use, at the time it is in
> that state, run (matching the commands our mailing list user has
> run):
>
> Work out which PIDs are high CPU use:
>
> netstat -avp | grep <PID>
>
> gdb -p <PID>
> bt full
>
> strace -p <PID>
>
> as well as a FlameGraph:
> https://www.brendangregg.com/FlameGraphs/cpuflamegraphs.html#Instructions
>
> I would love to get to the bottom of this.
> Andrew Bartlett
>
>
> --
> Andrew Bartlett (he/him) https://samba.org/~abartlet/
> Samba Team Member (since 2001) https://samba.org
> Samba Team Lead, Catalyst IT https://catalyst.net.nz/services/samba
>
> Samba Development and Support, Catalyst IT - Expert Open Source
> Solutions
>
More information about the samba
mailing list