[Samba] Unable to use more than 1000 concurrent ntlm_auth processes

[Andrew Bartlett]

On Sat, 2012-08-18 at 23:03 +0930, Michael Hendrie wrote:
> Hi List,
> 
> I'm running a heavily loaded squid server that uses ntlm_auth to provide NTLM authentication.
> 
> As load has increased over time, I've found the need to increase the
> number of ntlm_auth processes available to squid as well as the
> "winbind max clients" value in the smb.conf file.  This has worked
> well up until now but seems I've hit some sort of limit.
> 
> If I keep the number of ntlm_auth processes under 1000, all is good.  Going above continually produces the messages below in /var/log/messages and the additional helpers unusable:
> 
> Aug 16 22:34:17 prox (ntlm_auth): [2012/08/16 22:34:17.342283,  0] utils/ntlm_auth.c:186(get_winbind_domain) 
> Aug 16 22:34:17 prox (ntlm_auth):   could not obtain winbind domain name! 
> Aug 16 22:34:17 prox (ntlm_auth): [2012/08/16 22:34:17.345335,  0] utils/ntlm_auth.c:186(get_winbind_domain) 
> Aug 16 22:34:17 prox (ntlm_auth):   could not obtain winbind domain name! 
> Aug 16 22:34:17 prox (ntlm_auth): [2012/08/16 22:34:17.353230,  0] utils/ntlm_auth.c:186(get_winbind_domain) 
> Aug 16 22:34:17 prox (ntlm_auth):   could not obtain winbind domain name! 
> Aug 16 22:34:17 prox (ntlm_auth): [2012/08/16 22:34:17.358237,  0] utils/ntlm_auth.c:186(get_winbind_domain) 
> Aug 16 22:34:17 prox (ntlm_auth):   could not obtain winbind domain name!
> 
> And with winbindd log level on 9, /var/log/samba/winbindd.log shows:
> 
> [2012/08/16 22:33:42.352991,  6] winbindd/winbindd.c:768(new_connection)
>   accepted socket 1032
> [2012/08/16 22:33:42.359183,  6] winbindd/winbindd.c:768(new_connection)
>   accepted socket 1036
> [2012/08/16 22:37:59.337941,  2] winbindd/winbindd.c:710(winbind_client_response_written)
>   Could not write response[14772:INTERFACE_VERSION] to client: Broken pipe

> Running distro supplied samba versions:
> 
> samba3x.x86_64:  3.5.10-0.110.el5_8
> samba3x-common.x86_64:  3.5.10-0.110.el5_8
> samba3x-winbind.x86_64:  3.5.10-0.110.el5_8
> 
> Does anyone have any suggestions on how to overcome this issue, I am happy to compile from source if there are any options that could help?

In relation to a similar query, it was suggested that with master (or a
Samba 4.0 beta) you could set:

winbind max domain connections = <larger number than 1>

This might increase the throughput, and avoid the backlog getting to
1000.

I still think that you are hitting an OS limit somewhere (perhaps on the
total of the ntlm_auth children, rather than winbindd?), but having 1000
outstanding authentications would be painful in any case.

Andrew Bartlett

-- 
Andrew Bartlett                                http://samba.org/~abartlet/
Authentication Developer, Samba Team           http://samba.org