[Samba] authentication performance with 4.7.6 -> 4.7.8 upgrade (was: Re: gencache.tdb size and cache flush)

[Andrew Bartlett]

On Wed, 2018-08-29 at 15:36 +0200, Peter Eriksson via samba wrote:
> For what it’s worth you are not alone in seeing similar problems with Samba and gencache. 
> 
> Our site has some 110K users (university with staff & students (including former ones), and currently around 2000 active (SMB) clients connecting to 5 different Samba servers (around 400-500 clients per server). When we previously just let things “run” gencache.tdb would grow forever and authentication login performance would start to deteriorate after a little while (would take more than 10 seconds). So we now delete it (and locks/locking.tdb that also tends to grow forever) and restart our samba processes every morning at 7 am - which gives us much more stable performance.
> 
> - Servers with 256GB of RAM, 10Gbps ethernet interfaces and around 110TB of disk per server.
> - FreeBSD 11.2-p2
> - Samba 4.7.6 with some local patches to allow (much) bigger socket listening queues in order to handle the case of many clients connecting at the same time.
> 
> (We are trying to upgrade to a more recent Samba but 4.7.8 and 4.7.9 gave us horrible authentication performance every 10:th hour where the servers basically denied clients to login for about 2 hours so we had to back down to 4.7.6 again).

I realise testing in production is difficult, but is there any chance
you can pin down where between 4.7.6 and 4.7.8 it broke?  There are not
that many changes between, and while some appear authentication related
nothing stands out. 

Also, do you run Samba as an AD DC, or are these file servers in a
windows domain?

Thanks,

Andrew Bartlett

-- 
Andrew Bartlett
https://samba.org/~abartlet/
Authentication Developer, Samba Team         https://samba.org
Samba Development and Support, Catalyst IT   
https://catalyst.net.nz/services/samba