[Samba] howto optimize samba/kerberos for 20k requests per minute - help needed

Thu Jul 22 10:57:24 UTC 2021

Hi,

i indeed see a lot of incorrects - and mostly only for the kerberos traffic:

    172.16.2.2.88 > 172.16.2.4.46686: Flags [P.], cksum 0x628c 
(incorrect -> 0x235f), seq 1:1600, ack 263, win 235, options [nop,nop,TS 
val 194497255 ecr 1229415], length 1599
     172.16.2.2.88 > 172.16.2.4.46690: Flags [P.], cksum 0x628c 
(incorrect -> 0xff45), seq 1:1600, ack 263, win 235, options [nop,nop,TS 
val 194497275 ecr 1229435], length 1599
     172.16.2.2.88 > 172.16.2.4.46694: Flags [P.], cksum 0x628c 
(incorrect -> 0x9828), seq 1:1600, ack 263, win 235, options [nop,nop,TS 
val 194497296 ecr 1229456], length 1599
     172.16.2.2.88 > 172.16.2.4.46698: Flags [P.], cksum 0x628c 
(incorrect -> 0x56e2), seq 1:1600, ack 263, win 235, options [nop,nop,TS 
val 194497319 ecr 1229478], length 1599
     172.16.2.2.88 > 172.16.2.4.46702: Flags [P.], cksum 0x624a 
(incorrect -> 0x11a1), seq 1:1534, ack 266, win 235, options [nop,nop,TS 
val 194497533 ecr 1229694], length 1533

We are running proxmox instead of xen server.

Will try to get rid of all ofloading features and report back.

Thank you for that hint!.

Stefan

On 20.07.21 10:05, L.P.H. van Belle via samba wrote:
> Stefan,
>
> That its so slow, that can be becaused by UCS setup.
>
> Its setup with kopano and AD is like this.
> Samba AD -> ldap -> (kopano server) >  Ldap proxy > ldap DB > kopano
> And UCS is only "syncing" the data to the local kopano server.
> Kopano USC does NOT use the AD connector but that LDAP connector.
>
> Now, im thinking, your also running this on Xenserver? 8.1-8.2 for example?
>
> Can your run shortly : tcpdump -nn -vv -i NIC_ethX  |grep incorrect
> Do you see lots of incorrects here?
> Because after and update from XenServer, i had lots of delays due bad checksums on packages.
>
> If thats the case, try this.
> https://github.com/cloudnull/XenServer-Offloading-Off
>
>
> Greetz,
>
> Louis
>
>
>> -----Oorspronkelijk bericht-----
>> Van: samba [mailto:samba-bounces at lists.samba.org] Namens
>> Stefan Bauer via samba
>> Verzonden: dinsdag 20 juli 2021 9:35
>> Aan: samba at lists.samba.org
>> Onderwerp: Re: [Samba] howto optimize samba/kerberos for 20k
>> requests per minute - help needed
>>
>> I have a plain UCS4.4-8 (latest) with kopano inside correct.
>> Joined it
>> initially to a server 2012 windows DC.
>>
>> Later migrated that windows 2012 domain to a UCS domain.
>> Kopano is still
>> a member-server.
>>
>> /etc/kopano/ldap.conf contains:
>>
>> ldap_uri = ldap://kopano01.procorp.local:7389/
>>
>> (this is a the local ldap on kopano server)
>>
>> As the local ldap seems to do not get passwords synced UCS-master to
>> local LDAP. Due to this the system is doing huge amount of
>> kerberos-connections to the UCS Samba-DC to validate the user
>> credentials all the time (25k/minute)
>>
>> 09:28:55.718262 IP kopano01.procorp.local.42559 >
>> adm-ucs0.procorp.local.kerberos:  v5
>> 09:28:55.723467 IP adm-ucs0.procorp.local.kerberos >
>> kopano01.procorp.local.59107:
>> 09:28:55.726673 IP kopano01.procorp.local.33144 >
>> adm-ucs0.procorp.local.kerberos: Flags [S], seq 1850537116,
>> win 29200,
>> options [mss 1460,sackOK,TS val 279012852 ecr 0,nop,wscale
>> 7], length 0
>>
>> Due to this, the authentication time is very bad:
>>
>> root at kopano01:~# kopano-stats --system | grep ldap_avg_auth
>>       ldap_avg_auth        Average duration (µs) of
>> authentication made
>> to LDAP server            276250
>>
>> (connection to local ldap -> kerberos request to UCS-master
>> and return)
>>
>> I'm running:
>>
>> Webapp 5.1.0.0+167.1
>> Kopano core 8.7.20
>>
>>
>> On 20.07.21 08:45, L.P.H. van Belle via samba wrote:
>>> Exacly what i mean. ... Kopano, i was already thinking is kopano..
>>> But there is one big difference i think. With your setup and mine
>>>
>>> I think you run Kopano from within UCS 4.3
>>> ( and i tested also USC 5.0, no kopano there, the move to
>> licenced kopano. )
>>> I run Kopano on clean Debian 10 install.
>>>
>>> And what version kopano/web app is running because
>>> i dont see that here. IO looks normal.
>>>
>>> Greetz,
>>>
>>> Louis
>>>
>>>> -----Oorspronkelijk bericht-----
>>>> Van: samba [mailto:samba-bounces at lists.samba.org] Namens
>>>> Stefan Bauer via samba
>>>> Verzonden: dinsdag 20 juli 2021 7:57
>>>> Aan: samba at lists.samba.org
>>>> Onderwerp: Re: [Samba] howto optimize samba/kerberos for 20k
>>>> requests per minute - help needed
>>>>
>>>> Hi,
>>>>
>>>> it is kopano. They state on their website¹:
>>>>
>>>> -----------------
>>>>
>>>> Note
>>>>
>>>> Please note that due to performance problems in Samba 4,
>>>> Samba 4 is not
>>>> supported as a user source for setups larger than 50 users.
>>>>
>>>> -----------------
>>>>
>>>> And that is indeed what i notice. The high amount of
>>>> kerberos-requests
>>>> between samba-DC and kopano server is causing a very high
>>>> io-load on the
>>>> samba system - renders the system unstable. Also the time an
>>>> authentication request takes, is between 200-400ms.
>>>>
>>>> As Kopano is doing an authentication every time a user clicks
>>>> a single
>>>> mail and no caching is possible, it is hitting the samba
>>>> system too hard.
>>>>
>>>>
>>>> Stefan
>>>>
>>>>
>>>> ¹
>>>> https://documentation.kopano.io/kopanocore_administrator_manua
>>>> l/user_management.html
>>>>
>>>>
>>>> On 19.07.21 11:27, Rowland Penny via samba wrote:
>>>>> On Mon, 2021-07-19 at 11:13 +0200, Stefan Bauer via samba wrote:
>>>>>> Hi and thank you for your time.
>>>>>>
>>>>>> We got now the confirmation that samba 4 is not supported by our
>>>>>> software-vendor.
>>>>> If I might ask, who is your software vendor and what is the
>>>> software ?
>>>>> In most cases, when a supplier says they do not support
>>>> Samba 4, they
>>>>> do support AD.
>>>> -- 
>>>> To unsubscribe from this list go to the following URL and read the
>>>> instructions:  https://lists.samba.org/mailman/options/samba
>>>>
>>>>
>> -- 
>> To unsubscribe from this list go to the following URL and read the
>> instructions:  https://lists.samba.org/mailman/options/samba
>>
>>
>