[Samba] [Samba 4.5] Very slow LDAP Queries (almost unusable), performance tunning ?
L.P.H. van Belle
belle at bazuin.nl
Mon Mar 27 10:05:52 UTC 2017
I reviewed you indexing list again against mine.
A diff between your indexing list and mine shows just a small difference.
> @IDXATTR: otherMailbox
109d109
< @IDXATTR: uidNumber
116a117
> @IDXATTR: zarafaAccount
Left (<) me, right (>) yours.
You did you install the zarafa schema in the AD?
I did use : zarafa_schema_add.sh
Beside that i dont see much.
Greetz,
Louis
Van: Gaetan SLONGO [mailto:gslongo at it-optics.com]
Verzonden: maandag 27 maart 2017 11:12
Aan: Andrew Bartlett
CC: samba at lists.samba.org; L.P.H. van Belle
Onderwerp: Re: [Samba] [Samba 4.5] Very slow LDAP Queries (almost unusable), performance tunning ?
Thank you for your message Andrew. Then .. Waiting for this improvement... :-)
Indexes seem correct to me. But I'm not sure at 100% because I cannot find clear explanation regarding the searchFlags attribute value .. 1, 2, ... ? Maybe you have one ?
Thanks !
De: "Andrew Bartlett" <abartlet at samba.org>
À: "Gaetan SLONGO" <gslongo at it-optics.com>, "L.P.H. van Belle" <belle at bazuin.nl>
Cc: samba at lists.samba.org
Envoyé: Lundi 27 Mars 2017 11:06:56
Objet: Re: [Samba] [Samba 4.5] Very slow LDAP Queries (almost unusable), performance tunning ?
On Mon, 2017-03-27 at 10:43 +0200, Gaetan SLONGO via samba wrote:
> Zarafa is not on the same server as Samba
>
> We only have 2 AD/DC Samba 4.5 (CentOS 7) and we put required indexes
> on LDAP .
>
> Arround 1000 mailboxes but not all are simultaneously in use (approx
> 1/3 in use).
> MTA is postfix (and is still connected to Samba AD, this one is not
> causing the issue).
> As a workarround, we currently deployed a synchronization connector
> from AD to OpenLDAP. It solves the performance issue during the
> investigation because Zarafa was totally unusable at all when
> connected to Samba... But We plan to move to Zimbra by the end of the
> year so I think the work arround can stay in place until the
> migration. However this performance issue could be a bottleneck in
> other applications, in the future...
>
> I did not found any config setting allowing tu enable multi-threading
> on Samba LDAP backend (maybe an hidden one ?).. I think it could help
> a lot
Given these discussions, I'm keen to add it. I was going to add this
for Samba 4.6.0, but the initial approach I used was slower in some
cases (the connect/bind/disconnect case). Sadly at the time there
wasn't this level of concern regarding the LDAP performance, so we
focussed on what we could achieve, which was making NETLOGON multi-
process.
This remains on my radar, along with any other approaches we find along
the way to make search-heavy operation practical.
I'm sorry this is causing so much trouble, and I look forward to
helping improve this area.
In the meantime, adding the indexes that your client tools need will
help a lot.
Andrew Bartlett
> ----- Mail original -----
>
> De: "L.P.H. van Belle via samba" <samba at lists.samba.org>
> À: samba at lists.samba.org
> Envoyé: Lundi 27 Mars 2017 10:26:22
> Objet: Re: [Samba] [Samba 4.5] Very slow LDAP Queries (almost
> unusable), performance tunning ?
>
> Can you tell more about your setup?
>
> Is zarafa and samba on the same server for example.
>
> Which MTA are you using postfix/exim?
>
>
>
> My top was about 150 users, and all my printers are connected also so
> about 200 devices do ldap searches.
>
> but my setup is split over 10+ servers ( 2 are AD DC )
>
>
>
> So best is to tell what you can about your setup, anonimize if
> needed.
>
>
>
> Greetz,
>
>
>
> Louis
>
>
>
>
>
>
>
>
> Van: Gaetan SLONGO [mailto:gslongo at it-optics.com]
> Verzonden: maandag 27 maart 2017 10:12
> Aan: L.P.H. van Belle
> CC: samba at lists.samba.org
> Onderwerp: Re: [Samba] [Samba 4.5] Very slow LDAP Queries (almost
> unusable), performance tunning ?
>
>
>
>
> What we found is Zarafa makes a very big amount of queries, which
> makes Samba run at 100% CPU (one process, LDAP does not seems to be
> multi-threaded..?)... but we have hundreds of users...
>
> What do you think could be wrong in the current database/setup ? We
> verified all the setup and everything seems OK
>
>
> De: "L.P.H. van Belle via samba" <samba at lists.samba.org>
> À: samba at lists.samba.org
> Envoyé: Lundi 27 Mars 2017 09:58:55
> Objet: Re: [Samba] [Samba 4.5] Very slow LDAP Queries (almost
> unusable), performance tunning ?
>
> No, you have to do that manualy, or look the the samba4 ADS script
> for kopano ( or zarafa )
>
> But I mostly follow the documentation.
>
>
>
> And when i run :
>
> time ldbsearch -H /var/lib/samba/private/sam.ldb -s base -b
> @INDEXLIST
>
> ....
>
> real 0m0.230s
>
> user 0m0.184s
>
> sys 0m0.044s
>
>
>
> so if yours take more that 20 sec there is something very wrong.
>
> I suggest check you samba AD database and samba4 ADDC setup,
>
> i dont think this is zarafa related.
>
>
>
>
>
> Greetz,
>
>
>
> Louis
>
>
>
>
>
>
>
>
>
>
>
>
> Van: Gaetan SLONGO [mailto:gslongo at it-optics.com]
> Verzonden: maandag 27 maart 2017 8:46
> Aan: L.P.H. van Belle
> CC: samba at lists.samba.org
> Onderwerp: Re: [Samba] [Samba 4.5] Very slow LDAP Queries (almost
> unusable), performance tunning ?
>
>
>
>
> Hi !
>
> Thanks for answer. Yes we use zarafaAccount in search filter.
> There is an installer provided for Samba4 to install new schemas ?
>
> Thanks !
>
>
> De: "L.P.H. van Belle via samba" <samba at lists.samba.org>
> À: samba at lists.samba.org
> Envoyé: Jeudi 23 Mars 2017 11:54:50
> Objet: Re: [Samba] [Samba 4.5] Very slow LDAP Queries (almost
> unusable), performance tunning ?
>
>
> Are use using zarafaAccount=1 withing the search filters?
> I use this things like this :
>
> (&(objectClass=person)(zarafaAccount=1)(|(mail=%s)(otherMailbox=%s)))
>
> Or for groups.
> (&(objectclass=group)(zarafaAccount=1)(|(mail=%s)(otherMailbox=%s)))
>
> That helps a lot.
>
> ! If you switch to kopano beware to change the SCHEMA and filters
> zarafaAccount changed to kopanoAccount
>
>
> Greetz.
>
> Louis
>
>
> > -----Oorspronkelijk bericht-----
> > Van: samba [mailto:samba-bounces at lists.samba.org] Namens Gaetan
> > SLONGO via
> > samba
> > Verzonden: donderdag 23 maart 2017 11:12
> > Aan: samba at lists.samba.org
> > Onderwerp: [Samba] [Samba 4.5] Very slow LDAP Queries (almost
> > unusable),
> > performance tunning ?
> > Urgentie: Hoog
> >
> >
> > Dear users,
> >
> > We are facing to a big latency issue regarding the LDAP Server
> > (both
> > encrypted & plain).
> >
> > We have a Zarafa mail server which makes a lot of queries and puts
> > a samba
> > process to 100% usage. This latency makes the mail server
> > unusable.. The
> > mail server was previously on OpenLDAP and there was not
> > performance
> > issues.
> >
> > A simple LDAP query can take up to 25 sec to perform !!
> >
> > We have added some indexes :
> >
> > [root at califix ~]# ldbsearch -H /var/lib/samba/private/sam.ldb -s
> > base -b
> > @INDEXLIST
> > # record 1
> > dn: @INDEXLIST
> > @IDXONE: 1
> > @IDXVERSION: 2
> > @IDXATTR: objectClass
> > @IDXATTR: msDS-Cached-Membership-Time-Stamp
> > @IDXATTR: userPrincipalName
> > @IDXATTR: rpcNsInterfaceID
> > @IDXATTR: fileExtPriority
> > @IDXATTR: dnsRoot
> > @IDXATTR: mSMQLabelEx
> > @IDXATTR: dNSTombstoned
> > @IDXATTR: msDS-PhoneticCompanyName
> > @IDXATTR: msSFU30Domains
> > @IDXATTR: dhcpType
> > @IDXATTR: ou
> > @IDXATTR: gidNumber
> > @IDXATTR: msFVE-VolumeGuid
> > @IDXATTR: msTSManagingLS2
> > @IDXATTR: implementedCategories
> > @IDXATTR: oMTIndxGuid
> > @IDXATTR: cOMClassID
> > @IDXATTR: volTableIdxGUID
> > @IDXATTR: l
> > @IDXATTR: mSMQDigests
> > @IDXATTR: msTSExpireDate4
> > @IDXATTR: flatName
> > @IDXATTR: msSFU30YpServers
> > @IDXATTR: packageFlags
> > @IDXATTR: mSMQOwnerID
> > @IDXATTR: objectCategory
> > @IDXATTR: msSFU30IsValidContainer
> > @IDXATTR: msTSProperty02
> > @IDXATTR: mS-DS-CreatorSID
> > @IDXATTR: proxyAddresses
> > @IDXATTR: msPKI-Cert-Template-OID
> > @IDXATTR: uNCName
> > @IDXATTR: mS-SQL-Name
> > @IDXATTR: fSMORoleOwner
> > @IDXATTR: msSFU30NisDomain
> > @IDXATTR: otherMailbox
> > @IDXATTR: location
> > @IDXATTR: msSFU30NetgroupHostAtDomain
> > @IDXATTR: uSNChanged
> > @IDXATTR: sIDHistory
> > @IDXATTR: birthLocation
> > @IDXATTR: msDS-SecondaryKrbTgtNumber
> > @IDXATTR: msTSProperty01
> > @IDXATTR: msTSManagingLS4
> > @IDXATTR: msSFU30OrderNumber
> > @IDXATTR: msDS-HABSeniorityIndex
> > @IDXATTR: primaryGroupID
> > @IDXATTR: mSMQQueueType
> > @IDXATTR: msDFSR-ReplicationGroupGuid
> > @IDXATTR: msDS-PhoneticDepartment
> > @IDXATTR: mail
> > @IDXATTR: msSFU30Name
> > @IDXATTR: msSFU30NetgroupUserAtDomain
> > @IDXATTR: fromServer
> > @IDXATTR: displayName
> > @IDXATTR: msTSLicenseVersion2
> > @IDXATTR: groupType
> > @IDXATTR: msTSLicenseVersion3
> > @IDXATTR: msTSLicenseVersion4
> > @IDXATTR: userAccountControl
> > @IDXATTR: physicalLocationObject
> > @IDXATTR: servicePrincipalName
> > @IDXATTR: msTSExpireDate
> > @IDXATTR: serviceClassName
> > @IDXATTR: lDAPDisplayName
> > @IDXATTR: zarafaAccount
> > @IDXATTR: terminalServer
> > @IDXATTR: givenName
> > @IDXATTR: msTSManagingLS3
> > @IDXATTR: msSFU30MaxUidNumber
> > @IDXATTR: msDS-Entry-Time-To-Die
> > @IDXATTR: msTSLSProperty01
> > @IDXATTR: msDS-PhoneticFirstName
> > @IDXATTR: trustPartner
> > @IDXATTR: msTSLSProperty02
> > @IDXATTR: msTSExpireDate3
> > @IDXATTR: objectGUID
> > @IDXATTR: showInAdvancedViewOnly
> > @IDXATTR: rpcNsTransferSyntax
> > @IDXATTR: sAMAccountName
> > @IDXATTR: mS-SQL-Version
> > @IDXATTR: msDS-Site-Affinity
> > @IDXATTR: sn
> > @IDXATTR: name
> > @IDXATTR: nETBIOSName
> > @IDXATTR: sAMAccountType
> > @IDXATTR: msTSManagingLS
> > @IDXATTR: msDFSR-DfsPath
> > @IDXATTR: altSecurityIdentities
> > @IDXATTR: USNIntersite
> > @IDXATTR: msSFU30MasterServerName
> > @IDXATTR: msDS-PhoneticLastName
> > @IDXATTR: cn
> > @IDXATTR: netbootGUID
> > @IDXATTR: lastLogonTimestamp
> > @IDXATTR: legacyExchangeDN
> > @IDXATTR: mSMQLabel
> > @IDXATTR: uSNCreated
> > @IDXATTR: mS-SQL-Database
> > @IDXATTR: msDS-PhoneticDisplayName
> > @IDXATTR: msSFU30MaxGidNumber
> > @IDXATTR: rpcNsObjectID
> > @IDXATTR: timeVolChange
> > @IDXATTR: msTSExpireDate2
> > @IDXATTR: groupAttributes
> > @IDXATTR: physicalDeliveryOfficeName
> > @IDXATTR: msFVE-RecoveryGuid
> > @IDXATTR: msDS-AdditionalSamAccountName
> > @IDXATTR: objectSid
> > @IDXATTR: keywords
> > @IDXATTR: mS-SQL-Alias
> > @IDXATTR: invocationId
> > @IDXATTR: msTSLicenseVersion
> > @IDXATTR: requiredCategories
> > @IDXATTR: msDS-AzObjectGuid
> > distinguishedName: @INDEXLIST
> >
> > There is any way to improve LDAP responses times ? It seems there
> > is only
> > one process which is managing LDAP queries (no forks/threads?)
> >
> > Thank you in advance for your help !!
> >
> > --
> > To unsubscribe from this list go to the following URL and read the
> > instructions: https://lists.samba.org/mailman/options/samba
>
>
>
> --
> To unsubscribe from this list go to the following URL and read the
> instructions: https://lists.samba.org/mailman/options/samba
>
>
>
>
>
> --
>
>
>
> www.it-optics.com
>
> Gaëtan SLONGO | Head of Infrastructure Department
> Boulevard Initialis, 28 - 7000 Mons, BELGIUM
>
> Company :
>
> +32 (0)65 84 23 85
>
> Direct :
>
> +32 (0)65 32 85 88
>
> Fax :
>
> +32 (0)65 84 66 76
>
> Skype ID :
>
> gslongo.pro
>
> GPG Key :
>
> gslongo-gpg_key.asc
>
>
>
>
>
>
>
> - Please consider your environmental responsibility before printing
> this e-mail -
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> --
> To unsubscribe from this list go to the following URL and read the
> instructions: https://lists.samba.org/mailman/options/samba
>
>
>
>
>
> --
>
>
>
> www.it-optics.com
>
> Gaëtan SLONGO | Head of Infrastructure Department
> Boulevard Initialis, 28 - 7000 Mons, BELGIUM
>
> Company :
>
> +32 (0)65 84 23 85
>
> Direct :
>
> +32 (0)65 32 85 88
>
> Fax :
>
> +32 (0)65 84 66 76
>
> Skype ID :
>
> gslongo.pro
>
> GPG Key :
>
> gslongo-gpg_key.asc
>
>
>
>
>
>
>
> - Please consider your environmental responsibility before printing
> this e-mail -
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> --
> To unsubscribe from this list go to the following URL and read the
> instructions: https://lists.samba.org/mailman/options/samba
>
>
>
> --
>
>
>
>
> www.it-optics.com
>
> Gaëtan SLONGO | Head of Infrastructure Department
> Boulevard Initialis, 28 - 7000 Mons, BELGIUM
> Company : +32 (0)65 84 23 85
> Direct : +32 (0)65 32 85 88
> Fax : +32 (0)65 84 66 76
> Skype ID : gslongo.pro
> GPG Key : gslongo-gpg_key.asc
>
>
> - Please consider your environmental responsibility before printing
> this e-mail -
>
>
>
>
>
>
>
>
--
Andrew Bartlett http://samba.org/~abartlet/
Authentication Developer, Samba Team http://samba.org
Samba Developer, Catalyst IT http://catalyst.net.nz/services/samba
--
www.it-optics.com
Gaëtan SLONGO | Head of Infrastructure Department
Boulevard Initialis, 28 - 7000 Mons, BELGIUM
Company :
+32 (0)65 84 23 85
Direct :
+32 (0)65 32 85 88
Fax :
+32 (0)65 84 66 76
Skype ID :
gslongo.pro
GPG Key :
gslongo-gpg_key.asc
- Please consider your environmental responsibility before printing this e-mail -
More information about the samba
mailing list