[Samba] [Samba 4.5] Very slow LDAP Queries (almost unusable), performance tunning ?
Andrew Bartlett
abartlet at samba.org
Mon Mar 27 17:26:44 UTC 2017
On Mon, 2017-03-27 at 11:11 +0200, Gaetan SLONGO via samba wrote:
> Thank you for your message Andrew. Then .. Waiting for this
> improvement... :-)
>
> Indexes seem correct to me. But I'm not sure at 100% because I cannot
> find clear explanation regarding the searchFlags attribute value ..
> 1, 2, ... ? Maybe you have one ?
https://msdn.microsoft.com/en-us/library/ms679765(v=vs.85).aspx
> Thanks !
>
> ----- Mail original -----
>
> De: "Andrew Bartlett" <abartlet at samba.org>
> À: "Gaetan SLONGO" <gslongo at it-optics.com>, "L.P.H. van Belle" <belle
> @bazuin.nl>
> Cc: samba at lists.samba.org
> Envoyé: Lundi 27 Mars 2017 11:06:56
> Objet: Re: [Samba] [Samba 4.5] Very slow LDAP Queries (almost
> unusable), performance tunning ?
>
> On Mon, 2017-03-27 at 10:43 +0200, Gaetan SLONGO via samba wrote:
> > Zarafa is not on the same server as Samba
> >
> > We only have 2 AD/DC Samba 4.5 (CentOS 7) and we put required
> > indexes
> > on LDAP .
> >
> > Arround 1000 mailboxes but not all are simultaneously in use
> > (approx
> > 1/3 in use).
> > MTA is postfix (and is still connected to Samba AD, this one is
> > not
> > causing the issue).
> > As a workarround, we currently deployed a synchronization
> > connector
> > from AD to OpenLDAP. It solves the performance issue during the
> > investigation because Zarafa was totally unusable at all when
> > connected to Samba... But We plan to move to Zimbra by the end of
> > the
> > year so I think the work arround can stay in place until the
> > migration. However this performance issue could be a bottleneck in
> > other applications, in the future...
> >
> > I did not found any config setting allowing tu enable multi-
> > threading
> > on Samba LDAP backend (maybe an hidden one ?).. I think it could
> > help
> > a lot
>
> Given these discussions, I'm keen to add it. I was going to add this
> for Samba 4.6.0, but the initial approach I used was slower in some
> cases (the connect/bind/disconnect case). Sadly at the time there
> wasn't this level of concern regarding the LDAP performance, so we
> focussed on what we could achieve, which was making NETLOGON multi-
> process.
>
> This remains on my radar, along with any other approaches we find
> along
> the way to make search-heavy operation practical.
>
> I'm sorry this is causing so much trouble, and I look forward to
> helping improve this area.
>
> In the meantime, adding the indexes that your client tools need will
> help a lot.
>
> Andrew Bartlett
>
> > ----- Mail original -----
> >
> > De: "L.P.H. van Belle via samba" <samba at lists.samba.org>
> > À: samba at lists.samba.org
> > Envoyé: Lundi 27 Mars 2017 10:26:22
> > Objet: Re: [Samba] [Samba 4.5] Very slow LDAP Queries (almost
> > unusable), performance tunning ?
> >
> > Can you tell more about your setup?
> >
> > Is zarafa and samba on the same server for example.
> >
> > Which MTA are you using postfix/exim?
> >
> >
> >
> > My top was about 150 users, and all my printers are connected also
> > so
> > about 200 devices do ldap searches.
> >
> > but my setup is split over 10+ servers ( 2 are AD DC )
> >
> >
> >
> > So best is to tell what you can about your setup, anonimize if
> > needed.
> >
> >
> >
> > Greetz,
> >
> >
> >
> > Louis
> >
> >
> >
> >
> >
> >
> >
> >
> > Van: Gaetan SLONGO [mailto:gslongo at it-optics.com]
> > Verzonden: maandag 27 maart 2017 10:12
> > Aan: L.P.H. van Belle
> > CC: samba at lists.samba.org
> > Onderwerp: Re: [Samba] [Samba 4.5] Very slow LDAP Queries (almost
> > unusable), performance tunning ?
> >
> >
> >
> >
> > What we found is Zarafa makes a very big amount of queries, which
> > makes Samba run at 100% CPU (one process, LDAP does not seems to
> > be
> > multi-threaded..?)... but we have hundreds of users...
> >
> > What do you think could be wrong in the current database/setup ?
> > We
> > verified all the setup and everything seems OK
> >
> >
> > De: "L.P.H. van Belle via samba" <samba at lists.samba.org>
> > À: samba at lists.samba.org
> > Envoyé: Lundi 27 Mars 2017 09:58:55
> > Objet: Re: [Samba] [Samba 4.5] Very slow LDAP Queries (almost
> > unusable), performance tunning ?
> >
> > No, you have to do that manualy, or look the the samba4 ADS script
> > for kopano ( or zarafa )
> >
> > But I mostly follow the documentation.
> >
> >
> >
> > And when i run :
> >
> > time ldbsearch -H /var/lib/samba/private/sam.ldb -s base -b
> > @INDEXLIST
> >
> > ....
> >
> > real 0m0.230s
> >
> > user 0m0.184s
> >
> > sys 0m0.044s
> >
> >
> >
> > so if yours take more that 20 sec there is something very wrong.
> >
> > I suggest check you samba AD database and samba4 ADDC setup,
> >
> > i dont think this is zarafa related.
> >
> >
> >
> >
> >
> > Greetz,
> >
> >
> >
> > Louis
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > Van: Gaetan SLONGO [mailto:gslongo at it-optics.com]
> > Verzonden: maandag 27 maart 2017 8:46
> > Aan: L.P.H. van Belle
> > CC: samba at lists.samba.org
> > Onderwerp: Re: [Samba] [Samba 4.5] Very slow LDAP Queries (almost
> > unusable), performance tunning ?
> >
> >
> >
> >
> > Hi !
> >
> > Thanks for answer. Yes we use zarafaAccount in search filter.
> > There is an installer provided for Samba4 to install new schemas ?
> >
> > Thanks !
> >
> >
> > De: "L.P.H. van Belle via samba" <samba at lists.samba.org>
> > À: samba at lists.samba.org
> > Envoyé: Jeudi 23 Mars 2017 11:54:50
> > Objet: Re: [Samba] [Samba 4.5] Very slow LDAP Queries (almost
> > unusable), performance tunning ?
> >
> >
> > Are use using zarafaAccount=1 withing the search filters?
> > I use this things like this :
> >
> > (&(objectClass=person)(zarafaAccount=1)(|(mail=%s)(otherMailbox=%s)
> > ))
> >
> > Or for groups.
> > (&(objectclass=group)(zarafaAccount=1)(|(mail=%s)(otherMailbox=%s))
> > )
> >
> > That helps a lot.
> >
> > ! If you switch to kopano beware to change the SCHEMA and filters
> > zarafaAccount changed to kopanoAccount
> >
> >
> > Greetz.
> >
> > Louis
> >
> >
> > > -----Oorspronkelijk bericht-----
> > > Van: samba [mailto:samba-bounces at lists.samba.org] Namens Gaetan
> > > SLONGO via
> > > samba
> > > Verzonden: donderdag 23 maart 2017 11:12
> > > Aan: samba at lists.samba.org
> > > Onderwerp: [Samba] [Samba 4.5] Very slow LDAP Queries (almost
> > > unusable),
> > > performance tunning ?
> > > Urgentie: Hoog
> > >
> > >
> > > Dear users,
> > >
> > > We are facing to a big latency issue regarding the LDAP Server
> > > (both
> > > encrypted & plain).
> > >
> > > We have a Zarafa mail server which makes a lot of queries and
> > > puts
> > > a samba
> > > process to 100% usage. This latency makes the mail server
> > > unusable.. The
> > > mail server was previously on OpenLDAP and there was not
> > > performance
> > > issues.
> > >
> > > A simple LDAP query can take up to 25 sec to perform !!
> > >
> > > We have added some indexes :
> > >
> > > [root at califix ~]# ldbsearch -H /var/lib/samba/private/sam.ldb -s
> > > base -b
> > > @INDEXLIST
> > > # record 1
> > > dn: @INDEXLIST
> > > @IDXONE: 1
> > > @IDXVERSION: 2
> > > @IDXATTR: objectClass
> > > @IDXATTR: msDS-Cached-Membership-Time-Stamp
> > > @IDXATTR: userPrincipalName
> > > @IDXATTR: rpcNsInterfaceID
> > > @IDXATTR: fileExtPriority
> > > @IDXATTR: dnsRoot
> > > @IDXATTR: mSMQLabelEx
> > > @IDXATTR: dNSTombstoned
> > > @IDXATTR: msDS-PhoneticCompanyName
> > > @IDXATTR: msSFU30Domains
> > > @IDXATTR: dhcpType
> > > @IDXATTR: ou
> > > @IDXATTR: gidNumber
> > > @IDXATTR: msFVE-VolumeGuid
> > > @IDXATTR: msTSManagingLS2
> > > @IDXATTR: implementedCategories
> > > @IDXATTR: oMTIndxGuid
> > > @IDXATTR: cOMClassID
> > > @IDXATTR: volTableIdxGUID
> > > @IDXATTR: l
> > > @IDXATTR: mSMQDigests
> > > @IDXATTR: msTSExpireDate4
> > > @IDXATTR: flatName
> > > @IDXATTR: msSFU30YpServers
> > > @IDXATTR: packageFlags
> > > @IDXATTR: mSMQOwnerID
> > > @IDXATTR: objectCategory
> > > @IDXATTR: msSFU30IsValidContainer
> > > @IDXATTR: msTSProperty02
> > > @IDXATTR: mS-DS-CreatorSID
> > > @IDXATTR: proxyAddresses
> > > @IDXATTR: msPKI-Cert-Template-OID
> > > @IDXATTR: uNCName
> > > @IDXATTR: mS-SQL-Name
> > > @IDXATTR: fSMORoleOwner
> > > @IDXATTR: msSFU30NisDomain
> > > @IDXATTR: otherMailbox
> > > @IDXATTR: location
> > > @IDXATTR: msSFU30NetgroupHostAtDomain
> > > @IDXATTR: uSNChanged
> > > @IDXATTR: sIDHistory
> > > @IDXATTR: birthLocation
> > > @IDXATTR: msDS-SecondaryKrbTgtNumber
> > > @IDXATTR: msTSProperty01
> > > @IDXATTR: msTSManagingLS4
> > > @IDXATTR: msSFU30OrderNumber
> > > @IDXATTR: msDS-HABSeniorityIndex
> > > @IDXATTR: primaryGroupID
> > > @IDXATTR: mSMQQueueType
> > > @IDXATTR: msDFSR-ReplicationGroupGuid
> > > @IDXATTR: msDS-PhoneticDepartment
> > > @IDXATTR: mail
> > > @IDXATTR: msSFU30Name
> > > @IDXATTR: msSFU30NetgroupUserAtDomain
> > > @IDXATTR: fromServer
> > > @IDXATTR: displayName
> > > @IDXATTR: msTSLicenseVersion2
> > > @IDXATTR: groupType
> > > @IDXATTR: msTSLicenseVersion3
> > > @IDXATTR: msTSLicenseVersion4
> > > @IDXATTR: userAccountControl
> > > @IDXATTR: physicalLocationObject
> > > @IDXATTR: servicePrincipalName
> > > @IDXATTR: msTSExpireDate
> > > @IDXATTR: serviceClassName
> > > @IDXATTR: lDAPDisplayName
> > > @IDXATTR: zarafaAccount
> > > @IDXATTR: terminalServer
> > > @IDXATTR: givenName
> > > @IDXATTR: msTSManagingLS3
> > > @IDXATTR: msSFU30MaxUidNumber
> > > @IDXATTR: msDS-Entry-Time-To-Die
> > > @IDXATTR: msTSLSProperty01
> > > @IDXATTR: msDS-PhoneticFirstName
> > > @IDXATTR: trustPartner
> > > @IDXATTR: msTSLSProperty02
> > > @IDXATTR: msTSExpireDate3
> > > @IDXATTR: objectGUID
> > > @IDXATTR: showInAdvancedViewOnly
> > > @IDXATTR: rpcNsTransferSyntax
> > > @IDXATTR: sAMAccountName
> > > @IDXATTR: mS-SQL-Version
> > > @IDXATTR: msDS-Site-Affinity
> > > @IDXATTR: sn
> > > @IDXATTR: name
> > > @IDXATTR: nETBIOSName
> > > @IDXATTR: sAMAccountType
> > > @IDXATTR: msTSManagingLS
> > > @IDXATTR: msDFSR-DfsPath
> > > @IDXATTR: altSecurityIdentities
> > > @IDXATTR: USNIntersite
> > > @IDXATTR: msSFU30MasterServerName
> > > @IDXATTR: msDS-PhoneticLastName
> > > @IDXATTR: cn
> > > @IDXATTR: netbootGUID
> > > @IDXATTR: lastLogonTimestamp
> > > @IDXATTR: legacyExchangeDN
> > > @IDXATTR: mSMQLabel
> > > @IDXATTR: uSNCreated
> > > @IDXATTR: mS-SQL-Database
> > > @IDXATTR: msDS-PhoneticDisplayName
> > > @IDXATTR: msSFU30MaxGidNumber
> > > @IDXATTR: rpcNsObjectID
> > > @IDXATTR: timeVolChange
> > > @IDXATTR: msTSExpireDate2
> > > @IDXATTR: groupAttributes
> > > @IDXATTR: physicalDeliveryOfficeName
> > > @IDXATTR: msFVE-RecoveryGuid
> > > @IDXATTR: msDS-AdditionalSamAccountName
> > > @IDXATTR: objectSid
> > > @IDXATTR: keywords
> > > @IDXATTR: mS-SQL-Alias
> > > @IDXATTR: invocationId
> > > @IDXATTR: msTSLicenseVersion
> > > @IDXATTR: requiredCategories
> > > @IDXATTR: msDS-AzObjectGuid
> > > distinguishedName: @INDEXLIST
> > >
> > > There is any way to improve LDAP responses times ? It seems
> > > there
> > > is only
> > > one process which is managing LDAP queries (no forks/threads?)
> > >
> > > Thank you in advance for your help !!
> > >
> > > --
> > > To unsubscribe from this list go to the following URL and read
> > > the
> > > instructions: https://lists.samba.org/mailman/options/samba
> >
> >
> >
> > --
> > To unsubscribe from this list go to the following URL and read the
> > instructions: https://lists.samba.org/mailman/options/samba
> >
> >
> >
> >
> >
> > --
> >
> >
> >
> > www.it-optics.com
> >
> > Gaëtan SLONGO | Head of Infrastructure Department
> > Boulevard Initialis, 28 - 7000 Mons, BELGIUM
> >
> > Company :
> >
> > +32 (0)65 84 23 85
> >
> > Direct :
> >
> > +32 (0)65 32 85 88
> >
> > Fax :
> >
> > +32 (0)65 84 66 76
> >
> > Skype ID :
> >
> > gslongo.pro
> >
> > GPG Key :
> >
> > gslongo-gpg_key.asc
> >
> >
> >
> >
> >
> >
> >
> > - Please consider your environmental responsibility before
> > printing
> > this e-mail -
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > --
> > To unsubscribe from this list go to the following URL and read the
> > instructions: https://lists.samba.org/mailman/options/samba
> >
> >
> >
> >
> >
> > --
> >
> >
> >
> > www.it-optics.com
> >
> > Gaëtan SLONGO | Head of Infrastructure Department
> > Boulevard Initialis, 28 - 7000 Mons, BELGIUM
> >
> > Company :
> >
> > +32 (0)65 84 23 85
> >
> > Direct :
> >
> > +32 (0)65 32 85 88
> >
> > Fax :
> >
> > +32 (0)65 84 66 76
> >
> > Skype ID :
> >
> > gslongo.pro
> >
> > GPG Key :
> >
> > gslongo-gpg_key.asc
> >
> >
> >
> >
> >
> >
> >
> > - Please consider your environmental responsibility before
> > printing
> > this e-mail -
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > --
> > To unsubscribe from this list go to the following URL and read the
> > instructions: https://lists.samba.org/mailman/options/samba
> >
> >
> >
> > --
> >
> >
> >
> >
> > www.it-optics.com
> >
> > Gaëtan SLONGO | Head of Infrastructure Department
> > Boulevard Initialis, 28 - 7000 Mons, BELGIUM
> > Company : +32 (0)65 84 23 85
> > Direct : +32 (0)65 32 85 88
> > Fax : +32 (0)65 84 66 76
> > Skype ID : gslongo.pro
> > GPG Key : gslongo-gpg_key.asc
> >
> >
> > - Please consider your environmental responsibility before
> > printing
> > this e-mail -
> >
> >
> >
> >
> >
> >
> >
> >
>
> --
> Andrew Bartlett http://samba.org/~abartlet/
> Authentication Developer, Samba Team http://samba.org
> Samba Developer, Catalyst IT http://catalyst.net.nz/services/samba
>
>
>
>
> --
>
>
>
>
> www.it-optics.com
>
> Gaëtan SLONGO | Head of Infrastructure Department
> Boulevard Initialis, 28 - 7000 Mons, BELGIUM
> Company : +32 (0)65 84 23 85
> Direct : +32 (0)65 32 85 88
> Fax : +32 (0)65 84 66 76
> Skype ID : gslongo.pro
> GPG Key : gslongo-gpg_key.asc
>
>
> - Please consider your environmental responsibility before printing
> this e-mail -
>
>
>
>
>
>
>
>
--
Andrew Bartlett http://samba.org/~abartlet/
Authentication Developer, Samba Team http://samba.org
Samba Developer, Catalyst IT http://catalyst.net.nz/services/samba
More information about the samba
mailing list