samba-tool ldapcmp hangs when too much objects in Samba database

[mathias dufresne]

Unfortunately, no. The command itself hangs with no more than:
---------------------------------------------------------------------------------
samba-tool ldapcmp ldap://deb1.domain.tld ldap://deb2.domain.tld domain

* Comparing [DOMAIN] context...

* Objects to be compared: 39790
ERROR(ldb): uncaught exception - LDAP client internal error:
NT_STATUS_UNEXPECTED_NETWORK_ERROR
  File "/usr/lib/python2.7/dist-packages/samba/netcmd/__init__.py", line
175, in _run
    return self.run(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/samba/netcmd/ldapcmp.py", line
983, in run
    if b1 == b2:
  File "/usr/lib/python2.7/dist-packages/samba/netcmd/ldapcmp.py", line
774, in __eq__
    outf=self.outf, errf=self.errf)
  File "/usr/lib/python2.7/dist-packages/samba/netcmd/ldapcmp.py", line
396, in __init__
    self.attributes = self.con.get_attributes(self.dn)
  File "/usr/lib/python2.7/dist-packages/samba/netcmd/ldapcmp.py", line
207, in get_attributes
    res = self.ldb.search(base=object_dn, scope=SCOPE_BASE, attrs=["*"])
---------------------------------------------------------------------------------

And I don't know how to extend logs verbosity without being submerged by
them because there is a lot of SMB traffic on the LAN where are my test DCs.

If you are interested in that I can build a new platform at home (where
there is no SMB traffic) with enough objects to make that command hang.
Doing that I should be able to get interesting logs...

Do you want I proceed? I would do that next days if you want to. If you
have some hints to increase logs verbosity, that would be great.

Beset regards,

mathias


2015-11-04 0:50 GMT+01:00 Jeremy Allison <jra at samba.org>:

> On Wed, Oct 28, 2015 at 02:17:55PM +0100, mathias dufresne wrote:
> > Hi all,
> >
> > samba-tool ldapcmp always hangs when database contains too much object.
> > This happened to me only when checking "domain" using that tool, most
> > certainly because I have very few objects in others contexts.
> >
> > The limit seems to be around 40 000 objects. That limit was defined by
> > running samba-tool ldapcmp on a new domain on which I pushed users by
> > bunches of 500 users, until the command refuse to finish successfully.
> >
> > As a workaround an option is to avoid containers with more than a certain
> > amount of object, with this amount less than 40 000, then to proceed with
> > ldapcmp on each container declared in AD with a scope equal to "one", to
> > not check this containers recursively.
> >
> > The main bad point of this workaround is time needed to compare the whole
> > tree. This tool is already needing an amount of time relatively important
> > to succeed, I don't expect launching it once on each AD container to be
> > something which would accelerate that process.
> >
> > I fully understand that kind of issue only happen for big company which
> > have enough objects to include in AD and I totally agree that kind of
> > company which is big enough can raise funds to help open source software
> to
> > be developed when they need some improvement.
> > So I asked the company I'm working for to raise funds and they are
> > discussing internally for months now about that funds raising, things are
> > going further, but very slowly.
> >
> > So the question is the following: did someone have noticed such an issue
> > and/or did someone have an idea on how to improve that?
>
> Do you have more data on exactly what in Samba is failing in this case ?
>