[Samba] possible memory leak in ldb module while dbcheck on RODC
L.P.H. van Belle
belle at bazuin.nl
Fri Mar 31 08:08:40 UTC 2017
Hai, Andrej,
Report a bug is best you can do now.
Greetz,
Louis
> -----Oorspronkelijk bericht-----
> Van: samba [mailto:samba-bounces at lists.samba.org] Namens Andrej Gessel via
> samba
> Verzonden: vrijdag 31 maart 2017 10:04
> Aan: samba at lists.samba.org
> Onderwerp: Re: [Samba] possible memory leak in ldb module while dbcheck on
> RODC
>
> Hi Louis,
>
> i follow your tip and build samba 4.6.2 with ldb 1.1.29.
>
> I can reproduce the same behavior. Memory is increasing continuously.
>
>
>
> Andrej
>
> -----Ursprüngliche Nachricht-----
> Von: samba [mailto:samba-bounces at lists.samba.org] Im Auftrag von L.P.H.
> van Belle via samba
> Gesendet: Freitag, 31. März 2017 09:28
> An: samba at lists.samba.org
> Betreff: Re: [Samba] possible memory leak in ldb module while dbcheck on
> RODC
>
> Hai Andrej,
>
> Oeps, Sorry, my mistake 4.6.x needs at least 1.1.29.
>
> But you can upgrade to 4.6.x ;-)
>
> Check again and if that problem then still exists, then i think you can
> better ask this one in the samba-technical list.
> Or if Rowland knows something more about this.
>
> One tip, wait untill tomorrow or the weekend, there wil be a new release
> due to the regression bug.
> There planning is to be release today.
>
>
> Louis
>
>
> > -----Oorspronkelijk bericht-----
> > Van: Andrej Gessel [mailto:Andrej.Gessel at janztec.com]
> > Verzonden: vrijdag 31 maart 2017 9:11
> > Aan: L.P.H. van Belle; samba at lists.samba.org
> > Onderwerp: AW: [Samba] possible memory leak in ldb module while
> > dbcheck on RODC
> >
> > Hello,
> >
> > i just checkout git repository and run # ./configure.develorer # make
> > # make install
> >
> > As i can read from lib/ldb/wscript it is version 1.1.27?
> >
> > How can i change to the version 1.1.29?
> >
> > Regards,
> >
> > Andrej
> >
> >
> > -----Ursprüngliche Nachricht-----
> > Von: samba [mailto:samba-bounces at lists.samba.org] Im Auftrag von L.P.H.
> > van Belle via samba
> > Gesendet: Freitag, 31. März 2017 09:03
> > An: samba at lists.samba.org
> > Betreff: Re: [Samba] possible memory leak in ldb module while dbcheck
> > on RODC
> >
> > What is the ldb version your using?
> > You need : 1.1.29 with samba 4.5.7.
> >
> > Greetz,
> >
> > Louis
> >
> >
> > > -----Oorspronkelijk bericht-----
> > > Van: samba [mailto:samba-bounces at lists.samba.org] Namens Andrej
> > > Gessel via samba
> > > Verzonden: vrijdag 31 maart 2017 8:55
> > > Aan: samba at lists.samba.org
> > > Onderwerp: Re: [Samba] possible memory leak in ldb module while
> > > dbcheck on RODC
> > >
> > > Hello,
> > >
> > > my message was not send with my attachment, because content-type was
> > > not allowed, sorry for that.
> > >
> > > i am trying to use Samba 4.5.7 as an RODC and maybe found a memory
> > > leak in ldb module.
> > >
> > > There is an known BUG
> > > https://bugzilla.samba.org/show_bug.cgi?id=11178
> > > and a known fix in dbcheck.
> > >
> > > If i run "samba-tool dbcheck --cross-ncs --fix", this running
> > > process starts with ~300MB and use at the end about 1,6GB of memory
> > > on test domain with 15000 users and 15000 groups(ntds.dit is only
> > > ~150MB). I tried to find out why it is increasing continuously.
> > >
> > >
> > > 1) I used memory_profiler and modified dbchecker.py:
> > >
> > > root at buildhost samba # git diff dbchecker.py diff --git
> > > a/dbchecker.py b/dbchecker.py index 032c0e7..6fe0d4d 100644
> > > --- a/dbchecker.py
> > > +++ b/dbchecker.py
> > > @@ -17,6 +17,7 @@
> > > # along with this program. If not, see
> <http://www.gnu.org/licenses/>.
> > > #
> > >
> > > +from memory_profiler import profile
> > > import ldb
> > > import samba
> > > import time
> > > @@ -356,6 +357,7 @@ systemFlags: -1946157056%s""" % (dn, guid_suffix),
> > > return False
> > > return True
> > >
> > > + @profile
> > > def do_modify(self, m, controls, msg, validate=True):
> > > '''perform a modify with optional verbose output'''
> > > if self.verbose:
> > > @@ -751,6 +753,7 @@ newSuperior: %s""" % (str(from_dn), str(to_rdn),
> > > str(to_base)))
> > > "Failed to rename object %s into %s" %
> > > (obj.dn,
> > > new_dn)):
> > > self.report("Renamed %s into %s" % (obj.dn, new_dn))
> > >
> > > + @profile
> > > def err_wrong_instancetype(self, obj, calculated_instancetype):
> > > '''handle a wrong instanceType'''
> > > self.report("ERROR: wrong instanceType %s on %s, should be
> > > %d" % (obj["instanceType"], obj.dn, calculated_instancetype)) @@
> > > -1541,6 +1544,7 @@ newSuperior: %s""" % (str(from_dn), str(to_rdn),
> > > str(to_base)))
> > >
> > > raise KeyError
> > >
> > > + @profile(precision=4)
> > > def check_object(self, dn, attrs=['*']):
> > > '''check one object'''
> > > if self.verbose:
> > >
> > > Output of memory_profiler shows me additional ~2MB of memory usage
> > > after every run of check_object function (show memory_profiler.txt
> > > for details partial, because of mailing list limits).
> > >
> > > 2) After i comment out the line in dbchecker.py, memory is not
> > > increasing anymore.
> > >
> > > @@ -356,13 +357,14 @@ systemFlags: -1946157056%s""" % (dn,
> guid_suffix),
> > > return False
> > > return True
> > >
> > > + @profile
> > > def do_modify(self, m, controls, msg, validate=True):
> > > '''perform a modify with optional verbose output'''
> > > if self.verbose:
> > > self.report(self.samdb.write_ldif(m,
> > ldb.CHANGETYPE_MODIFY))
> > > try:
> > > controls = controls + ["local_oid:%s:0" %
> > > dsdb.DSDB_CONTROL_DBCHECK]
> > > - self.samdb.modify(m, controls=controls,
> validate=validate)
> > > +# self.samdb.modify(m, controls=controls,
> validate=validate)
> > > except Exception, err:
> > > if self.in_transaction:
> > > raise CommandError("%s : %s" % (msg, err))
> > >
> > >
> > >
> > > 3) My first idea was that it is python problem with starting c
> > > modules, but then i found pyldb.c and py_ldb_modify function in it.
> > > Commenting out the line with ldb_request memory stays on the same
> > > value and does not increase anymore.
> > >
> > > root at buildhost samba (git)-[samba-4.5.7] # git diff lib/ldb/pyldb.c
> > > diff --git a/lib/ldb/pyldb.c b/lib/ldb/pyldb.c index
> > > bea837f..ed5309a
> > > 100644
> > > --- a/lib/ldb/pyldb.c
> > > +++ b/lib/ldb/pyldb.c
> > > @@ -1255,7 +1255,7 @@ static PyObject *py_ldb_modify(PyLdbObject
> > > *self, PyObject *args, PyObject *kwar
> > > return NULL;
> > > }
> > >
> > > - ret = ldb_request(ldb_ctx, req);
> > > + //ret = ldb_request(ldb_ctx, req);
> > > if (ret == LDB_SUCCESS) {
> > > ret = ldb_wait(req->handle, LDB_WAIT_ALL);
> > > }
> > >
> > >
> > >
> > > So now to my question:
> > > Is it really a bug or does ldb library do something, that i did not
> > > see/understand till now?
> > >
> > >
> > > regards,
> > >
> > > Andrej
> > >
> > > Von: samba [mailto:samba-bounces at lists.samba.org] Im Auftrag von
> > > Andrej Gessel via samba
> > > Gesendet: Donnerstag, 30. März 2017 23:33
> > > An: samba at lists.samba.org
> > > Betreff: [Samba] possible memory leak in ldb module while dbcheck on
> > > RODC
> > >
> > >
> > > --
> > > To unsubscribe from this list go to the following URL and read the
> > > instructions: https://lists.samba.org/mailman/options/samba
> >
> >
> >
> > --
> > To unsubscribe from this list go to the following URL and read the
> > instructions: https://lists.samba.org/mailman/options/samba
>
>
> --
> To unsubscribe from this list go to the following URL and read the
> instructions: https://lists.samba.org/mailman/options/samba
> --
> To unsubscribe from this list go to the following URL and read the
> instructions: https://lists.samba.org/mailman/options/samba
More information about the samba
mailing list