[Samba] possible memory leak in ldb module while dbcheck on RODC

[Andrej Gessel]

Hello,

my message was not send with my attachment, because content-type was not allowed,  sorry for that.

i am trying to use Samba 4.5.7 as an RODC and maybe found a memory leak in ldb module.

There is an known BUG https://bugzilla.samba.org/show_bug.cgi?id=11178 and a known fix in dbcheck.

If i run "samba-tool dbcheck --cross-ncs --fix", this running process starts with ~300MB and use at the end about 1,6GB of memory on test domain with 15000 users and 15000 groups(ntds.dit is only ~150MB). I tried to find out why it is increasing continuously.


1) I used memory_profiler and modified dbchecker.py:

root at buildhost samba # git diff dbchecker.py
diff --git a/dbchecker.py b/dbchecker.py
index 032c0e7..6fe0d4d 100644
--- a/dbchecker.py
+++ b/dbchecker.py
@@ -17,6 +17,7 @@
# along with this program.  If not, see <http://www.gnu.org/licenses/>.
#

+from memory_profiler import profile
import ldb
import samba
import time
@@ -356,6 +357,7 @@ systemFlags: -1946157056%s""" % (dn, guid_suffix),
             return False
         return True

+    @profile
     def do_modify(self, m, controls, msg, validate=True):
         '''perform a modify with optional verbose output'''
         if self.verbose:
@@ -751,6 +753,7 @@ newSuperior: %s""" % (str(from_dn), str(to_rdn), str(to_base)))
                           "Failed to rename object %s into %s" % (obj.dn, new_dn)):
             self.report("Renamed %s into %s" % (obj.dn, new_dn))

+    @profile
     def err_wrong_instancetype(self, obj, calculated_instancetype):
         '''handle a wrong instanceType'''
         self.report("ERROR: wrong instanceType %s on %s, should be %d" % (obj["instanceType"], obj.dn, calculated_instancetype))
@@ -1541,6 +1544,7 @@ newSuperior: %s""" % (str(from_dn), str(to_rdn), str(to_base)))

         raise KeyError

+    @profile(precision=4)
     def check_object(self, dn, attrs=['*']):
         '''check one object'''
         if self.verbose:

Output of memory_profiler shows me additional ~2MB of memory usage after every run of check_object function (show memory_profiler.txt for details partial, because of mailing list limits).

2) After i comment out the line in dbchecker.py, memory is not increasing anymore.

@@ -356,13 +357,14 @@ systemFlags: -1946157056%s""" % (dn, guid_suffix),
             return False
         return True

+    @profile
     def do_modify(self, m, controls, msg, validate=True):
         '''perform a modify with optional verbose output'''
         if self.verbose:
             self.report(self.samdb.write_ldif(m, ldb.CHANGETYPE_MODIFY))
         try:
             controls = controls + ["local_oid:%s:0" %
dsdb.DSDB_CONTROL_DBCHECK]
-            self.samdb.modify(m, controls=controls, validate=validate)
+#            self.samdb.modify(m, controls=controls, validate=validate)
         except Exception, err:
            if self.in_transaction:
                 raise CommandError("%s : %s" % (msg, err))



3) My first idea was that it is python problem with starting c modules, but then i found pyldb.c and py_ldb_modify function in it. Commenting out the line with ldb_request memory stays on the same value and does not increase anymore.

root at buildhost samba (git)-[samba-4.5.7] # git diff lib/ldb/pyldb.c
diff --git a/lib/ldb/pyldb.c b/lib/ldb/pyldb.c
index bea837f..ed5309a 100644
--- a/lib/ldb/pyldb.c
+++ b/lib/ldb/pyldb.c
@@ -1255,7 +1255,7 @@ static PyObject *py_ldb_modify(PyLdbObject *self, PyObject *args, PyObject *kwar
                return NULL;
        }

-       ret = ldb_request(ldb_ctx, req);
+       //ret = ldb_request(ldb_ctx, req);
        if (ret == LDB_SUCCESS) {
                ret = ldb_wait(req->handle, LDB_WAIT_ALL);
        }



So now to my question:
Is it really a bug or does ldb library do something, that i did not see/understand till now?


regards,

Andrej

Von: samba [mailto:samba-bounces at lists.samba.org] Im Auftrag von Andrej Gessel via samba
Gesendet: Donnerstag, 30. März 2017 23:33
An: samba at lists.samba.org
Betreff: [Samba] possible memory leak in ldb module while dbcheck on RODC