memory usage and performance during schema modifications in samba4

[Anatoliy Atanasov]

Hi List,

I've stumbled upon an performance issue with samba4 during schema modifications so i've been busy lately looking into it. I couldn't figure out and solve the issue by myself i'll be happy if someone has time to help me resolve it.

I attached some useful scripts and patches that i did working on this.
1. I prepared a script, which is in the called flood_schema.py
run it against samba process:
python lib/ldb/tests/python/flood_schema.py ldap://localhost:389 -UAdministrator%pass

you can compare this against windows virtual box, the diff in the speed is 
	win2k8		samba
real	0m0.536s	0m7.469s

also running the script will cause huge increase in the memory usage: 250 adds are ~ 1G of memory

2. valgrind log after running the script - valgrind_flood.txt
kamen and i did some fixes(0001-mem-fixes.patch) to reduce mem usage but this didn't fix the bigger problem

3. i did run this script profiling with google perf tools
	* first patch to modify the waf build to link google perf tools - 0001-modify-waf....(this patch needs code review)
	* cpu.gif and memcheck.gif are the profiles i did on my machine


Looking at valgrind logs kamen thinks we load schemas in the context that we never free because of too much references so this might be it but i don't understand the area enough to debug and fix it.

Regards,
Anatoliy
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0001-mem-fixes.patch
Type: text/x-diff
Size: 3879 bytes
Desc: not available
URL: <http://lists.samba.org/pipermail/samba-technical/attachments/20100629/8644358e/attachment-0002.patch>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0001-modify-waf-build-to-link-google-perf-tools.patch
Type: text/x-diff
Size: 787 bytes
Desc: not available
URL: <http://lists.samba.org/pipermail/samba-technical/attachments/20100629/8644358e/attachment-0003.patch>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: flood_schema.py
Type: text/x-python
Size: 4070 bytes
Desc: not available
URL: <http://lists.samba.org/pipermail/samba-technical/attachments/20100629/8644358e/attachment-0001.py>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: valgrind_flood.txt
URL: <http://lists.samba.org/pipermail/samba-technical/attachments/20100629/8644358e/attachment-0001.txt>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: cpu.gif
Type: image/gif
Size: 243868 bytes
Desc: not available
URL: <http://lists.samba.org/pipermail/samba-technical/attachments/20100629/8644358e/attachment-0002.gif>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: memcheck.gif
Type: image/gif
Size: 228074 bytes
Desc: not available
URL: <http://lists.samba.org/pipermail/samba-technical/attachments/20100629/8644358e/attachment-0003.gif>