[Samba] How to profile a samba function

[Alex Matthews]

On 17/10/2013 22:53, Andrew Bartlett wrote:
> On Thu, 2013-10-17 at 15:18 +0100, Alex Matthews wrote:
>> Hi all,
>>
>> I am trying to track down what is causing the slow running of
>> sysvolreset on my system.
>> I am aware that I have a great deal of files in my 'netlogon' directory
>> which is why the problem is so apparent for me.
>>
>> sysvolreset takes approx 5+ minutes to complete.
>>
>> Tracing out samba-tool shows that the slowdown is caused by the samba
>> function 'set_nt_acl'.
>> This function takes approx 120ms to run, which when multiplied up by a
>> few thousand files (2,000 in my case) you very quickly end up in minutes.
>>
>> running samba-tool using cProfile ('python2.7 -m cProfile
>> /usr/bin/samba-tool ntacl sysvolreset') gives the following results:
>> http://pastebin.com/98rYXRxe
>>
>> As you can see the only call that takes any 'significant' time is
>> "samba.samba3.smbd.set_nt_acl".
>> Because this function is part of the samba source rather than part of a
>> python script cProfile cannot profile any further.
>> How can I profile this function in smbd (without having to recompile
>> samba would be ideal).
>> I've seen 'perf' mentioned in other threads, is there any way this tool
>> can be used?
>> If I have to recompile and add my own timing code I will, but only
>> really as a last resort.
>>
>> This problem can't be isolated to just sysvolreset as the 'set_nt_acl'
>> function must be called quite frequently elsewhere.
>> I think 120ms for a call to return is quite a long time. I would expect
>> to see an overall performance gain if it can be optimised a bit!
> Perf is ideal for this, and has quickly become my favourite profiling
> tool, because it's the first one I've found that I can just pick up and
> use!.  'perf record -g /usr/bin/samba-tool ntacl sysvolreset' and then
> 'perf report -g' will give a lot of really useful output, in particular
> call graphs to isolate not only where the time is spent, but who is
> requesting the time be spent.
>
> I've CC'ed Nikos, who needs help profiling samba-tool dbcheck.  I wonder
> if you could help each other out with the profiling?
>
> Thanks,
>
> Andrew Bartlett
>
<sigh>

Isn't it just sods law that when I run sysvolreset on the test VM I've 
set up at home the set_nt_acl function works a great deal quicker... 
30ms per hit rather than 120ms.

Plan B.... when I've worked out what that is...


Yeah, I'm happy to help out Nikos, once I have a working setup.

Thanks,

Alex