New LDAP C API

[Howard Chu]

Michael B Allen wrote:
> On Tue, 13 Feb 2007 17:06:49 -0800
> Howard Chu <hyc at highlandsun.com> wrote:
>
>   
>> Love Hörnquist Åstrand wrote:
>>     
>>> I don't see malloc happy as a big problem.
>>>       
>> Most people don't, they've been trained to believe that malloc is ok. 
>> But in the optimization work that went on between OpenLDAP 2.0 and 2.1 
>> we found that 50% of our CPU time was in libc, 30% malloc() related and 
>> 20% strlen(). One major reason that slapd in 2.1 benchmarks 200 times 
>> faster than slapd in 2.0 is because we eliminated most of those calls. 
>> Using BerVals everywhere was also a part of that.
>>
>> It's true that using a better malloc library can help (see my malloc 
>> benchmark results http://www.highlandsun.com/hyc/malloc/ ) but it's 
>> better not to use it at all if it can be avoided.
>>     
>
> Flow control in a server is simpler wrt the stack and alloction and is
> hidden whereas with a client the inverse is true. I think making the
> user supply buffers could be ugly. Believe me, I would love to do that. A
> lot of my code operates on buffers as opposed to using allocation. But
> with the sprawling trees produced by LDAP responses I just don't see a
> way around using allocation.
>
> Here's a possibly better way to handle memory allocation though:
>
> First, parameterize allocation by allowing the user to set function
> pointers (a typical "handler" struture) for an LDAP context. Samba devs
> will want this anyway so they can use their "talloc" code and "steal"
> objects. I too would like to use my allocators to elimitate copies. All
> memory for data that will be returned to the user would be allocated
> using these user supplied functions (but not for internal stuff).
>
> Second, make the default allocator one that does not implement the 'free'
> function since memory management for decodeing responses is pretty much
> all 'malloc' and no 'free' anyway. The allocator would be backed by a
> "chunk chain" of increasingly larger memory chunks allocated with real
> malloc(3).
>
>   
Yes. We do this already as well, but while it's exposed in the API 
generally only slapd uses it. It's actually set at the lber level since 
all encode/decode really happens there. The integration isn't ideal 
though, it was obviously grafted in after the fact and it shows...

-- 
  -- Howard Chu
  Chief Architect, Symas Corp.  http://www.symas.com
  Director, Highland Sun        http://highlandsun.com/hyc
  Chief Architect, OpenLDAP     http://www.openldap.org/project/