New talloc feature: memlimits

[David Disseldorp]

Hi Simo,

On Sun, 23 Sep 2012 00:10:18 -0600
idra at samba.org wrote:

> Hello list,
> 
> during the recent SDC Conference we had the Samba4 LDAP server hammered by
> the Codenomicon guys. A few bugs were found where we ended up allocating huge
> amounts of memory.
> 
> These bugs will need fixing, but the situation reminded me that we still have
> little or no control on what users can do over ldap. In particular we have no
> good way to limit resources, and it is relatively easy to DoS the LDAP server
> by making it allocate huge amounts of memory.
> 
> So I had the idea of limiting memory allocation to arbitrarily settable sizes
> based on talloc contextes.
> 
> Attached you can find an initial implementation of this feature with basic
> tests.

...

> Well, enough said, if there are any objections on committin gthis change please
> speak up, otherwise I will push by the end of the week.

This looks like a useful debugging feature, but I don't think it should
be enabled by default.

Firstly, it adds extra weight and complexity to one of the hottest code
paths in Samba. The TALLOC VS MALLOC SPEED test should be able to
provide data on the performance hit incurred under various limit
conditions.

As a measure against DOS attacks, it sits at the wrong layer IMO.
Requests resulting in excessively large tallocs should be intercepted
explicitly during unmarshalling.
cgroups / ulimit already provide the ability to restrict memory usage
per-process.

Furthermore, functions called with talloc_ctx arguments require intimate
knowledge of the memory context passed in by the caller to ensure they
do not exceed any pre-imposed limits.

Finally, increased reliance is placed on graceful memory allocation
failure error handling, a code path which often receives little testing.

Cheers, David