New talloc feature: memlimits

[idra at samba.org]

On Mon, Sep 24, 2012 at 04:06:35PM +0200, David Disseldorp wrote:
> Hi Simo,
> 
> On Sun, 23 Sep 2012 00:10:18 -0600
> idra at samba.org wrote:
> 
> > Hello list,
> > 
> > during the recent SDC Conference we had the Samba4 LDAP server hammered by
> > the Codenomicon guys. A few bugs were found where we ended up allocating huge
> > amounts of memory.
> > 
> > These bugs will need fixing, but the situation reminded me that we still have
> > little or no control on what users can do over ldap. In particular we have no
> > good way to limit resources, and it is relatively easy to DoS the LDAP server
> > by making it allocate huge amounts of memory.
> > 
> > So I had the idea of limiting memory allocation to arbitrarily settable sizes
> > based on talloc contextes.
> > 
> > Attached you can find an initial implementation of this feature with basic
> > tests.
> 
> ...
> 
> > Well, enough said, if there are any objections on committin gthis change please
> > speak up, otherwise I will push by the end of the week.
> 
> This looks like a useful debugging feature, but I don't think it should
> be enabled by default.

As you can see 'enabling' it depends entirely on using tallos_set_memlimits(),
if you don't use it it is not enabled.
 
> Firstly, it adds extra weight and complexity to one of the hottest code
> paths in Samba. The TALLOC VS MALLOC SPEED test should be able to
> provide data on the performance hit incurred under various limit
> conditions.

It is true that talloc is a hot path, but I am sure this feature will not cost much.
I will run some test later on so we have more data.

> As a measure against DOS attacks, it sits at the wrong layer IMO.
> Requests resulting in excessively large tallocs should be intercepted
> explicitly during unmarshalling.

Ok, I do not agree with this point.
This feature is not the primary defence for sure, but it is a useful tool in
that regard.
However my main idea was to use it to implement memory limits in the ldap server.
Most LDAP server implementations have both size and memory limits per connection,
and this feature allows us to retrofit those limits in easily w/o having to change
tons of code.

> cgroups / ulimit already provide the ability to restrict memory usage
> per-process.

I want to use this per (authenticated) connection :)

> Furthermore, functions called with talloc_ctx arguments require intimate
> knowledge of the memory context passed in by the caller to ensure they
> do not exceed any pre-imposed limits.

It requires to be careful, and should certainly not be adopted too liberally.

> Finally, increased reliance is placed on graceful memory allocation
> failure error handling, a code path which often receives little testing.

This is true, however what better way than incresing test coverage than having a
way to test with random limits to improve these error handling paths ? :-)

Simo.

-- 
Simo Sorce       idra at samba.org
-------------------------------
Samba Team http://www.samba.org