Talloc: pool optimizations

[Stefan (metze) Metzmacher]

Am 04.04.2011 08:25, schrieb Rusty Russell:
> On Thu, 31 Mar 2011 20:17:38 +0200, "Stefan (metze) Metzmacher" <metze at samba.org> wrote:
>> Hi,
>>
>> using 2k from a 8k talloc stackframe seems to be bad.
>>
>> I've coded some talloc_free and talloc_realloc optimizations
>> which will try to reclaim unused memory from the talloc pool
>> if it's the last chunk in the pool or if the pool is empty.
>>
>> See
>> http://gitweb.samba.org/?p=metze/samba/wip.git;a=shortlog;h=refs/heads/master4-talloc
>>
>> I need to add some more VALGRIND_MAKE_MEM_NOACCESS and
>> talloc_fill logic, but the main code seems to work.

I've fixed the VALGRIND and TALLOC_FILL stuff.

The top 2 commits are unrelated, Tridge and Rusty please comment on them
if they're correct or not.

Is it ok to push everything without TODO in the commit message?

> Looks like some nice cleanups, but do you have any benchmarks showing
> the results?

It's no optimization is speed, but in memory usage.

In the speed tests it's 1-2% slower, but should lower the risk
that we falldown to 50% when the pool is exhausted.

As

pool = talloc_pool(parent,  4096);

while(1) {
   mem = talloc_size(pool, 2048);
   talloc_free(mem);
}

will now only trigger one malloc() call.

While it used malloc() in each cycle,
except for the first time.

Here're the results of the old code CFLAGS="-O3" ./configure...:

test: speed
# TALLOC VS MALLOC SPEED
talloc: 12781835 ops/sec
talloc_pool: 31559571 ops/sec
malloc: 17022362 ops/sec
success: speed
test: speed
# TALLOC VS MALLOC SPEED
talloc: 13085579 ops/sec
talloc_pool: 30293455 ops/sec
malloc: 16527790 ops/sec
success: speed
test: speed
# TALLOC VS MALLOC SPEED
talloc: 12803849 ops/sec
talloc_pool: 30976775 ops/sec
malloc: 16860522 ops/sec
success: speed
test: speed
# TALLOC VS MALLOC SPEED
talloc: 13043272 ops/sec
talloc_pool: 31211875 ops/sec
malloc: 17051322 ops/sec
success: speed
test: speed
# TALLOC VS MALLOC SPEED
talloc: 13041261 ops/sec
talloc_pool: 30367133 ops/sec
malloc: 16896181 ops/sec
success: speed
test: speed
# TALLOC VS MALLOC SPEED
talloc: 13053070 ops/sec
talloc_pool: 31137027 ops/sec
malloc: 17174976 ops/sec
success: speed
test: speed
# TALLOC VS MALLOC SPEED
talloc: 13439987 ops/sec
talloc_pool: 31830984 ops/sec
malloc: 17240897 ops/sec
success: speed
test: speed
# TALLOC VS MALLOC SPEED
talloc: 13400188 ops/sec
talloc_pool: 31849303 ops/sec
malloc: 17214331 ops/sec
success: speed
test: speed
# TALLOC VS MALLOC SPEED
talloc: 13269733 ops/sec
talloc_pool: 31488898 ops/sec
malloc: 17125516 ops/sec
success: speed
test: speed
# TALLOC VS MALLOC SPEED
talloc: 13238643 ops/sec
talloc_pool: 31475073 ops/sec
malloc: 17107697 ops/sec
success: speed

And here're the results of the new code:

test: speed
# TALLOC VS MALLOC SPEED
talloc: 13186341 ops/sec
talloc_pool: 30851005 ops/sec
malloc: 17216979 ops/sec
success: speed
test: speed
# TALLOC VS MALLOC SPEED
talloc: 13181831 ops/sec
talloc_pool: 30896357 ops/sec
malloc: 17231835 ops/sec
success: speed
test: speed
# TALLOC VS MALLOC SPEED
talloc: 13187549 ops/sec
talloc_pool: 30855458 ops/sec
malloc: 17210460 ops/sec
success: speed
test: speed
# TALLOC VS MALLOC SPEED
talloc: 13190478 ops/sec
talloc_pool: 30869938 ops/sec
malloc: 17202335 ops/sec
success: speed
test: speed
# TALLOC VS MALLOC SPEED
talloc: 13158495 ops/sec
talloc_pool: 30863593 ops/sec
malloc: 17202801 ops/sec
success: speed
test: speed
# TALLOC VS MALLOC SPEED
talloc: 13090292 ops/sec
talloc_pool: 30055817 ops/sec
malloc: 16899066 ops/sec
success: speed
test: speed
# TALLOC VS MALLOC SPEED
talloc: 12977385 ops/sec
talloc_pool: 30633079 ops/sec
malloc: 17228473 ops/sec
success: speed
test: speed
# TALLOC VS MALLOC SPEED
talloc: 13182423 ops/sec
talloc_pool: 30494187 ops/sec
malloc: 16934261 ops/sec
success: speed
test: speed
# TALLOC VS MALLOC SPEED
talloc: 12981972 ops/sec
talloc_pool: 29675275 ops/sec
malloc: 17103556 ops/sec
success: speed
test: speed
# TALLOC VS MALLOC SPEED
talloc: 12978224 ops/sec
talloc_pool: 29975959 ops/sec
malloc: 16895312 ops/sec
success: speed



-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 262 bytes
Desc: OpenPGP digital signature
URL: <http://lists.samba.org/pipermail/samba-technical/attachments/20110404/81aac3fc/attachment.pgp>