Samba3 memory usage, iconv

[Andrew Tridgell]

Dmitry,

 > It turns out that quite a lot of memory (~300 Kbytes) is allocated right 
 > at the start of each smbd process by init_iconv() (lib/charcnv.c).

I tried adding lazy_initialize_conv() before the fork in
server.c. That allows us to run 4900 smbd processes on the test system
in Mainz before the system dies. Without that change the it was around
4700, although it varies somewhat between runs. The system has 4G of
ram, but about 1.4G is used by system services (such as GPFS).

So I don't think it's really costing 300k per process for the iconv
tables. I suspect that massif is giving us misleading results.

I think we will need to fall back on more primitive techniques for
looking for high memory usage. For example DEBUG() lines showing
mallinfo() information would be useful.

To really scale Samba to large numbers of connections I think we will
need to adopt a pre-fork process model, more like apache. So we'd
specify how many smbd processes to fork (say 32 or 64 for a typical
server) then spread the load across those processes. I suspect we'd be
able to support between 10x and 50x as many connections on a given
server with that model.

Cheers, Tridge