[Report] autobuild memory usage

[Jamie McClymont]

Hi all

Some background on the ldap-connection-closing patches from earlier:


I have been running some autobuilds on a 16GB vm, monitoring the memory
usage over time, with the aim of being able to reliably run them on a
smaller (read: cheaper) machine.

In both attached graphs, the red line is available memory throughout a
build from an unmodified master, and the vertical bars show when a new
testsuite starts in the green line.

Note: in both graphs there is a ~1wk difference between the base commit
of the two lines, so the visible difference in runtime is likely not
necessarily attributable to these changes -- I'll be doing more
performance testing on that soon.

Note large temporary spikes in memory usage which are not present in the
green version: these are ldap connections handlers from unclosed SamDBs,
which build up over the course of a suite. They put peak memory usage
well over 8GB, requiring a step up in machine size to escape the wrath
of the oom-killer.

del_patches.png shows the positive impact of the connection-closing
patchset on this.

Another obvious solution is to modify Samba4.pm, switching the process
models of the environments' DCs from standard to prefork. Unfortunately,
as seen in prefork.png, this increases the final memory usage by around
2.5GB. This is to be expected: by the end of the autobuild there are
about a dozen DCs running (some of which are only used for a few tests),
each of which is now running several pre-forked handler processes.


This brings me to my current project: enable selftest.pl to reorder
tests so they are clustered by environment (while maintaining the
defined order within a given environment). This would allow us to run
just one environment (and its dependencies) at a time: preliminary
results suggest this could get peak memory consumption under 4GB, the
magic number which would allow us to make use of DigitalOcean's free
GitLab CI runners.


Thanks,
- Jamie
-------------- next part --------------
A non-text attachment was scrubbed...
Name: del_patches.png
Type: image/png
Size: 155129 bytes
Desc: not available
URL: <http://lists.samba.org/pipermail/samba-technical/attachments/20180110/f26866d4/del_patches-0001.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: prefork.png
Type: image/png
Size: 162306 bytes
Desc: not available
URL: <http://lists.samba.org/pipermail/samba-technical/attachments/20180110/f26866d4/prefork-0001.png>