CTDB scaling?

Richard Sharpe realrichardsharpe at gmail.com
Thu Nov 20 10:33:14 MST 2014

On Wed, Nov 19, 2014 at 5:44 PM, ronnie sahlberg
<ronniesahlberg at gmail.com> wrote:
> On Wed, Nov 19, 2014 at 3:49 PM, Richard Sharpe
> <realrichardsharpe at gmail.com> wrote:
>> Hi folks,
>> In Tridge's 2007 paper:
>> he claims the following performance scaling:
>> https://www.samba.org/~tridge/sambaxp-07/ctdb.pdf
>> NEW (CTDB) approach
>> 1 node 42 Mbytes/sec
>> 2 nodes 168 MBytes/sec
>> 3 nodes 211 MBytes/sec
>> 4 nodes 243 MBytes/sec
>> This seems counter intuitive. 2 nodes gets four times what one node
>> gets and four nodes gets almost six times what 1 node does?
>> What is the explanation for that?
> The superlinear scaling is likely due to the increase of memory for caching.
> This is recall is the uncontended case where you have little cross node
> traffic.

Hmm, it is still not obvious. There seems to be several things going on here.

Is it possible that the same NBENCH load was offered across all four

That would make more sense. Then, in the one node case we were hitting
the one node limit, and as you say, with two nodes and the load
divided between them, more memory was available for caching so we see
a big boost there. After that, we seem to be hitting the IO limit of
the cluster because more memory does not seem to help that much ... By
the time we hit six nodes it looks like we would probably be seeing
only another 20MB/s or less for the additional nodes.

Richard Sharpe

More information about the samba-technical mailing list