[PATCH] Cache messaging dgm connections

[Ralph Böhme]

On Mon, Sep 12, 2016 at 07:28:54AM +1200, Andrew Bartlett wrote:
> On Sun, 2016-09-11 at 20:33 +0200, Ralph Böhme wrote:
> > On Sun, Sep 11, 2016 at 10:51:20AM -0700, Jeremy Allison wrote:
> > > 
> > > On Sun, Sep 11, 2016 at 04:07:15PM +0200, Ralph Böhme wrote:
> > > > 
> > > > Hi!
> > > > 
> > > > Attached is a performance improvement for our dgram messaging.
> > > > 
> > > > In most cases, the current code opens a new connection to peers
> > > > for
> > > > every message and closes it when done. This patchset adds caching
> > > > of
> > > > connections to peers.
> > > > 
> > > > Please review & push if ok.
> > > 
> > > OK, I have to ask :-) - I'm guessing you've already tested
> > > this and have a good idea of how much performance improvement you
> > > get under load...
> > > 
> > > Care to share the numbers ?
> > 
> > copying the Linux kernel sources with robocopy (with /MT:16) from a
> > Windows client to the server, in a perf profile
> > messaging_send_iov_from() showed up with taken 3%. Only 0.5% was for
> > the sendmsg(), the rest is (un)become_root() (2.2%) and
> > socket+connect.
> 
> And the overall performance?

hopefully better, I don't have any numbers. :)

> I ask because Douglas did a very similar analysis of some 'obvious'
> performance patches I did:
> 
> https://www.samba.org/~dbagnall/perf-tests/ldb.html
> 
> Likewise, when I added a cache of LDB index values between
> transactions, it was obvious that it would help, because an index read
> would be avoided, except when Douglas and I checked the performance, we
> found the cache management costs took up the same space.

The cache is merely a list of active connections to messaging peers,
so searching a cached connection means traversing a list which will
typically contain just a few elements.

Cheerio!
-slow