ctdb: Adding memory pool for queue callback
Swen Schillig
swen at vnet.ibm.com
Mon Oct 1 10:55:12 UTC 2018
On Fri, 2018-09-28 at 15:48 +1000, Martin Schwenke wrote:
> Hi Swen,
>
> This is generally a good improvement.
>
> I think the following part of the commit message probably needs to be
> updated:
>
> All received messages are processed synchronously per queue,
> therefore, a larger memory pool is not requried.
>
> There are definitely packets that are processed asynchronously. Any
> of
> the control handling functions called by ctdb_control_dispatch() that
> have async_reply passed to them can talloc_steal() the control
> structure
> (i.e. the packet, because structs on the wire!). Similarly for
> things
> like ctdb_reply_call().
>
> However, even with some packets processed asynchronously I think that
> this should provide a useful reduction in malloc()s.
>
> Can you please take a look at these code paths and, if convinced,
> update the commit message?
Agree.
Patch description updated.
Cheers Swen
-------------- next part --------------
From 97bc3dc7f35a0fbd3401a18cd01dfcb2be1c6eef Mon Sep 17 00:00:00 2001
From: Swen Schillig <swen at vnet.ibm.com>
Date: Mon, 12 Mar 2018 17:56:21 +0100
Subject: [PATCH] ctdb: Adding memory pool for queue callback
The received packet is copied into a newly allocated memory chunk for further
processing by the assigned callback. Once this is done, the memory is free'd.
This is repeated for each received packet making the memory allocation / free
an expensive task. To optimize this process, a memory pool is defined which
is sized identically to the queue's buffer.
During tests it could be seen that more than 95% of all messages were sized
below the standard buffer_size of 1k.
Signed-off-by: Swen Schillig <swen at vnet.ibm.com>
---
ctdb/common/ctdb_io.c | 9 ++++++++-
1 file changed, 8 insertions(+), 1 deletion(-)
diff --git a/ctdb/common/ctdb_io.c b/ctdb/common/ctdb_io.c
index 5bed7a61b31..073992fbce1 100644
--- a/ctdb/common/ctdb_io.c
+++ b/ctdb/common/ctdb_io.c
@@ -65,6 +65,7 @@ struct ctdb_queue {
size_t alignment;
void *private_data;
ctdb_queue_cb_fn_t callback;
+ char *data_pool;
const char *name;
uint32_t buffer_size;
};
@@ -115,7 +116,7 @@ static void queue_process(struct ctdb_queue *queue)
}
/* Extract complete packet */
- data = talloc_memdup(queue,
+ data = talloc_memdup(queue->data_pool,
queue->buffer.data + queue->buffer.offset,
pkt_size);
@@ -479,5 +480,11 @@ struct ctdb_queue *ctdb_queue_setup(struct ctdb_context *ctdb,
queue->buffer_size = 1024;
}
+ queue->data_pool = talloc_pool(queue, queue->buffer_size);
+ if (queue->data_pool == NULL) {
+ TALLOC_FREE(queue);
+ return NULL;
+ }
+
return queue;
}
--
2.17.1
More information about the samba-technical
mailing list