[PATCH] Netlogon prefork support
Gary Lockyer
gary at catalyst.net.nz
Mon Nov 5 19:30:23 UTC 2018
And again with the patch.
This patch set:
* switches the ad_dc test environment to the pre-fork process model
* increases the default number of pre-fork workers from 1 to 4
* Allows the netlogon service to run on pre-forked worker processes
CI results: https://gitlab.com/samba-team/devel/samba/pipelines/35387410
The rpc_server can now run in the pre-fork process model. Due to the use
of shared handles and resources all of the rpc end points are serviced
in the first worker process. Those end points that can be run in
multiple processes (currently only Netlogon and management) are serviced
in the first and any subsequent workers.
Still to come, pre-fork worker restart and limits on the number of
connection processes started by the standard process model.
Review and comments appreciated.
Thanks
Gary.
>
>
-------------- next part --------------
From 1475efc63e1fc9f64bd836c86c382730c2d327f4 Mon Sep 17 00:00:00 2001
From: Gary Lockyer <gary at catalyst.net.nz>
Date: Tue, 28 Aug 2018 15:26:03 +1200
Subject: [PATCH 1/6] autobuild: make ad_dc use prefork process model
Convert the ad_dc environment from single process model to the prefork
process model.
Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
---
selftest/target/Samba4.pm | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/selftest/target/Samba4.pm b/selftest/target/Samba4.pm
index 41e550b51c0..1f142336293 100755
--- a/selftest/target/Samba4.pm
+++ b/selftest/target/Samba4.pm
@@ -2554,7 +2554,7 @@ sub setup_ad_dc
return undef;
}
- if (not defined($self->check_or_start($env, "single"))) {
+ if (not defined($self->check_or_start($env, "prefork"))) {
return undef;
}
--
2.17.1
From 8041924716c027bd34c1481a6a72a67011320a61 Mon Sep 17 00:00:00 2001
From: Gary Lockyer <gary at catalyst.net.nz>
Date: Wed, 5 Sep 2018 13:59:34 +1200
Subject: [PATCH 2/6] source4 smbd prefork: Increase default worklers to 4
Increase the default number of worker processes started by the pre-fork
process model from 1 to 4.
Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
---
docs-xml/smbdotconf/base/preforkchildren.xml | 8 ++++----
lib/param/loadparm.c | 3 +--
source3/param/loadparm.c | 2 +-
3 files changed, 6 insertions(+), 7 deletions(-)
diff --git a/docs-xml/smbdotconf/base/preforkchildren.xml b/docs-xml/smbdotconf/base/preforkchildren.xml
index 720e43909cb..ab13efce405 100644
--- a/docs-xml/smbdotconf/base/preforkchildren.xml
+++ b/docs-xml/smbdotconf/base/preforkchildren.xml
@@ -6,9 +6,9 @@
<para>This option controls the number of worker processes that are
started for each service when prefork process model is enabled.
The prefork children are only started for those services that
- support prefork (currently only ldap). For processes that don't
- support preforking all requests are handled by a single process
- for that service.
+ support prefork (currently ldap, kdc and netlogon).
+ For processes that don't support preforking all requests are
+ handled by a single process for that service.
</para>
<para>This should be set to a small multiple of the number of CPU's
@@ -20,5 +20,5 @@
worker processes.</para>
</description>
-<value type="default">1</value>
+<value type="default">4</value>
</samba:parameter>
diff --git a/lib/param/loadparm.c b/lib/param/loadparm.c
index 1debcfff127..484c8913009 100644
--- a/lib/param/loadparm.c
+++ b/lib/param/loadparm.c
@@ -2772,7 +2772,6 @@ struct loadparm_context *loadparm_init(TALLOC_CTX *mem_ctx)
lpcfg_do_global_parameter(lp_ctx, "tls certfile", "tls/cert.pem");
lpcfg_do_global_parameter(lp_ctx, "tls cafile", "tls/ca.pem");
lpcfg_do_global_parameter(lp_ctx, "tls priority", "NORMAL:-VERS-SSL3.0");
- lpcfg_do_global_parameter(lp_ctx, "prefork children:smb", "4");
lpcfg_do_global_parameter(lp_ctx, "rndc command", "/usr/sbin/rndc");
lpcfg_do_global_parameter(lp_ctx, "nsupdate command", "/usr/bin/nsupdate -g");
@@ -2997,7 +2996,7 @@ struct loadparm_context *loadparm_init(TALLOC_CTX *mem_ctx)
"rpc server dynamic port range",
"49152-65535");
- lpcfg_do_global_parameter(lp_ctx, "prefork children", "1");
+ lpcfg_do_global_parameter(lp_ctx, "prefork children", "4");
lpcfg_do_global_parameter(lp_ctx, "check parent directory delete on close", "no");
diff --git a/source3/param/loadparm.c b/source3/param/loadparm.c
index 322934c55f0..495683083cb 100644
--- a/source3/param/loadparm.c
+++ b/source3/param/loadparm.c
@@ -957,7 +957,7 @@ static void init_globals(struct loadparm_context *lp_ctx, bool reinit_globals)
"49152-65535");
Globals.rpc_low_port = SERVER_TCP_LOW_PORT;
Globals.rpc_high_port = SERVER_TCP_HIGH_PORT;
- Globals.prefork_children = 1;
+ Globals.prefork_children = 4;
/* Now put back the settings that were set with lp_set_cmdline() */
apply_lp_set_cmdline();
--
2.17.1
From 8d82472666b2789adb17e8e4db416033ceddf461 Mon Sep 17 00:00:00 2001
From: Gary Lockyer <gary at catalyst.net.nz>
Date: Wed, 19 Sep 2018 15:13:49 +1200
Subject: [PATCH 3/6] WHATSNEW: 'prefork children' default
Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
---
WHATSNEW.txt | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/WHATSNEW.txt b/WHATSNEW.txt
index 7a7a88d4152..211133447a8 100644
--- a/WHATSNEW.txt
+++ b/WHATSNEW.txt
@@ -33,6 +33,12 @@ kdc prefork
The KDC now supports the pre-fork process model and worker processes will be
forked for the KDC when the pre-fork process model is selected for samba.
+prefork 'prefork children'
+--------------------------
+
+The default value for this smdb.conf parameter has been increased from 1 to
+4.
+
REMOVED FEATURES
================
--
2.17.1
From b7c912947f3d524d8dc1fa7c0bd90996e4583809 Mon Sep 17 00:00:00 2001
From: Gary Lockyer <gary at catalyst.net.nz>
Date: Wed, 29 Aug 2018 13:16:08 +1200
Subject: [PATCH 4/6] source4 smbd: pass instance number to post_fork
Pass the instance number to the post_fork hook. This is required to
allow the rpc_server to support the prefork process model.
Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
---
source4/kdc/kdc-heimdal.c | 2 +-
source4/smbd/process_prefork.c | 11 +++++++----
source4/smbd/process_single.c | 3 ++-
source4/smbd/process_standard.c | 3 ++-
source4/smbd/service.h | 10 +++++++++-
5 files changed, 21 insertions(+), 8 deletions(-)
diff --git a/source4/kdc/kdc-heimdal.c b/source4/kdc/kdc-heimdal.c
index 407de92054e..67c07c3d81a 100644
--- a/source4/kdc/kdc-heimdal.c
+++ b/source4/kdc/kdc-heimdal.c
@@ -316,7 +316,7 @@ static NTSTATUS kdc_task_init(struct task_server *task)
/*
initialise the kdc task after a fork
*/
-static void kdc_post_fork(struct task_server *task)
+static void kdc_post_fork(struct task_server *task, struct process_details *pd)
{
struct kdc_server *kdc;
krb5_kdc_configuration *kdc_config = NULL;
diff --git a/source4/smbd/process_prefork.c b/source4/smbd/process_prefork.c
index 788aa976390..016385c762c 100644
--- a/source4/smbd/process_prefork.c
+++ b/source4/smbd/process_prefork.c
@@ -240,6 +240,7 @@ static void prefork_new_task(
struct tevent_context *ev2;
struct task_server *task = NULL;
+ struct process_details pd = initial_process_details;
t = tfork_create();
if (t == NULL) {
@@ -284,7 +285,7 @@ static void prefork_new_task(
* The task does not support pre-fork
*/
if (task != NULL && service_details->post_fork != NULL) {
- service_details->post_fork(task);
+ service_details->post_fork(task, &pd);
}
tevent_loop_wait(ev);
TALLOC_FREE(ev);
@@ -348,6 +349,7 @@ static void prefork_new_task(
"after fork");
}
tevent_fd_set_auto_close(fde);
+ pd.instances++;
} else {
/*
* tfork uses malloc
@@ -355,12 +357,13 @@ static void prefork_new_task(
free(w);
TALLOC_FREE(ev);
- setproctitle("task[%s] pre-forked worker",
- service_name);
+ setproctitle("task[%s] pre-forked worker(%d)",
+ service_name,
+ pd.instances);
prefork_reload_after_fork();
setup_handlers(ev2, from_parent_fd);
if (service_details->post_fork != NULL) {
- service_details->post_fork(task);
+ service_details->post_fork(task, &pd);
}
tevent_loop_wait(ev2);
talloc_free(ev2);
diff --git a/source4/smbd/process_single.c b/source4/smbd/process_single.c
index 242622b3b8f..174c1572812 100644
--- a/source4/smbd/process_single.c
+++ b/source4/smbd/process_single.c
@@ -114,7 +114,8 @@ static void single_new_task(struct tevent_context *ev,
*/
task = new_task(ev, lp_ctx, cluster_id(pid, taskid++), private_data, NULL);
if (task != NULL && service_details->post_fork != NULL) {
- service_details->post_fork(task);
+ struct process_details pd = initial_process_details;
+ service_details->post_fork(task, &pd);
}
}
diff --git a/source4/smbd/process_standard.c b/source4/smbd/process_standard.c
index 62620096af5..91dfa9753c5 100644
--- a/source4/smbd/process_standard.c
+++ b/source4/smbd/process_standard.c
@@ -494,7 +494,8 @@ static void standard_new_task(struct tevent_context *ev,
* is forked in standard_accept_connection.
*/
if (task != NULL && service_details->post_fork != NULL) {
- service_details->post_fork(task);
+ struct process_details pd = initial_process_details;
+ service_details->post_fork(task, &pd);
}
diff --git a/source4/smbd/service.h b/source4/smbd/service.h
index 467cb34f6ca..0b6f937b63c 100644
--- a/source4/smbd/service.h
+++ b/source4/smbd/service.h
@@ -27,6 +27,14 @@
#include "smbd/service_stream.h"
#include "smbd/service_task.h"
+struct process_details {
+ unsigned int instances;
+};
+
+static const struct process_details initial_process_details = {
+ .instances = 0
+};
+
struct service_details {
/*
* Prevent the standard process model from forking a new worker
@@ -64,7 +72,7 @@ struct service_details {
* process is forked on a new connection. It is instead called
* immediately after the task_init.
*/
- void (*post_fork) (struct task_server *);
+ void (*post_fork) (struct task_server *, struct process_details *);
};
#include "smbd/service_proto.h"
--
2.17.1
From 7695377cd47be6e2661004a44de4121298af09e3 Mon Sep 17 00:00:00 2001
From: Gary Lockyer <gary at catalyst.net.nz>
Date: Wed, 29 Aug 2018 15:07:32 +1200
Subject: [PATCH 5/6] source4 smdb rpc_server: Support prefork process model.
Allow the rpc_server to run in the prefork process model. Due to the use
of shared handles and resources all of the rpc end points are serviced
in the first worker process. Those end points that can be run in
multiple processes (currently only Netlogon and management) are serviced in
the first and any subsequent workers.
Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
---
source4/rpc_server/service_rpc.c | 195 ++++++++++++++++++-------------
1 file changed, 115 insertions(+), 80 deletions(-)
diff --git a/source4/rpc_server/service_rpc.c b/source4/rpc_server/service_rpc.c
index 07d6f700687..6cd7d417c19 100644
--- a/source4/rpc_server/service_rpc.c
+++ b/source4/rpc_server/service_rpc.c
@@ -39,122 +39,157 @@
#include "../libcli/named_pipe_auth/npa_tstream.h"
#include "smbd/process_model.h"
+/*
+ * Need to run the majority of the RPC endpoints in a single process to allow
+ * for shared handles, and the sharing of ldb contexts.
+ *
+ * However other endpoints are capable of being run in multiple processes
+ * e.g. NETLOGON.
+ *
+ * To support this the process model is manipulated to force those end points
+ * not supporting multiple processes into the single process model. The code
+ * responsible for this is in dcesrv_init_endpoints
+ *
+ */
NTSTATUS server_service_rpc_init(TALLOC_CTX *);
/*
- open the dcerpc server sockets
+ * Initialise the rpc endpoints.
+ */
+static NTSTATUS dcesrv_init_endpoints(struct task_server *task,
+ struct dcesrv_context *dce_ctx,
+ bool use_single_process)
+{
+
+ struct dcesrv_endpoint *e;
+ const struct model_ops *model_ops = NULL;
+
+ /*
+ * For those RPC services that run with shared context we need to
+ * ensure that they don't fork a new process on accept (standard_model).
+ * And as there is only one process handling these requests we need
+ * to handle accept errors in a similar manner to the single process
+ * model.
+ *
+ * To do this we override the process model operations with the single
+ * process operations. This is not the most elegant solution, but it is
+ * the least ugly, and is confined to the next block of code.
+ */
+ if (use_single_process == true) {
+ model_ops = process_model_startup("single");
+ if (model_ops == NULL) {
+ DBG_ERR("Unable to load single process model");
+ return NT_STATUS_INTERNAL_ERROR;
+ }
+ } else {
+ model_ops = task->model_ops;
+ }
+
+ for (e = dce_ctx->endpoint_list; e; e = e->next) {
+
+ enum dcerpc_transport_t transport =
+ dcerpc_binding_get_transport(e->ep_description);
+
+ if (transport == NCACN_HTTP) {
+ /*
+ * We don't support ncacn_http yet
+ */
+ continue;
+ }
+ if (e->use_single_process == use_single_process) {
+ NTSTATUS status;
+ status = dcesrv_add_ep(dce_ctx,
+ task->lp_ctx,
+ e,
+ task->event_ctx,
+ model_ops,
+ task->process_context);
+ if (!NT_STATUS_IS_OK(status)) {
+ return status;
+ }
+ }
+ }
+ return NT_STATUS_OK;
+}
+/*
+ * Initialise the RPC service.
+ * And those end points that can be serviced by multiple processes.
+ * The endpoints that need to be run in a single process are setup in the
+ * post_fork hook.
*/
static NTSTATUS dcesrv_task_init(struct task_server *task)
{
NTSTATUS status = NT_STATUS_UNSUCCESSFUL;
struct dcesrv_context *dce_ctx;
- struct dcesrv_endpoint *e;
- const struct model_ops *single_model_ops;
dcerpc_server_init(task->lp_ctx);
task_server_set_title(task, "task[dcesrv]");
- /*
- * run the rpc server as a single process to allow for shard
- * handles, and sharing of ldb contexts.
- *
- * We make an exception for NETLOGON below, and this follows
- * whatever the top level is.
- */
- single_model_ops = process_model_startup("single");
- if (!single_model_ops) goto failed;
-
status = dcesrv_init_context(task->event_ctx,
task->lp_ctx,
lpcfg_dcerpc_endpoint_servers(task->lp_ctx),
&dce_ctx);
- if (!NT_STATUS_IS_OK(status)) goto failed;
+ if (!NT_STATUS_IS_OK(status)) {
+ return status;
+ }
/* Make sure the directory for NCALRPC exists */
if (!directory_exist(lpcfg_ncalrpc_dir(task->lp_ctx))) {
mkdir(lpcfg_ncalrpc_dir(task->lp_ctx), 0755);
}
+ status = dcesrv_init_endpoints(task, dce_ctx, false);
+ if (!NT_STATUS_IS_OK(status)) {
+ return status;
+ }
- for (e=dce_ctx->endpoint_list;e;e=e->next) {
- const struct model_ops *this_model_ops = single_model_ops;
-
- enum dcerpc_transport_t transport =
- dcerpc_binding_get_transport(e->ep_description);
- const char *transport_str
- = derpc_transport_string_by_transport(transport);
-
- struct dcesrv_if_list *iface_list;
-
- /*
- * Ensure that -Msingle sets e->use_single_process for
- * consistency
- */
+ task->private_data = dce_ctx;
+ return NT_STATUS_OK;
+}
- if (task->model_ops == single_model_ops) {
- e->use_single_process = true;
- }
+/*
+ * Initialise the endpoints that need to run in a single process fork.
+ * The endpoint registration is only done for the first process instance.
+ *
+ */
+static void dcesrv_post_fork(struct task_server *task,
+ struct process_details *pd)
+{
- if (transport == NCACN_HTTP) {
- /*
- * We don't support ncacn_http yet
- */
- continue;
+ NTSTATUS status = NT_STATUS_UNSUCCESSFUL;
+ struct dcesrv_context *dce_ctx;
- /*
- * For the next two cases, what we are trying
- * to do is put the NETLOGON server into the
- * standard process model, not single, as it
- * has no shared handles and takes a very high
- * load. We only do this for ncacn_np and
- * ncacn_ip_tcp as otherwise it is too hard as
- * all servers share a socket for ncalrpc and
- * unix.
- */
- } else if (e->use_single_process == false) {
- this_model_ops = task->model_ops;
- }
+ if (task->private_data == NULL) {
+ task_server_terminate(task, "dcerpc: No dcesrv_context", true);
+ return;
+ }
+ dce_ctx =
+ talloc_get_type_abort(task->private_data, struct dcesrv_context);
- status = dcesrv_add_ep(dce_ctx, task->lp_ctx, e, task->event_ctx,
- this_model_ops, task->process_context);
+ /*
+ * Ensure the single process endpoints are only available to the
+ * first instance.
+ */
+ if (pd->instances == 0) {
+ status = dcesrv_init_endpoints(task, dce_ctx, true);
if (!NT_STATUS_IS_OK(status)) {
- goto failed;
+ task_server_terminate(
+ task,
+ "dcerpc: Failed to initialise end points",
+ true);
+ return;
}
-
- DEBUG(5,("Added endpoint on %s "
- "using process model %s for",
- transport_str,
- this_model_ops->name));
-
- for (iface_list = e->interface_list;
- iface_list != NULL;
- iface_list = iface_list->next) {
- DEBUGADD(5, (" %s", iface_list->iface.name));
- }
- DEBUGADD(5, ("\n"));
}
irpc_add_name(task->msg_ctx, "rpc_server");
- return NT_STATUS_OK;
-failed:
- task_server_terminate(task, "Failed to startup dcerpc server task", true);
- return status;
}
NTSTATUS server_service_rpc_init(TALLOC_CTX *ctx)
{
static const struct service_details details = {
- /*
- * This is a SNOWFLAKE, but sadly one that we
- * will have to keep for now. The RPC server
- * code above overstamps the SINGLE process model
- * most of the time, but we need to be in forking
- * mode by defult to get a forking NETLOGON server
- */
- .inhibit_fork_on_accept = false,
- .inhibit_pre_fork = true,
- .task_init = dcesrv_task_init,
- .post_fork = NULL
- };
+ .inhibit_fork_on_accept = false,
+ .inhibit_pre_fork = false,
+ .task_init = dcesrv_task_init,
+ .post_fork = dcesrv_post_fork};
return register_server_service(ctx, "rpc", &details);
}
--
2.17.1
From 0f0995016601d38b7bb98c805924021bd0559429 Mon Sep 17 00:00:00 2001
From: Gary Lockyer <gary at catalyst.net.nz>
Date: Wed, 19 Sep 2018 15:04:35 +1200
Subject: [PATCH 6/6] WHATSNEW: NETLOGON prefork support
Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
---
WHATSNEW.txt | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/WHATSNEW.txt b/WHATSNEW.txt
index 211133447a8..2177988c25f 100644
--- a/WHATSNEW.txt
+++ b/WHATSNEW.txt
@@ -39,6 +39,12 @@ prefork 'prefork children'
The default value for this smdb.conf parameter has been increased from 1 to
4.
+netlogon prefork
+----------------
+
+DCERPC now supports pre-forked NETLOGON processes. The netlogon processes are
+pre-forked when the prefork process model is selected for samba.
+
REMOVED FEATURES
================
--
2.17.1
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: OpenPGP digital signature
URL: <http://lists.samba.org/pipermail/samba-technical/attachments/20181106/cd2935b9/signature.sig>
More information about the samba-technical
mailing list