[PATCH] tevent-glib-glue
Ralph Böhme
slow at samba.org
Mon Mar 25 15:53:11 UTC 2019
On Mon, Mar 25, 2019 at 12:37:04PM +0000, Noel Power wrote:
>On 22/03/2019 15:21, Ralph Böhme wrote:
>>> I tried to run make test TESTS=samba3.tevent_glib_glue with no luck,
>>> doesn't seem to think the test exists (I don't know what I did wrong :/)
>>> Note: I can run the test manually from test_tevent_glib_glue.sh
>>
>> Hm, works here.
>
>I believe you!! I will have to figure out why this doesn't run here, it
>must be something specific to my environment as it clearly runs in the CI
Let me know if I can help!
>> All changes are in the FIXUP commits, other commit are unchanged
>> compared to the previous iteration.
>
>
>I find the do_glib_refresh checking a bit confusing
>
>
>to me
>
> if (point == TEVENT_TRACE_AFTER_LOOP_ONCE) {
> if (glue->skip_glib_refresh == false) {
> tevent_glib_prepare(glue);
> }
>}
>
>or
>
> if (point == TEVENT_TRACE_AFTER_LOOP_ONCE &&
> glue->skip_glib_refresh == false) {
> tevent_glib_prepare(glue);
> }
>
>is much simpler to read then
>
> bool do_glib_refresh = false;
>
> if (point == TEVENT_TRACE_AFTER_LOOP_ONCE) {
> do_glib_refresh = true;
> }
> if (glue->skip_glib_refresh) {
> /*
> * Don't call tevent_glib_prepare() if explicity told so. This
> * is an optimisation for the case that glib event sources are
> * created from glib event callbacks.
> */
> do_glib_refresh = false;
> }
> if (do_glib_refresh) {
> tevent_glib_prepare(glue);
> }
>
>Also I'd feel safer if glue->skip_glib_refresh was reset inside the 'if
>(point == TEVENT_TRACE_AFTER_LOOP_ONCE) ' check, maybe I am being
>paranoid but I am not sure for example what way the trace event order
>might be if for example the discouraged (but still used) nested tevent
>processing happens
I'm a review whore, changed the check to match your preferred style. :) I've
also added paranoia bit, I think you're right about the nested tevent loop.
>I had a look at the remainder mdssvc changes so
>
>[PATCH 7/9] s3-mdssvc: add tevent context arg to mds_init_ctx
>
>[PATCH 8/9] s3-mdssvc: call [un]become_authenticated_pipe_user()
>
>af as I understand the changes these look good, the
>'become_authenticated_pipe_user' would have been necessary anyway
>regardless of the glue changes. right ?
Yup.
>[[PATCH 9/9] s3-mdssvc: use tevent_glib_glue in mdssvc RPC service
>
>These all look fine to me except I confess that I don't understand the
>reason for using g_main_context_new() rather than the default context
>and then additional use of g_main_context_push_thread_default() From
>what I read I understood this is something you may want to do in a
>multithreaded environment to ensure gio threads for example to deliver
>events to thread default context (and associated main loop) rather than
>the global default context. If we are not multitheaded should we just
>not use the default global context anyway (and avoid
>g_main_context_push/pop_thread_default context calls) To be honest this
>stuff is pretty unfamiliar to me so I actually have no clue :-)
Same here. Iirc it was needed because otherwise libtracker-sparql wouldn't
dispatch the callbacks in the right context. Maybe it was a bug, it's woring now
with the default gmain context. Fixed in attached patchset (extra patch).
I've added a few more small mdssvc related patches on-top of the patchset.
Please review&push if happy. Thanks!
-slow
--
Ralph Boehme, Samba Team https://samba.org/
Samba Developer, SerNet GmbH https://sernet.de/en/samba/
GPG-Fingerprint FAE2C6088A24252051C559E4AA1E9B7126399E46
-------------- next part --------------
From 063df0b0e74f06c8dca88d51f92992bc754511c2 Mon Sep 17 00:00:00 2001
From: Ralph Boehme <slow at samba.org>
Date: Mon, 15 Feb 2016 10:42:52 +0100
Subject: [PATCH 01/11] s3: build: seperate out check for Gnome Tracker from
Spotlight
Signed-off-by: Ralph Boehme <slow at samba.org>
---
source3/wscript | 54 ++++++++++++++++++++++++++++++-------------------
1 file changed, 33 insertions(+), 21 deletions(-)
diff --git a/source3/wscript b/source3/wscript
index e0db9839795..2bc10b3d23e 100644
--- a/source3/wscript
+++ b/source3/wscript
@@ -1623,35 +1623,47 @@ main() {
conf.fatal('AFS headers not available, but --with-fake-kaserver was specified')
conf.env['libtracker']=''
+ tracker_versions = ['2.0', '1.0', '0.16', '0.14']
+
+ for version in tracker_versions:
+ testlib = 'tracker-sparql-' + version
+ if conf.CHECK_CFG(package=testlib,
+ args='--cflags --libs',
+ mandatory=False):
+ conf.SET_TARGET_TYPE(testlib, 'SYSLIB')
+ conf.env['libtracker'] = testlib
+ conf.DEFINE('HAVE_TRACKER', '1')
+ break
+
+ Logs.info("Checking for bison")
+ bison.configure(conf)
+ conf.CHECK_COMMAND('%s --version | head -n1' % conf.env['BISON'],
+ msg='Using bison version',
+ define=None,
+ on_target=False)
+
+ Logs.info("Checking for flex")
+ flex.configure(conf)
+ conf.CHECK_COMMAND('%s --version' % conf.env['FLEX'],
+ msg='Using flex version',
+ define=None,
+ on_target=False)
+
conf.env.with_spotlight = False
if Options.options.with_spotlight:
-
- Logs.info("Requested Spotlight support, checking for bison")
- bison.configure(conf)
if not conf.env['BISON']:
conf.fatal("Spotlight support requested but bison missing")
- conf.CHECK_COMMAND('%s --version | head -n1' % conf.env['BISON'], msg='Using bison version', define=None, on_target=False)
- Logs.info("Requested Spotlight support, checking for flex")
- flex.configure(conf)
+
if not conf.env['FLEX']:
conf.fatal("Spotlight support requested but flex missing")
- conf.CHECK_COMMAND('%s --version' % conf.env['FLEX'], msg='Using flex version', define=None, on_target=False)
- versions = ['2.0', '1.0', '0.16', '0.14']
- for version in versions:
- testlib = 'tracker-sparql-' + version
- if conf.CHECK_CFG(package=testlib,
- args='--cflags --libs',
- mandatory=False):
- conf.SET_TARGET_TYPE(testlib, 'SYSLIB')
- conf.env['libtracker'] = testlib
- conf.env.with_spotlight = True
- conf.DEFINE('WITH_SPOTLIGHT', '1')
- break
-
- if not conf.env.with_spotlight:
- conf.fatal("Spotlight support requested but tracker-sparql library missing")
+
+ if not conf.CONFIG_SET('HAVE_TRACKER'):
+ conf.fatal('Missing Gnome Tracker development files')
+
Logs.info("building with Spotlight support")
default_static_modules.extend(TO_LIST('rpc_mdssvc_module'))
+ conf.DEFINE('WITH_SPOTLIGHT', '1')
+ conf.env.with_spotlight = True
conf.CHECK_HEADERS('rpc/xdr.h', lib='tirpc')
--
2.17.2
From a7d4a535a0c190b5691b1af871bb3c5f9c297afa Mon Sep 17 00:00:00 2001
From: Ralph Boehme <slow at samba.org>
Date: Wed, 20 Jan 2016 15:08:31 +0100
Subject: [PATCH 02/11] s3/lib: new tevent_glib_glue subsystem
tevent_glib_glue_create() takes glib GMainContext and adds its event
sources to a tevent context. tevent will poll the sources and run
handlers for pending events as detailed in the glib documentation:
https://developer.gnome.org/glib/stable/glib-The-Main-Event-Loop.html
Signed-off-by: Ralph Boehme <slow at samba.org>
---
source3/lib/tevent_glib_glue.c | 875 +++++++++++++++++++++++++++++++++
source3/lib/tevent_glib_glue.h | 68 +++
source3/wscript | 13 +
source3/wscript_build | 6 +
4 files changed, 962 insertions(+)
create mode 100644 source3/lib/tevent_glib_glue.c
create mode 100644 source3/lib/tevent_glib_glue.h
diff --git a/source3/lib/tevent_glib_glue.c b/source3/lib/tevent_glib_glue.c
new file mode 100644
index 00000000000..73593b56805
--- /dev/null
+++ b/source3/lib/tevent_glib_glue.c
@@ -0,0 +1,875 @@
+/*
+ Unix SMB/CIFS implementation.
+ Integration of a glib g_main_context into a tevent_context
+ Copyright (C) Stefan Metzmacher 2016
+ Copyright (C) Ralph Boehme 2016
+
+ ** NOTE! The following LGPL license applies to the tevent
+ ** library. This does NOT imply that all of Samba is released
+ ** under the LGPL
+
+ This library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 3 of the License, or (at your option) any later version.
+
+ This library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with this library; if not, see <http://www.gnu.org/licenses/>.
+*/
+
+#include "replace.h"
+#include "system/filesys.h"
+#include "lib/util/debug.h"
+#include "lib/util/select.h"
+#include <tevent.h>
+
+#undef DBGC_CLASS
+#define DBGC_CLASS DBGC_TEVENT
+
+#ifdef HAVE_GLIB
+#include <glib.h>
+#include "tevent_glib_glue.h"
+
+struct fd_map {
+ struct tevent_glib_glue *glue;
+ int fd;
+ struct tevent_fd *fd_event;
+};
+
+struct tevent_glib_glue {
+ /*
+ * The tevent context we're feeding.
+ */
+ struct tevent_context *ev;
+
+ /*
+ * The glib gmain context we're polling and whether we're currently
+ * owning it by virtue of g_main_context_acquire().
+ */
+ GMainContext *gmain_ctx;
+ bool gmain_owner;
+
+ /*
+ * Set by samba_tevent_glib_glue_quit().
+ */
+ bool quit;
+
+ /*
+ * tevent trace callback and data we got from tevent_get_trace_callback()
+ * before installing our own trace callback.
+ */
+ tevent_trace_callback_t prev_tevent_trace_cb;
+ void *prev_tevent_trace_data;
+
+ /*
+ * Don't call tevent_glib_prepare() in the tevent tracepoint handler if
+ * explicity told so. This is an optimisation for the case that glib
+ * event sources are created from glib event callbacks.
+ */
+ bool skip_glib_refresh;
+
+ /*
+ * Used when acquiring the glib gmain context failed.
+ */
+ struct tevent_timer *acquire_retry_timer;
+
+ /*
+ * glib gmain context timeout and priority for the current event look
+ * iteration. gtimeout is translated to a tevent timer event, unless it
+ * is 0 which signals some event source is pending. In that case we
+ * dispatch an immediate event. gpriority is ignored by us, just passed
+ * to the glib relevant functions.
+ */
+ gint gtimeout;
+ gint gpriority;
+ struct tevent_timer *timer;
+ struct tevent_immediate *im;
+ bool scheduled_im;
+
+ /*
+ * glib gmain context fds returned from g_main_context_query(). These
+ * get translated to tevent fd events.
+ */
+ GPollFD *gpollfds;
+ gint num_gpollfds;
+
+ /*
+ * A copy of gpollfds and num_gpollfds from the previous event loop
+ * iteration, used to detect changes in the set of fds.
+ */
+ GPollFD *prev_gpollfds;
+ gint num_prev_gpollfds;
+
+ /*
+ * An array of pointers to fd_map's. The fd_map'd contain the tevent
+ * event fd as well as a pointer to the corresponding glib GPollFD.
+ */
+ struct fd_map **fd_map;
+ size_t num_maps;
+};
+
+static bool tevent_glib_prepare(struct tevent_glib_glue *glue);
+static bool tevent_glib_process(struct tevent_glib_glue *glue);
+static bool tevent_glib_glue_reinit(struct tevent_glib_glue *glue);
+static void tevent_glib_fd_handler(struct tevent_context *ev,
+ struct tevent_fd *fde,
+ uint16_t flags,
+ void *private_data);
+
+typedef int (*gfds_cmp_cb)(const void *fd1, const void *fd2);
+typedef bool (*gfds_found_cb)(struct tevent_glib_glue *glue,
+ const GPollFD *new,
+ const GPollFD *old);
+typedef bool (*gfds_new_cb)(struct tevent_glib_glue *glue,
+ const GPollFD *fd);
+typedef bool (*gfds_removed_cb)(struct tevent_glib_glue *glue,
+ const GPollFD *fd);
+
+/**
+ * Compare two sorted GPollFD arrays
+ *
+ * For every element that exists in gfds and prev_gfds found_fn() is called.
+ * For every element in gfds but not in prev_gfds, new_fn() is called.
+ * For every element in prev_gfds but not in gfds removed_fn() is called.
+ **/
+static bool cmp_gfds(struct tevent_glib_glue *glue,
+ GPollFD *gfds,
+ GPollFD *prev_gfds,
+ size_t num_gfds,
+ size_t num_prev_gfds,
+ gfds_cmp_cb cmp_cb,
+ gfds_found_cb found_cb,
+ gfds_new_cb new_cb,
+ gfds_removed_cb removed_cb)
+{
+ bool ok;
+ size_t i = 0, j = 0;
+ int cmp;
+
+ while (i < num_gfds && j < num_prev_gfds) {
+ cmp = cmp_cb(&gfds[i], &prev_gfds[j]);
+ if (cmp == 0) {
+ ok = found_cb(glue, &gfds[i], &prev_gfds[j]);
+ if (!ok) {
+ return false;
+ }
+ i++;
+ j++;
+ } else if (cmp < 0) {
+ ok = new_cb(glue, &gfds[i]);
+ if (!ok) {
+ return false;
+ }
+ i++;
+ } else {
+ ok = removed_cb(glue, &prev_gfds[j]);
+ if (!ok) {
+ return false;
+ }
+ j++;
+ }
+ }
+
+ while (i < num_gfds) {
+ ok = new_cb(glue, &gfds[i++]);
+ if (!ok) {
+ return false;
+ }
+ }
+
+ while (j < num_prev_gfds) {
+ ok = removed_cb(glue, &prev_gfds[j++]);
+ if (!ok) {
+ return false;
+ }
+ }
+
+ return true;
+}
+
+static int glib_fd_cmp_func(const void *p1, const void *p2)
+{
+ const GPollFD *lhs = p1;
+ const GPollFD *rhs = p2;
+
+ if (lhs->fd < rhs->fd) {
+ return -1;
+ } else if (lhs->fd > rhs->fd) {
+ return 1;
+ }
+
+ return 0;
+}
+
+/*
+ * We already have a tevent fd event for the glib GPollFD, but we may have to
+ * update flags.
+ */
+static bool match_gfd_cb(struct tevent_glib_glue *glue,
+ const GPollFD *new_gfd,
+ const GPollFD *old_gfd)
+{
+ size_t i;
+ struct fd_map *fd_map = NULL;
+ struct tevent_fd *fd_event = NULL;
+
+ if (new_gfd->events == old_gfd->events) {
+ return true;
+ }
+
+ for (i = 0; i < glue->num_maps; i++) {
+ if (glue->fd_map[i]->fd == new_gfd->fd) {
+ break;
+ }
+ }
+
+ if (i == glue->num_maps) {
+ DBG_ERR("match_gfd_cb: glib fd %d not in map\n", new_gfd->fd);
+ return false;
+ }
+
+ fd_map = glue->fd_map[i];
+ if (fd_map == NULL) {
+ DBG_ERR("fd_map for fd %d is NULL\n", new_gfd->fd);
+ return false;
+ }
+
+ fd_event = fd_map->fd_event;
+ if (fd_event == NULL) {
+ DBG_ERR("fd_event for fd %d is NULL\n", new_gfd->fd);
+ return false;
+ }
+
+ tevent_fd_set_flags(fd_event, 0);
+
+ if (new_gfd->events & (G_IO_IN | G_IO_HUP | G_IO_ERR)) {
+ TEVENT_FD_READABLE(fd_event);
+ }
+ if (new_gfd->events & G_IO_OUT) {
+ TEVENT_FD_WRITEABLE(fd_event);
+ }
+
+ return true;
+}
+
+static bool new_gfd_cb(struct tevent_glib_glue *glue, const GPollFD *gfd)
+{
+ struct tevent_fd *fd_event = NULL;
+ struct fd_map *fd_map = NULL;
+ uint16_t events = 0;
+ bool revent;
+ bool wevent;
+
+ revent = (gfd->events & (G_IO_IN | G_IO_HUP | G_IO_ERR));
+ wevent = (gfd->events & G_IO_OUT);
+ if (revent) {
+ events |= TEVENT_FD_READ;
+ }
+ if (wevent) {
+ events |= TEVENT_FD_WRITE;
+ }
+
+ glue->fd_map = talloc_realloc(glue,
+ glue->fd_map,
+ struct fd_map *,
+ glue->num_maps + 1);
+ if (glue->fd_map == NULL) {
+ DBG_ERR("talloc_realloc failed\n");
+ return false;
+ }
+ fd_map = talloc_zero(glue->fd_map, struct fd_map);
+ if (fd_map == NULL) {
+ DBG_ERR("talloc_realloc failed\n");
+ return false;
+ }
+ glue->fd_map[glue->num_maps] = fd_map;
+ glue->num_maps++;
+
+ fd_event = tevent_add_fd(glue->ev,
+ glue->fd_map,
+ gfd->fd,
+ events,
+ tevent_glib_fd_handler,
+ fd_map);
+ if (fd_event == NULL) {
+ DBG_ERR("tevent_add_fd failed\n");
+ return false;
+ }
+
+ *fd_map = (struct fd_map) {
+ .glue = glue,
+ .fd = gfd->fd,
+ .fd_event = fd_event,
+ };
+
+ DBG_DEBUG("added tevent_fd for glib fd %d\n", gfd->fd);
+
+ return true;
+}
+
+static bool remove_gfd_cb(struct tevent_glib_glue *glue, const GPollFD *gfd)
+{
+ size_t i;
+
+ for (i = 0; i < glue->num_maps; i++) {
+ if (glue->fd_map[i]->fd == gfd->fd) {
+ break;
+ }
+ }
+
+ if (i == glue->num_maps) {
+ DBG_ERR("remove_gfd_cb: glib fd %d not in map\n", gfd->fd);
+ return false;
+ }
+
+ TALLOC_FREE(glue->fd_map[i]->fd_event);
+ TALLOC_FREE(glue->fd_map[i]);
+
+ if (i + 1 < glue->num_maps) {
+ memmove(&glue->fd_map[i],
+ &glue->fd_map[i+1],
+ (glue->num_maps - (i + 1)) * sizeof(struct fd_map *));
+ }
+
+ glue->fd_map = talloc_realloc(glue,
+ glue->fd_map,
+ struct fd_map *,
+ glue->num_maps - 1);
+ if (glue->num_maps > 0 && glue->fd_map == NULL) {
+ DBG_ERR("talloc_realloc failed\n");
+ return false;
+ }
+ glue->num_maps--;
+
+ return true;
+}
+
+static short gpoll_to_poll_event(gushort gevent)
+{
+ short pevent = 0;
+
+ if (gevent & G_IO_IN) {
+ pevent |= POLLIN;
+ }
+ if (gevent & G_IO_OUT) {
+ pevent |= POLLOUT;
+ }
+ if (gevent & G_IO_HUP) {
+ pevent |= POLLHUP;
+ }
+ if (gevent & G_IO_ERR) {
+ pevent |= POLLERR;
+ }
+
+ return pevent;
+}
+
+static gushort poll_to_gpoll_event(short pevent)
+{
+ gushort gevent = 0;
+
+ if (pevent & POLLIN) {
+ gevent |= G_IO_IN;
+ }
+ if (pevent & POLLOUT) {
+ gevent |= G_IO_OUT;
+ }
+ if (pevent & POLLHUP) {
+ gevent |= G_IO_HUP;
+ }
+ if (pevent & POLLERR) {
+ gevent |= G_IO_ERR;
+ }
+
+ return gevent;
+}
+
+static void tevent_glib_fd_handler(struct tevent_context *ev,
+ struct tevent_fd *fde,
+ uint16_t flags,
+ void *private_data)
+{
+ struct fd_map *fd_map = talloc_get_type_abort(
+ private_data, struct fd_map);
+ struct tevent_glib_glue *glue = NULL;
+ GPollFD *gpollfd = NULL;
+ struct pollfd fd;
+ int ret;
+ int i;
+
+ glue = fd_map->glue;
+
+ for (i = 0; i < glue->num_gpollfds; i++) {
+ if (glue->gpollfds[i].fd != fd_map->fd) {
+ continue;
+ }
+ gpollfd = &glue->gpollfds[i];
+ break;
+ }
+ if (gpollfd == NULL) {
+ DBG_ERR("No gpollfd for fd_map [%p] fd [%d]\n",
+ fd_map, fd_map->fd);
+ return;
+ }
+ /*
+ * We have to poll() the fd to get the correct fd event for glib. tevent
+ * only tells us about readable/writable in flags, but we need the full
+ * glory for glib.
+ */
+
+ fd = (struct pollfd) {
+ .fd = gpollfd->fd,
+ .events = gpoll_to_poll_event(gpollfd->events),
+ };
+
+ ret = sys_poll_intr(&fd, 1, 0);
+ if (ret == -1) {
+ DBG_ERR("poll: %s\n", strerror(errno));
+ return;
+ }
+ if (ret == 0) {
+ return;
+ }
+
+ gpollfd->revents = poll_to_gpoll_event(fd.revents);
+
+ tevent_glib_process(glue);
+ return;
+}
+
+static void tevent_glib_timer_handler(struct tevent_context *ev,
+ struct tevent_timer *te,
+ struct timeval current_time,
+ void *private_data)
+{
+ struct tevent_glib_glue *glue = talloc_get_type_abort(
+ private_data, struct tevent_glib_glue);
+
+ glue->timer = NULL;
+ tevent_glib_process(glue);
+ return;
+}
+
+static void tevent_glib_im_handler(struct tevent_context *ev,
+ struct tevent_immediate *im,
+ void *private_data)
+{
+ struct tevent_glib_glue *glue = talloc_get_type_abort(
+ private_data, struct tevent_glib_glue);
+
+ glue->scheduled_im = false;
+ tevent_glib_process(glue);
+ return;
+}
+
+static bool save_current_fdset(struct tevent_glib_glue *glue)
+{
+ /*
+ * Save old glib fds. We only grow the prev array.
+ */
+
+ if (glue->num_prev_gpollfds < glue->num_gpollfds) {
+ glue->prev_gpollfds = talloc_realloc(glue,
+ glue->prev_gpollfds,
+ GPollFD,
+ glue->num_gpollfds);
+ if (glue->prev_gpollfds == NULL) {
+ DBG_ERR("talloc_realloc failed\n");
+ return false;
+ }
+ }
+ glue->num_prev_gpollfds = glue->num_gpollfds;
+ if (glue->num_gpollfds > 0) {
+ memcpy(glue->prev_gpollfds, glue->gpollfds,
+ sizeof(GPollFD) * glue->num_gpollfds);
+ memset(glue->gpollfds, 0, sizeof(GPollFD) * glue->num_gpollfds);
+ }
+
+ return true;
+}
+
+static bool get_glib_fds_and_timeout(struct tevent_glib_glue *glue)
+{
+ bool ok;
+ gint num_fds;
+
+ ok = save_current_fdset(glue);
+ if (!ok) {
+ return false;
+ }
+
+ while (true) {
+ num_fds = g_main_context_query(glue->gmain_ctx,
+ glue->gpriority,
+ &glue->gtimeout,
+ glue->gpollfds,
+ glue->num_gpollfds);
+ if (num_fds == glue->num_gpollfds) {
+ break;
+ }
+ glue->gpollfds = talloc_realloc(glue,
+ glue->gpollfds,
+ GPollFD,
+ num_fds);
+ if (num_fds > 0 && glue->gpollfds == NULL) {
+ DBG_ERR("talloc_realloc failed\n");
+ return false;
+ }
+ glue->num_gpollfds = num_fds;
+ };
+
+ if (glue->num_gpollfds > 0) {
+ qsort(glue->gpollfds,
+ num_fds,
+ sizeof(GPollFD),
+ glib_fd_cmp_func);
+ }
+
+ DBG_DEBUG("num fds: %d, timeout: %d ms\n",
+ num_fds, glue->gtimeout);
+
+ return true;
+}
+
+static bool tevent_glib_update_events(struct tevent_glib_glue *glue)
+{
+ uint64_t microsec;
+ struct timeval tv;
+ bool ok;
+
+ ok = cmp_gfds(glue,
+ glue->gpollfds,
+ glue->prev_gpollfds,
+ glue->num_gpollfds,
+ glue->num_prev_gpollfds,
+ glib_fd_cmp_func,
+ match_gfd_cb,
+ new_gfd_cb,
+ remove_gfd_cb);
+ if (!ok) {
+ return false;
+ }
+
+ TALLOC_FREE(glue->timer);
+
+ if (glue->gtimeout == -1) {
+ return true;
+ }
+
+ if (glue->gtimeout == 0) {
+ /*
+ * glue->gtimeout is 0 if g_main_context_query() returned
+ * timeout=0. That means there are pending events ready to be
+ * dispatched. We only want to run one event handler per loop
+ * iteration, so we schedule an immediate event to run it in the
+ * next iteration.
+ */
+ if (glue->scheduled_im) {
+ return true;
+ }
+ tevent_schedule_immediate(glue->im,
+ glue->ev,
+ tevent_glib_im_handler,
+ glue);
+ glue->scheduled_im = true;
+ return true;
+ }
+
+ microsec = glue->gtimeout * 1000;
+ tv = tevent_timeval_current_ofs(microsec / 1000000,
+ microsec % 1000000);
+
+ glue->timer = tevent_add_timer(glue->ev,
+ glue,
+ tv,
+ tevent_glib_timer_handler,
+ glue);
+ if (glue->timer == NULL) {
+ DBG_ERR("tevent_add_timer failed\n");
+ return false;
+ }
+
+ return true;
+}
+
+static void tevent_glib_retry_timer(struct tevent_context *ev,
+ struct tevent_timer *te,
+ struct timeval current_time,
+ void *private_data)
+{
+ struct tevent_glib_glue *glue = talloc_get_type_abort(
+ private_data, struct tevent_glib_glue);
+
+ glue->acquire_retry_timer = NULL;
+ (void)tevent_glib_prepare(glue);
+}
+
+/**
+ * Fetch glib event sources and add them to tevent
+ *
+ * Fetch glib event sources and attach corresponding tevent events to our tevent
+ * context. get_glib_fds_and_timeout() gets the relevant glib event sources: the
+ * set of active fds and the next timer. tevent_glib_update_events() then
+ * translates those to tevent and creates tevent events.
+ *
+ * When called, the thread must NOT be the owner to the glib main
+ * context. tevent_glib_prepare() is either the first function when the
+ * tevent_glib_glue is created, or after tevent_glib_process() has been called
+ * to process pending event, which will have ceased ownership.
+ **/
+static bool tevent_glib_prepare(struct tevent_glib_glue *glue)
+{
+ bool ok;
+ gboolean gok;
+
+ if (glue->quit) {
+ /* Set via samba_tevent_glib_glue_quit() */
+ return true;
+ }
+
+ if (glue->acquire_retry_timer != NULL) {
+ /*
+ * We're still waiting on the below g_main_context_acquire() to
+ * succeed, just return.
+ */
+ return true;
+ }
+
+ if (glue->gmain_owner) {
+ g_main_context_release(glue->gmain_ctx);
+ glue->gmain_owner = false;
+ }
+
+ gok = g_main_context_acquire(glue->gmain_ctx);
+ if (!gok) {
+ DBG_ERR("couldn't acquire g_main_context\n");
+
+ /*
+ * Ensure no tevent event fires while we're not the gmain
+ * context owner. The event handler would call
+ * tevent_glib_process() and that expects being the owner of the
+ * context.
+ */
+ ok = tevent_glib_glue_reinit(glue);
+ if (!ok) {
+ DBG_ERR("tevent_glib_glue_reinit failed\n");
+ samba_tevent_glib_glue_quit(glue);
+ return false;
+ }
+
+ glue->acquire_retry_timer = tevent_add_timer(
+ glue->ev,
+ glue,
+ tevent_timeval_current_ofs(0, 1000),
+ tevent_glib_retry_timer,
+ glue);
+ if (glue->acquire_retry_timer == NULL) {
+ DBG_ERR("tevent_add_timer failed\n");
+ samba_tevent_glib_glue_quit(glue);
+ return false;
+ }
+ return true;
+ }
+ glue->gmain_owner = true;
+
+ /*
+ * Discard "ready" return value from g_main_context_prepare(). We don't
+ * want to dispatch events here, thats only done in from the tevent loop.
+ */
+ (void)g_main_context_prepare(glue->gmain_ctx, &glue->gpriority);
+
+ ok = get_glib_fds_and_timeout(glue);
+ if (!ok) {
+ DBG_ERR("get_glib_fds_and_timeout failed\n");
+ samba_tevent_glib_glue_quit(glue);
+ return false;
+ }
+
+ ok = tevent_glib_update_events(glue);
+ if (!ok) {
+ DBG_ERR("tevent_glib_update_events failed\n");
+ samba_tevent_glib_glue_quit(glue);
+ return false;
+ }
+
+ return true;
+}
+
+/**
+ * Process pending glib events
+ *
+ * tevent_glib_process() gets called to process pending glib events via
+ * g_main_context_check() and then g_main_context_dispatch().
+ *
+ * After pending event handlers are dispatched, we rearm the glib glue event
+ * handlers in tevent by calling tevent_glib_prepare().
+ *
+ * When tevent_glib_process() is called the thread must own the glib
+ * gmain_ctx. That is achieved by tevent_glib_prepare() being the only function
+ * that acuires context ownership.
+ *
+ * To give other threads that are blocked on g_main_context_acquire(gmain_ctx) a
+ * chance to acquire context ownership (eg needed to attach event sources), we
+ * release context ownership before calling tevent_glib_prepare() which will
+ * acquire it again.
+ */
+static bool tevent_glib_process(struct tevent_glib_glue *glue)
+{
+ bool ok;
+ bool sources_ready;
+
+ DBG_DEBUG("tevent_glib_process\n");
+
+ sources_ready = g_main_context_check(glue->gmain_ctx,
+ glue->gpriority,
+ glue->gpollfds,
+ glue->num_gpollfds);
+ if (sources_ready) {
+ g_main_context_dispatch(glue->gmain_ctx);
+ }
+
+ ok = tevent_glib_prepare(glue);
+ if (!ok) {
+ return false;
+ }
+ glue->skip_glib_refresh = true;
+ return true;
+}
+
+static void tevent_glib_glue_trace_callback(enum tevent_trace_point point,
+ void *private_data)
+{
+ struct tevent_glib_glue *glue = talloc_get_type_abort(
+ private_data, struct tevent_glib_glue);
+
+ if (point == TEVENT_TRACE_AFTER_LOOP_ONCE &&
+ glue->skip_glib_refresh == false)
+ {
+ tevent_glib_prepare(glue);
+ }
+
+ /* chain previous handler */
+ if (glue->prev_tevent_trace_cb != NULL) {
+ glue->prev_tevent_trace_cb(point, glue->prev_tevent_trace_data);
+ }
+}
+
+static void tevent_glib_glue_cleanup(struct tevent_glib_glue *glue)
+{
+ size_t n = talloc_array_length(glue->fd_map);
+ size_t i;
+
+ for (i = 0; i < n; i++) {
+ TALLOC_FREE(glue->fd_map[i]->fd_event);
+ TALLOC_FREE(glue->fd_map[i]);
+ }
+
+ tevent_set_trace_callback(glue->ev,
+ glue->prev_tevent_trace_cb,
+ glue->prev_tevent_trace_data);
+ glue->prev_tevent_trace_cb = NULL;
+ glue->prev_tevent_trace_data = NULL;
+
+ TALLOC_FREE(glue->fd_map);
+ glue->num_maps = 0;
+
+ TALLOC_FREE(glue->gpollfds);
+ glue->num_gpollfds = 0;
+
+ TALLOC_FREE(glue->prev_gpollfds);
+ glue->num_prev_gpollfds = 0;
+
+ TALLOC_FREE(glue->timer);
+ TALLOC_FREE(glue->acquire_retry_timer);
+ TALLOC_FREE(glue->im);
+
+ /*
+ * These are not really needed, but let's wipe the slate clean.
+ */
+ glue->skip_glib_refresh = false;
+ glue->gtimeout = 0;
+ glue->gpriority = 0;
+}
+
+static bool tevent_glib_glue_reinit(struct tevent_glib_glue *glue)
+{
+ tevent_glib_glue_cleanup(glue);
+
+ glue->im = tevent_create_immediate(glue);
+ if (glue->im == NULL) {
+ return false;
+ }
+
+ tevent_get_trace_callback(glue->ev,
+ &glue->prev_tevent_trace_cb,
+ &glue->prev_tevent_trace_data);
+ tevent_set_trace_callback(glue->ev,
+ tevent_glib_glue_trace_callback,
+ glue);
+
+ return true;
+}
+
+void samba_tevent_glib_glue_quit(struct tevent_glib_glue *glue)
+{
+ tevent_glib_glue_cleanup(glue);
+ glue->quit = true;
+ return;
+}
+
+struct tevent_glib_glue *samba_tevent_glib_glue_create(TALLOC_CTX *mem_ctx,
+ struct tevent_context *ev,
+ GMainContext *gmain_ctx)
+{
+ bool ok;
+ struct tevent_glib_glue *glue = NULL;
+
+ glue = talloc_zero(mem_ctx, struct tevent_glib_glue);
+ if (glue == NULL) {
+ DBG_ERR("talloc_zero failed\n");
+ return NULL;
+ }
+
+ *glue = (struct tevent_glib_glue) {
+ .ev = ev,
+ .gmain_ctx = gmain_ctx,
+ };
+
+ glue->im = tevent_create_immediate(glue);
+
+ tevent_get_trace_callback(glue->ev,
+ &glue->prev_tevent_trace_cb,
+ &glue->prev_tevent_trace_data);
+ tevent_set_trace_callback(glue->ev,
+ tevent_glib_glue_trace_callback,
+ glue);
+
+ ok = tevent_glib_prepare(glue);
+ if (!ok) {
+ TALLOC_FREE(glue);
+ return NULL;
+ }
+
+ return glue;
+}
+
+#else /* HAVE_GLIB */
+
+struct tevent_glib_glue *samba_tevent_glib_glue_create(TALLOC_CTX *mem_ctx,
+ struct tevent_context *ev,
+ GMainContext *gmain_ctx)
+{
+ errno = ENOSYS;
+ return NULL;
+}
+
+void samba_tevent_glib_glue_quit(struct tevent_glib_glue *glue)
+{
+ return;
+}
+#endif /* HAVE_GLIB */
diff --git a/source3/lib/tevent_glib_glue.h b/source3/lib/tevent_glib_glue.h
new file mode 100644
index 00000000000..0d001fa8ce9
--- /dev/null
+++ b/source3/lib/tevent_glib_glue.h
@@ -0,0 +1,68 @@
+/*
+ Unix SMB/CIFS implementation.
+ Poll glib event loop from tevent
+
+ Copyright (C) Ralph Boehme 2016
+
+ ** NOTE! The following LGPL license applies to the tevent
+ ** library. This does NOT imply that all of Samba is released
+ ** under the LGPL
+
+ This library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 3 of the License, or (at your option) any later version.
+
+ This library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with this library; if not, see <http://www.gnu.org/licenses/>.
+*/
+
+#ifndef _TEVENT_GLIB_GLUE_H
+#define _TEVENT_GLIB_GLUE_H
+
+#include <talloc.h>
+#include <tevent.h>
+
+/**
+ * @brief Add a glib GmainContext to a tevent context
+ *
+ * tevent will poll the glib event sources and run handlers for
+ * pending events as detailed in the glib documentation:
+ *
+ * https://developer.gnome.org/glib/stable/glib-The-Main-Event-Loop.html
+ *
+ * If tevent was built without glib support, this function will always return
+ * NULL with an error number ENOSYS.
+ *
+ * @param[in] mem_ctx Memory context to use
+ *
+ * @param[in] ev Event context to use
+ *
+ * @param[in] gmain_ctx GMainContext that will be added to tevent
+ *
+ * @return A handle on the glue context that binds the
+ * the GMainContext to tevent. Pass the glue handle to
+ * tevent_glib_glue_quit() in a callback when you want
+ * stop processing glib events.
+ * You must not call talloc_free() on the handle while
+ * the loop is still in use and attached to tevent.
+ */
+struct tevent_glib_glue *samba_tevent_glib_glue_create(TALLOC_CTX *mem_ctx,
+ struct tevent_context *ev,
+ GMainContext *gmain_ctx);
+
+/**
+ * @brief Stop polling a GMainContext
+ *
+ * Used in a callback when you want to stop processing glib events.
+ *
+ * @param[in] glue And tevent_glib_glue handle
+ */
+void samba_tevent_glib_glue_quit(struct tevent_glib_glue *glue);
+
+#endif
diff --git a/source3/wscript b/source3/wscript
index 2bc10b3d23e..2a585966d90 100644
--- a/source3/wscript
+++ b/source3/wscript
@@ -1622,6 +1622,16 @@ main() {
else:
conf.fatal('AFS headers not available, but --with-fake-kaserver was specified')
+ if conf.CHECK_CFG(package='glib-2.0',
+ args='--cflags --libs',
+ msg='Checking for glib-2.0',
+ uselib_store="GLIB-2.0"):
+ if (conf.CHECK_HEADERS('glib.h', lib='glib-2.0') and conf.CHECK_LIB('glib-2.0', shlib=True)):
+ conf.DEFINE('HAVE_GLIB', 1)
+
+ if conf.CONFIG_SET('HAVE_GLIB'):
+ conf.DEFINE('WITH_TEVENT_GLIB_GLUE', '1')
+
conf.env['libtracker']=''
tracker_versions = ['2.0', '1.0', '0.16', '0.14']
@@ -1660,6 +1670,9 @@ main() {
if not conf.CONFIG_SET('HAVE_TRACKER'):
conf.fatal('Missing Gnome Tracker development files')
+ if not conf.CONFIG_SET('HAVE_GLIB'):
+ conf.fatal('Missing glib-2.0 development files')
+
Logs.info("building with Spotlight support")
default_static_modules.extend(TO_LIST('rpc_mdssvc_module'))
conf.DEFINE('WITH_SPOTLIGHT', '1')
diff --git a/source3/wscript_build b/source3/wscript_build
index ed4de978fdc..c28b3e7621c 100644
--- a/source3/wscript_build
+++ b/source3/wscript_build
@@ -1073,6 +1073,12 @@ bld.SAMBA3_SUBSYSTEM('SPOOLSSD',
RPC_SOCK_HELPER
''')
+bld.SAMBA3_SUBSYSTEM('tevent-glib-glue',
+ source='lib/tevent_glib_glue.c',
+ deps='glib-2.0',
+ enabled=bld.CONFIG_SET('WITH_TEVENT_GLIB_GLUE'),
+)
+
########################## BINARIES #################################
bld.SAMBA3_BINARY('smbd/smbd',
--
2.17.2
From 7bd1dbbe455757d001c5c5bee0b4e0ba7ed76562 Mon Sep 17 00:00:00 2001
From: Ralph Boehme <slow at samba.org>
Date: Thu, 28 Jan 2016 08:29:28 +0100
Subject: [PATCH 03/11] s3/lib: add a tevent_glib_glue subsystem test
Tests adapted from glib2 glib/tests/mainloop.c.
Signed-off-by: Ralph Boehme <slow at samba.org>
---
source3/lib/tevent_glib_glue_tests.c | 397 ++++++++++++++++++
source3/script/tests/test_tevent_glib_glue.sh | 21 +
source3/selftest/tests.py | 3 +
source3/wscript_build | 10 +
4 files changed, 431 insertions(+)
create mode 100644 source3/lib/tevent_glib_glue_tests.c
create mode 100755 source3/script/tests/test_tevent_glib_glue.sh
diff --git a/source3/lib/tevent_glib_glue_tests.c b/source3/lib/tevent_glib_glue_tests.c
new file mode 100644
index 00000000000..d6cf66462f6
--- /dev/null
+++ b/source3/lib/tevent_glib_glue_tests.c
@@ -0,0 +1,397 @@
+/*
+ Unix SMB/CIFS implementation.
+
+ testing of the tevent glib glue subsystem
+
+ Copyright (C) Ralph Boehme 2016
+
+ glib tests adapted from glib2 glib/tests/mainloop.c
+ Copyright (C) 2011 Red Hat Inc., Matthias Clasen
+
+ ** NOTE! The following LGPL license applies to the tevent
+ ** library. This does NOT imply that all of Samba is released
+ ** under the LGPL
+
+ This library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 3 of the License, or (at your option) any later version.
+
+ This library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with this library; if not, see <http://www.gnu.org/licenses/>.
+*/
+
+#include "replace.h"
+
+/*
+ * glib uses TRUE and FALSE which may have redefined by "includes.h" to be
+ * unusable. Unndefine so glib can establish its own working replacement.
+ */
+#undef TRUE
+#undef FALSE
+#include <glib.h>
+#include <glib-unix.h>
+#include "lib/tevent_glib_glue.h"
+
+/*
+ * Unfortunately the glib test suite runner doesn't pass args to tests
+ * so we must keep a few globals here.
+ */
+static struct tevent_context *ev;
+
+static gboolean count_calls(gpointer data)
+{
+ gint *i = (gint *)data;
+
+ (*i)++;
+
+ return TRUE;
+}
+
+static gboolean quit_loop(gpointer data)
+{
+ struct tevent_glib_glue *glue = talloc_get_type_abort(
+ data, struct tevent_glib_glue);
+
+ samba_tevent_glib_glue_quit(glue);
+
+ return G_SOURCE_REMOVE;
+}
+
+static void test_timeouts(void)
+{
+ GMainContext *ctx = NULL;
+ struct tevent_glib_glue *glue = NULL;
+ GSource *source = NULL;
+ gint a;
+ gint b;
+ gint c;
+
+ a = b = c = 0;
+
+ ctx = g_main_context_new();
+ glue = samba_tevent_glib_glue_create(ev, ev, ctx);
+ g_assert(glue != NULL);
+
+ source = g_timeout_source_new(100);
+ g_source_set_callback(source, count_calls, &a, NULL);
+ g_source_attach(source, ctx);
+ g_source_unref(source);
+
+ source = g_timeout_source_new(250);
+ g_source_set_callback(source, count_calls, &b, NULL);
+ g_source_attach(source, ctx);
+ g_source_unref(source);
+
+ source = g_timeout_source_new(330);
+ g_source_set_callback(source, count_calls, &c, NULL);
+ g_source_attach(source, ctx);
+ g_source_unref(source);
+
+ source = g_timeout_source_new(1050);
+ g_source_set_callback(source, quit_loop, glue, NULL);
+ g_source_attach(source, ctx);
+ g_source_unref(source);
+
+ g_assert(tevent_loop_wait(ev) == 0);
+
+ /* We may be delayed for an arbitrary amount of time - for example,
+ * it's possible for all timeouts to fire exactly once.
+ */
+ g_assert_cmpint(a, >, 0);
+ g_assert_cmpint(a, >=, b);
+ g_assert_cmpint(b, >=, c);
+
+ g_assert_cmpint(a, <=, 10);
+ g_assert_cmpint(b, <=, 4);
+ g_assert_cmpint(c, <=, 3);
+
+ samba_tevent_glib_glue_quit(glue);
+ TALLOC_FREE(glue);
+ g_main_context_unref(ctx);
+}
+
+struct test_glib_ev_source_data {
+ GMainContext *ctx;
+ struct tevent_glib_glue *glue;
+};
+
+static gboolean test_glib_ev_source_quit_loop(gpointer data);
+
+static gboolean test_glib_ev_source_timeout_cb(gpointer data)
+{
+ struct test_glib_ev_source_data *state = talloc_get_type_abort(
+ data, struct test_glib_ev_source_data);
+ GSource *source = NULL;
+
+ source = g_timeout_source_new(100);
+ g_source_set_callback(source,
+ test_glib_ev_source_quit_loop,
+ state,
+ NULL);
+ g_source_attach(source, state->ctx);
+ g_source_unref(source);
+
+ return TRUE;
+}
+
+static gboolean test_glib_ev_source_quit_loop(gpointer data)
+{
+ struct test_glib_ev_source_data *state = talloc_get_type_abort(
+ data, struct test_glib_ev_source_data);
+
+ samba_tevent_glib_glue_quit(state->glue);
+
+ return G_SOURCE_REMOVE;
+}
+
+static void test_glib_ev_source(void)
+{
+ GMainContext *ctx = NULL;
+ struct tevent_glib_glue *glue = NULL;
+ struct test_glib_ev_source_data *state = NULL;
+ GSource *source = NULL;
+
+ ctx = g_main_context_new();
+ g_assert(ctx != NULL);
+
+ glue = samba_tevent_glib_glue_create(ev, ev, ctx);
+ g_assert(glue != NULL);
+
+ state = talloc_zero(glue, struct test_glib_ev_source_data);
+ g_assert(state != NULL);
+
+ state->ctx = ctx;
+ state->glue = glue;
+
+ source = g_timeout_source_new(100);
+ g_source_set_callback(source,
+ test_glib_ev_source_timeout_cb,
+ state,
+ NULL);
+ g_source_attach(source, ctx);
+ g_source_unref(source);
+
+ g_assert(tevent_loop_wait(ev) == 0);
+
+ TALLOC_FREE(glue);
+ g_main_context_unref(ctx);
+}
+
+struct test_tevent_ev_source_data {
+ GMainContext *ctx;
+ struct tevent_glib_glue *glue;
+};
+
+static gboolean test_tevent_ev_source_quit_loop(gpointer data);
+
+static void test_tevent_ev_source_timeout_cb(struct tevent_context *_ev,
+ struct tevent_timer *te,
+ struct timeval current_time,
+ void *data)
+{
+ struct test_tevent_ev_source_data *state = talloc_get_type_abort(
+ data, struct test_tevent_ev_source_data);
+ GSource *source = NULL;
+
+ source = g_timeout_source_new(100);
+ g_source_set_callback(source,
+ test_tevent_ev_source_quit_loop,
+ state,
+ NULL);
+ g_source_attach(source, state->ctx);
+ g_source_unref(source);
+
+ return;
+}
+
+static gboolean test_tevent_ev_source_quit_loop(gpointer data)
+{
+ struct test_tevent_ev_source_data *state = talloc_get_type_abort(
+ data, struct test_tevent_ev_source_data);
+
+ samba_tevent_glib_glue_quit(state->glue);
+
+ return G_SOURCE_REMOVE;
+}
+
+static void test_tevent_ev_source(void)
+{
+ GMainContext *ctx = NULL;
+ struct tevent_glib_glue *glue = NULL;
+ struct test_tevent_ev_source_data *state = NULL;
+ struct tevent_timer *timer = NULL;
+
+ ctx = g_main_context_new();
+ g_assert(ctx != NULL);
+
+ glue = samba_tevent_glib_glue_create(ev, ev, ctx);
+ g_assert(glue != NULL);
+
+ state = talloc_zero(glue, struct test_tevent_ev_source_data);
+ g_assert(state != NULL);
+
+ state->ctx = ctx;
+ state->glue = glue;
+
+ timer = tevent_add_timer(ev,
+ state,
+ tevent_timeval_current_ofs(0, 1000),
+ test_tevent_ev_source_timeout_cb,
+ state);
+ g_assert(timer != NULL);
+
+ g_assert(tevent_loop_wait(ev) == 0);
+
+ TALLOC_FREE(glue);
+ g_main_context_unref(ctx);
+}
+
+static gchar zeros[1024];
+
+static gsize fill_a_pipe(gint fd)
+{
+ gsize written = 0;
+ GPollFD pfd;
+
+ pfd.fd = fd;
+ pfd.events = G_IO_OUT;
+ while (g_poll(&pfd, 1, 0) == 1)
+ /* we should never see -1 here */
+ written += write(fd, zeros, sizeof zeros);
+
+ return written;
+}
+
+static gboolean write_bytes(gint fd,
+ GIOCondition condition,
+ gpointer user_data)
+{
+ gssize *to_write = user_data;
+ gint limit;
+
+ if (*to_write == 0)
+ return FALSE;
+
+ /* Detect if we run before we should */
+ g_assert(*to_write >= 0);
+
+ limit = MIN(*to_write, sizeof zeros);
+ *to_write -= write(fd, zeros, limit);
+
+ return TRUE;
+}
+
+static gboolean read_bytes(gint fd,
+ GIOCondition condition,
+ gpointer user_data)
+{
+ static gchar buffer[1024];
+ gssize *to_read = user_data;
+
+ *to_read -= read(fd, buffer, sizeof buffer);
+
+ /* The loop will exit when there is nothing else to read, then we will
+ * use g_source_remove() to destroy this source.
+ */
+ return TRUE;
+}
+
+static void test_unix_fd(void)
+{
+ gssize to_write = -1;
+ gssize to_read;
+ gint fds[2];
+ gint a, b;
+ gint s;
+ GSource *source_a = NULL;
+ GSource *source_b = NULL;
+ struct tevent_glib_glue *glue = NULL;
+
+ glue = samba_tevent_glib_glue_create(ev, ev, g_main_context_default());
+ g_assert(glue != NULL);
+
+ s = pipe(fds);
+ g_assert(s == 0);
+
+ to_read = fill_a_pipe(fds[1]);
+ /* write at higher priority to keep the pipe full... */
+ a = g_unix_fd_add_full(G_PRIORITY_HIGH,
+ fds[1],
+ G_IO_OUT,
+ write_bytes,
+ &to_write,
+ NULL);
+ source_a = g_source_ref(g_main_context_find_source_by_id(NULL, a));
+ /* make sure no 'writes' get dispatched yet */
+ while (tevent_loop_once(ev));
+
+ to_read += 128 * 1024 * 1024;
+ to_write = 128 * 1024 * 1024;
+ b = g_unix_fd_add(fds[0], G_IO_IN, read_bytes, &to_read);
+ source_b = g_source_ref(g_main_context_find_source_by_id(NULL, b));
+
+ /* Assuming the kernel isn't internally 'laggy' then there will always
+ * be either data to read or room in which to write. That will keep
+ * the loop running until all data has been read and written.
+ */
+ while (to_write > 0 || to_read > 0)
+ {
+ gssize to_write_was = to_write;
+ gssize to_read_was = to_read;
+
+ if (tevent_loop_once(ev) != 0)
+ break;
+
+ /* Since the sources are at different priority, only one of them
+ * should possibly have run.
+ */
+ g_assert(to_write == to_write_was || to_read == to_read_was);
+ }
+
+ g_assert(to_write == 0);
+ g_assert(to_read == 0);
+
+ /* 'a' is already removed by itself */
+ g_assert(g_source_is_destroyed(source_a));
+ g_source_unref(source_a);
+ g_source_remove(b);
+ g_assert(g_source_is_destroyed(source_b));
+ g_source_unref(source_b);
+
+ samba_tevent_glib_glue_quit(glue);
+ TALLOC_FREE(glue);
+
+ close(fds[1]);
+ close(fds[0]);
+}
+
+int main(int argc, const char *argv[])
+{
+ int test_argc = 3;
+ char *test_argv[] = {
+ discard_const("test_glib_glue"),
+ discard_const("-m"),
+ discard_const("no-undefined")
+ };
+ char **argvp = test_argv;
+
+ g_test_init(&test_argc, &argvp, NULL);
+
+ ev = tevent_context_init(NULL);
+ if (ev == NULL) {
+ exit(1);
+ }
+
+ g_test_add_func("/mainloop/timeouts", test_timeouts);
+ g_test_add_func("/mainloop/glib_ev_source", test_glib_ev_source);
+ g_test_add_func("/mainloop/tevent_ev_source", test_tevent_ev_source);
+ g_test_add_func("/mainloop/unix-fd", test_unix_fd);
+
+ return g_test_run();
+}
diff --git a/source3/script/tests/test_tevent_glib_glue.sh b/source3/script/tests/test_tevent_glib_glue.sh
new file mode 100755
index 00000000000..ce8cb3db0c1
--- /dev/null
+++ b/source3/script/tests/test_tevent_glib_glue.sh
@@ -0,0 +1,21 @@
+#!/bin/sh
+
+incdir=`dirname $0`/../../../testprogs/blackbox
+. $incdir/subunit.sh
+
+failed=0
+TESTNAME="tevent_glib_glue_test"
+
+if [ ! -x $BINDIR/tevent_glib_glue_test ] ; then
+ subunit_start_test "$TESTNAME"
+ subunit_skip_test "$TESTNAME" <<EOF
+Test needs glib2-devel
+EOF
+ testok $0 $failed
+fi
+
+
+testit "$TESTNAME" $VALGRIND $BINDIR/tevent_glib_glue_test ||
+ failed=`expr $failed + 1`
+
+testok $0 $failed
diff --git a/source3/selftest/tests.py b/source3/selftest/tests.py
index e8d516573dd..c6917219ab0 100755
--- a/source3/selftest/tests.py
+++ b/source3/selftest/tests.py
@@ -415,6 +415,9 @@ plantestsuite(
"samba3.resolvconf", "none",
[os.path.join(samba3srcdir, "script/tests/test_resolvconf.sh")])
+plantestsuite("samba3.tevent_glib_glue", "none",
+ [os.path.join(samba3srcdir, "script/tests/test_tevent_glib_glue.sh")])
+
plantestsuite("samba3.async_req", "nt4_dc",
[os.path.join(samba3srcdir, "script/tests/test_async_req.sh")])
diff --git a/source3/wscript_build b/source3/wscript_build
index c28b3e7621c..0a4e858d28e 100644
--- a/source3/wscript_build
+++ b/source3/wscript_build
@@ -1345,6 +1345,16 @@ bld.SAMBA3_BINARY('spotlight2sparql',
enabled=bld.env.with_spotlight,
install=False)
+bld.SAMBA3_BINARY('tevent_glib_glue_test',
+ source='lib/tevent_glib_glue_tests.c',
+ deps='''
+ talloc
+ libsmb
+ popt_samba3
+ tevent-glib-glue''',
+ enabled=bld.CONFIG_SET('WITH_TEVENT_GLIB_GLUE'),
+ install=False)
+
########################## INCLUDES #################################
bld.RECURSE('auth')
--
2.17.2
From 08fa4389c4b5447b0e055192da4e35f689ad2c0a Mon Sep 17 00:00:00 2001
From: Ralph Boehme <slow at samba.org>
Date: Fri, 22 Jan 2016 15:38:39 +0100
Subject: [PATCH 04/11] s3/lib: tevent-glib-glue test utiltity with Tracker
A small utilitly useful for tesing the tevent_glib_glue code. It runs a
tracker-sparql search query against your local tracker store that must
be setup and running.
Signed-off-by: Ralph Boehme <slow at samba.org>
---
source3/utils/async-tracker.c | 288 ++++++++++++++++++++++++++++++++++
source3/wscript_build | 10 ++
2 files changed, 298 insertions(+)
create mode 100644 source3/utils/async-tracker.c
diff --git a/source3/utils/async-tracker.c b/source3/utils/async-tracker.c
new file mode 100644
index 00000000000..4ccf26f907e
--- /dev/null
+++ b/source3/utils/async-tracker.c
@@ -0,0 +1,288 @@
+/*
+ * Copyright (C) 2011, Nokia <ivan.frade at nokia.com>
+ * Copyright (C) 2015, Noel Power <nopower at suse.com>
+ * Copyright (C) 2016, Ralph Boehme <slow at samba.org.>
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public
+ * License along with this library; if not, write to the
+ * Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor,
+ * Boston, MA 02110-1301, USA.
+ */
+
+#include "includes.h"
+#include "lib/util/debug.h"
+#include "popt_common.h"
+#include "param.h"
+/*
+ * glib uses TRUE and FALSE which was redefined by "includes.h" to be
+ * unusable, undefine so glib can establish its own working
+ * replacement.
+ */
+#undef TRUE
+#undef FALSE
+#include <glib.h>
+#include <libtracker-sparql/tracker-sparql.h>
+#include "lib/tevent_glib_glue.h"
+
+enum loop_type {TEVENT_LOOP, GLIB_LOOP};
+
+struct test_state {
+ enum loop_type loop_type;
+ TrackerSparqlConnection *connection;
+ GCancellable *cancellable;
+ GTimer *timer;
+ GMainLoop *loop;
+ struct tevent_context *ev;
+ struct tevent_glib_glue *glue;
+};
+
+static void cleanup(struct test_state *state)
+{
+ g_cancellable_cancel(state->cancellable);
+ g_object_unref(state->cancellable);
+ g_timer_destroy(state->timer);
+ if (state->connection != NULL) {
+ g_object_unref(state->connection);
+ state->connection = NULL;
+ }
+ if (state->loop_type == GLIB_LOOP) {
+ g_main_loop_quit(state->loop);
+ } else {
+ samba_tevent_glib_glue_quit(state->glue);
+ }
+}
+
+static void cursor_cb(GObject *object,
+ GAsyncResult *res,
+ gpointer user_data)
+{
+ struct test_state *state = talloc_get_type_abort(
+ user_data, struct test_state);
+ TrackerSparqlCursor *cursor = NULL;
+ GError *error = NULL;
+ gboolean more_results;
+ static gint i = 0;
+
+ cursor = TRACKER_SPARQL_CURSOR(object);
+ more_results = tracker_sparql_cursor_next_finish(cursor,
+ res,
+ &error);
+ if (error) {
+ g_critical("Could not run cursor next: %s", error->message);
+
+ if (cursor != NULL) {
+ g_object_unref(cursor);
+ }
+
+ g_error_free(error);
+ cleanup(state);
+ return;
+ }
+
+ if (!more_results) {
+ g_print("\n");
+ g_print("\nAsync cursor next took: %.6f (for all %d results)\n",
+ g_timer_elapsed (state->timer, NULL), i);
+
+ g_object_unref(cursor);
+ cleanup(state);
+ return;
+ }
+
+ if (i++ < 5) {
+ int num_cols = tracker_sparql_cursor_get_n_columns(cursor);
+ int col;
+
+ if (i == 1) {
+ g_print("Printing first 5 results:\n");
+ }
+ for (col = 0; col < num_cols; col++) {
+ g_print(" %s ", tracker_sparql_cursor_get_string(
+ cursor, col, NULL));
+ if (col == num_cols -1 ) {
+ g_print("\n");
+ }
+ }
+
+ if (i == 5) {
+ g_print(" ...\n");
+ g_print(" Printing nothing for remaining results\n");
+ }
+ }
+
+ tracker_sparql_cursor_next_async(cursor,
+ state->cancellable,
+ cursor_cb,
+ state);
+}
+
+static void query_cb(GObject *object,
+ GAsyncResult *res,
+ gpointer user_data)
+{
+ struct test_state *state = talloc_get_type_abort(
+ user_data, struct test_state);
+ TrackerSparqlCursor *cursor = NULL;
+ GError *error = NULL;
+
+ g_print("Async query took: %.6f\n", g_timer_elapsed(state->timer, NULL));
+
+ cursor = tracker_sparql_connection_query_finish(
+ TRACKER_SPARQL_CONNECTION(object),
+ res,
+ &error);
+ if (error) {
+ g_critical("Could not run query: %s", error->message);
+
+ if (cursor) {
+ g_object_unref(cursor);
+ }
+
+ g_error_free(error);
+ cleanup(state);
+ return;
+ }
+
+ g_timer_start(state->timer);
+
+ tracker_sparql_cursor_next_async(cursor,
+ state->cancellable,
+ cursor_cb,
+ state);
+}
+
+static void connection_cb(GObject *object,
+ GAsyncResult *res,
+ gpointer user_data)
+{
+ struct test_state *state = talloc_get_type_abort(
+ user_data, struct test_state);
+ GError *error = NULL;
+
+ g_print("Async connection took: %.6f\n",
+ g_timer_elapsed(state->timer, NULL));
+
+ state->connection = tracker_sparql_connection_get_finish(res, &error);
+ if (error) {
+ g_critical("Could not connect: %s", error->message);
+ g_error_free(error);
+ cleanup(state);
+ return;
+ }
+
+ g_timer_start(state->timer);
+
+ tracker_sparql_connection_query_async(
+ state->connection,
+ "SELECT ?name nie:mimeType(?s) nfo:fileName(?s) "
+ "WHERE { {?s nie:url ?name}}",
+ state->cancellable,
+ query_cb,
+ state);
+}
+
+static void debug_fn(void *private_data,
+ enum tevent_debug_level level,
+ const char *fmt,
+ va_list ap)
+{
+ dbgtext_va(fmt, ap);
+}
+
+int main(int argc, const char **argv)
+{
+ TALLOC_CTX *mem_ctx = NULL;
+ struct test_state *state = NULL;
+ int c;
+ poptContext pc;
+ struct poptOption long_options[] = {
+ POPT_AUTOHELP
+ {"tevent", 't', POPT_ARG_NONE, NULL, 't', "Use tevent loop" },
+ {"glib", 'g', POPT_ARG_NONE, NULL, 'g', "Use glib loop" },
+ POPT_COMMON_SAMBA
+ POPT_TABLEEND
+ };
+
+ mem_ctx = talloc_new(NULL);
+ if (mem_ctx == NULL) {
+ exit(1);
+ }
+
+ state = talloc_zero(mem_ctx, struct test_state);
+ if (state == NULL) {
+ exit(1);
+ }
+
+ state->loop_type = TEVENT_LOOP;
+
+ setup_logging(argv[0], DEBUG_STDERR);
+ smb_init_locale();
+
+ if (!lp_load_client(get_dyn_CONFIGFILE())) {
+ fprintf(stderr, "ERROR: Can't load %s\n",
+ get_dyn_CONFIGFILE());
+ exit(1);
+ }
+
+ pc = poptGetContext(NULL, argc, argv, long_options,
+ POPT_CONTEXT_KEEP_FIRST);
+
+ while ((c = poptGetNextOpt(pc)) != -1) {
+ switch (c) {
+ case 'g':
+ state->loop_type = GLIB_LOOP;
+ break;
+ case 't':
+ state->loop_type = TEVENT_LOOP;
+ break;
+ }
+ }
+
+ if (state->loop_type == GLIB_LOOP) {
+ state->loop = g_main_loop_new(NULL, false);
+ } else {
+ state->ev = tevent_context_init(mem_ctx);
+ if (CHECK_DEBUGLVL(10)) {
+ tevent_set_debug(state->ev, debug_fn, NULL);
+ }
+ state->glue = samba_tevent_glib_glue_create(
+ mem_ctx, state->ev, g_main_context_default());
+ if (state->glue == NULL) {
+ printf("tevent_glib_glue_create failed\n");
+ exit(1);
+ }
+ }
+
+ state->timer = g_timer_new();
+ state->cancellable = g_cancellable_new();
+
+ tracker_sparql_connection_get_async(state->cancellable,
+ connection_cb,
+ state);
+
+ if (state->loop_type == GLIB_LOOP) {
+ printf("entering g_main_loop_run\n");
+ g_main_loop_run(state->loop);
+ } else {
+ printf("entering tevent_loop_wait\n");
+ tevent_loop_wait(state->ev);
+
+ TALLOC_FREE(state->glue);
+ TALLOC_FREE(state->ev);
+ }
+
+ TALLOC_FREE(mem_ctx);
+ poptFreeContext(pc);
+
+ return 0;
+}
diff --git a/source3/wscript_build b/source3/wscript_build
index 0a4e858d28e..347ed0f8a87 100644
--- a/source3/wscript_build
+++ b/source3/wscript_build
@@ -1355,6 +1355,16 @@ bld.SAMBA3_BINARY('tevent_glib_glue_test',
enabled=bld.CONFIG_SET('WITH_TEVENT_GLIB_GLUE'),
install=False)
+bld.SAMBA3_BINARY('tevent_glib_tracker',
+ source='utils/async-tracker.c',
+ deps='''
+ talloc
+ libsmb
+ popt_samba3
+ tevent-glib-glue ''' + bld.env['libtracker'],
+ enabled=bld.CONFIG_SET('HAVE_TRACKER') and bld.CONFIG_SET('WITH_TEVENT_GLIB_GLUE'),
+ install=False)
+
########################## INCLUDES #################################
bld.RECURSE('auth')
--
2.17.2
From 9c9bee0b7b74687be005a230361f14ee85d78e9f Mon Sep 17 00:00:00 2001
From: Ralph Boehme <slow at samba.org>
Date: Wed, 27 Jan 2016 13:17:04 +0100
Subject: [PATCH 05/11] s3-mdssvc: add tevent context arg to mds_init_ctx
This is needed later when adding tevent_glib_glue support, not used for now.
Signed-off-by: Ralph Boehme <slow at samba.org>
---
source3/rpc_server/mdssvc/mdssvc.c | 1 +
source3/rpc_server/mdssvc/mdssvc.h | 1 +
source3/rpc_server/mdssvc/srv_mdssvc_nt.c | 6 +++++-
3 files changed, 7 insertions(+), 1 deletion(-)
diff --git a/source3/rpc_server/mdssvc/mdssvc.c b/source3/rpc_server/mdssvc/mdssvc.c
index 7c996941cfa..ea676f48681 100644
--- a/source3/rpc_server/mdssvc/mdssvc.c
+++ b/source3/rpc_server/mdssvc/mdssvc.c
@@ -1822,6 +1822,7 @@ static gboolean gmainloop_timer(gpointer user_data)
* Initialise a context per share handle
**/
struct mds_ctx *mds_init_ctx(TALLOC_CTX *mem_ctx,
+ struct tevent_context *ev,
const struct auth_session_info *session_info,
const char *path)
{
diff --git a/source3/rpc_server/mdssvc/mdssvc.h b/source3/rpc_server/mdssvc/mdssvc.h
index 4e08fe18667..310e59582ed 100644
--- a/source3/rpc_server/mdssvc/mdssvc.h
+++ b/source3/rpc_server/mdssvc/mdssvc.h
@@ -118,6 +118,7 @@ struct mds_ctx {
extern bool mds_init(struct messaging_context *msg_ctx);
extern bool mds_shutdown(void);
extern struct mds_ctx *mds_init_ctx(TALLOC_CTX *mem_ctx,
+ struct tevent_context *ev,
const struct auth_session_info *session_info,
const char *path);
extern int mds_ctx_destructor_cb(struct mds_ctx *mds_ctx);
diff --git a/source3/rpc_server/mdssvc/srv_mdssvc_nt.c b/source3/rpc_server/mdssvc/srv_mdssvc_nt.c
index ab64423d0df..c4bf995ce14 100644
--- a/source3/rpc_server/mdssvc/srv_mdssvc_nt.c
+++ b/source3/rpc_server/mdssvc/srv_mdssvc_nt.c
@@ -18,6 +18,7 @@
*/
#include "includes.h"
+#include "messages.h"
#include "ntdomain.h"
#include "rpc_server/rpc_service_setup.h"
#include "rpc_server/rpc_config.h"
@@ -114,7 +115,10 @@ static NTSTATUS create_mdssvc_policy_handle(TALLOC_CTX *mem_ctx,
ZERO_STRUCTP(handle);
- mds_ctx = mds_init_ctx(mem_ctx, p->session_info, path);
+ mds_ctx = mds_init_ctx(mem_ctx,
+ messaging_tevent_context(p->msg_ctx),
+ p->session_info,
+ path);
if (mds_ctx == NULL) {
DEBUG(1, ("error in mds_init_ctx for: %s\n", path));
return NT_STATUS_UNSUCCESSFUL;
--
2.17.2
From a950c76ea2ab22c2403bd3e1a1c7e4459e9285df Mon Sep 17 00:00:00 2001
From: Ralph Boehme <slow at samba.org>
Date: Mon, 11 Mar 2019 18:11:04 +0100
Subject: [PATCH 06/11] s3-mdssvc: call [un]become_authenticated_pipe_user()
This ensures we're running as the authenticated user int the tevent
callback which might be running in an arbitrary impersonation context.
Signed-off-by: Ralph Boehme <slow at samba.org>
---
source3/rpc_server/mdssvc/mdssvc.c | 23 ++++++++++++++++++++++-
source3/rpc_server/mdssvc/mdssvc.h | 3 ++-
2 files changed, 24 insertions(+), 2 deletions(-)
diff --git a/source3/rpc_server/mdssvc/mdssvc.c b/source3/rpc_server/mdssvc/mdssvc.c
index ea676f48681..f03cb9d7ebd 100644
--- a/source3/rpc_server/mdssvc/mdssvc.c
+++ b/source3/rpc_server/mdssvc/mdssvc.c
@@ -810,20 +810,39 @@ static void tracker_cursor_cb(GObject *object,
return;
}
+ /*
+ * We're in a tevent callback which means in the case of
+ * running as external RPC service we're running as root and
+ * not as the user.
+ */
+ if (!become_authenticated_pipe_user(slq->mds_ctx->pipe_session_info)) {
+ DBG_ERR("can't become authenticated user: %d\n", slq->mds_ctx->uid);
+ smb_panic("can't become authenticated user");
+ }
+
if (geteuid() != slq->mds_ctx->uid) {
DEBUG(0, ("uid mismatch: %d/%d\n", geteuid(), slq->mds_ctx->uid));
smb_panic("uid mismatch");
}
+ /*
+ * We've changed identity to the authenticated pipe user, so
+ * any function exit below must ensure we switch back
+ */
+
result = sys_stat(path, &sb, false);
if (result != 0) {
+ unbecome_authenticated_pipe_user();
goto done;
}
result = access(path, R_OK);
if (result != 0) {
+ unbecome_authenticated_pipe_user();
goto done;
}
+ unbecome_authenticated_pipe_user();
+
ino64 = sb.st_ex_ino;
if (slq->cnids) {
/*
@@ -1823,7 +1842,7 @@ static gboolean gmainloop_timer(gpointer user_data)
**/
struct mds_ctx *mds_init_ctx(TALLOC_CTX *mem_ctx,
struct tevent_context *ev,
- const struct auth_session_info *session_info,
+ struct auth_session_info *session_info,
const char *path)
{
struct mds_ctx *mds_ctx;
@@ -1839,6 +1858,8 @@ struct mds_ctx *mds_init_ctx(TALLOC_CTX *mem_ctx,
goto error;
}
+ mds_ctx->pipe_session_info = session_info;
+
if (session_info->security_token->num_sids < 1) {
goto error;
}
diff --git a/source3/rpc_server/mdssvc/mdssvc.h b/source3/rpc_server/mdssvc/mdssvc.h
index 310e59582ed..e1b1ccfc34d 100644
--- a/source3/rpc_server/mdssvc/mdssvc.h
+++ b/source3/rpc_server/mdssvc/mdssvc.h
@@ -97,6 +97,7 @@ struct sl_inode_path_map {
};
struct mds_ctx {
+ struct auth_session_info *pipe_session_info;
struct dom_sid sid;
uid_t uid;
const char *spath;
@@ -119,7 +120,7 @@ extern bool mds_init(struct messaging_context *msg_ctx);
extern bool mds_shutdown(void);
extern struct mds_ctx *mds_init_ctx(TALLOC_CTX *mem_ctx,
struct tevent_context *ev,
- const struct auth_session_info *session_info,
+ struct auth_session_info *session_info,
const char *path);
extern int mds_ctx_destructor_cb(struct mds_ctx *mds_ctx);
extern bool mds_dispatch(struct mds_ctx *query_ctx,
--
2.17.2
From 098cd62bbae195754c84614bbb37516726dadc2d Mon Sep 17 00:00:00 2001
From: Ralph Boehme <slow at samba.org>
Date: Wed, 27 Jan 2016 13:23:51 +0100
Subject: [PATCH 07/11] s3-mdssvc: use tevent_glib_glue in mdssvc RPC service
Signed-off-by: Ralph Boehme <slow at samba.org>
---
source3/rpc_server/mdssvc/mdssvc.c | 206 +++++++++--------------------
source3/rpc_server/mdssvc/mdssvc.h | 11 +-
source3/rpc_server/wscript_build | 4 +-
3 files changed, 75 insertions(+), 146 deletions(-)
diff --git a/source3/rpc_server/mdssvc/mdssvc.c b/source3/rpc_server/mdssvc/mdssvc.c
index f03cb9d7ebd..c8545a52e55 100644
--- a/source3/rpc_server/mdssvc/mdssvc.c
+++ b/source3/rpc_server/mdssvc/mdssvc.c
@@ -28,6 +28,7 @@
#include "libcli/security/dom_sid.h"
#include "mdssvc.h"
#include "rpc_server/mdssvc/sparql_parser.tab.h"
+#include "lib/tevent_glib_glue.h"
#undef DBGC_CLASS
#define DBGC_CLASS DBGC_RPC_SRV
@@ -64,6 +65,15 @@ struct slq_destroy_state {
struct sl_query *slq;
};
+/*
+ * This is a static global because we may be called multiple times and
+ * we only want one mdssvc_ctx per connection to Tracker.
+ *
+ * The client will bind multiple times to the mdssvc RPC service, once
+ * for every tree connect.
+ */
+static struct mdssvc_ctx *mdssvc_ctx = NULL;
+
/*
* If these functions return an error, they hit something like a non
* recoverable talloc error. Most errors are dealt with by returning
@@ -731,7 +741,6 @@ static void tracker_con_cb(GObject *object,
}
DEBUG(10, ("connected to Tracker\n"));
- g_main_loop_quit(mds_ctx->gmainloop);
}
static void tracker_cursor_cb_destroy_done(struct tevent_req *subreq);
@@ -770,7 +779,6 @@ static void tracker_cursor_cb(GObject *object,
* we return.
*/
SLQ_DEBUG(10, slq, "closed");
- g_main_loop_quit(slq->mds_ctx->gmainloop);
req = slq_destroy_send(slq, global_event_context(), &slq);
if (req == NULL) {
@@ -785,13 +793,11 @@ static void tracker_cursor_cb(GObject *object,
DEBUG(1, ("Tracker cursor: %s\n", error->message));
g_error_free(error);
slq->state = SLQ_STATE_ERROR;
- g_main_loop_quit(slq->mds_ctx->gmainloop);
return;
}
if (!more_results) {
slq->state = SLQ_STATE_DONE;
- g_main_loop_quit(slq->mds_ctx->gmainloop);
return;
}
@@ -799,14 +805,12 @@ static void tracker_cursor_cb(GObject *object,
if (uri == NULL) {
DEBUG(1, ("error fetching Tracker URI\n"));
slq->state = SLQ_STATE_ERROR;
- g_main_loop_quit(slq->mds_ctx->gmainloop);
return;
}
path = tracker_to_unix_path(slq->query_results, uri);
if (path == NULL) {
DEBUG(1, ("error converting Tracker URI to path: %s\n", uri));
slq->state = SLQ_STATE_ERROR;
- g_main_loop_quit(slq->mds_ctx->gmainloop);
return;
}
@@ -866,7 +870,6 @@ static void tracker_cursor_cb(GObject *object,
if (result != 0) {
DEBUG(1, ("dalloc error\n"));
slq->state = SLQ_STATE_ERROR;
- g_main_loop_quit(slq->mds_ctx->gmainloop);
return;
}
ok = add_filemeta(slq->reqinfo, slq->query_results->fm_array,
@@ -874,7 +877,6 @@ static void tracker_cursor_cb(GObject *object,
if (!ok) {
DEBUG(1, ("add_filemeta error\n"));
slq->state = SLQ_STATE_ERROR;
- g_main_loop_quit(slq->mds_ctx->gmainloop);
return;
}
@@ -882,7 +884,6 @@ static void tracker_cursor_cb(GObject *object,
if (!ok) {
DEBUG(1, ("inode_map_add error\n"));
slq->state = SLQ_STATE_ERROR;
- g_main_loop_quit(slq->mds_ctx->gmainloop);
return;
}
@@ -892,7 +893,6 @@ static void tracker_cursor_cb(GObject *object,
if (slq->query_results->num_results >= MAX_SL_RESULTS) {
slq->state = SLQ_STATE_FULL;
SLQ_DEBUG(10, slq, "full");
- g_main_loop_quit(slq->mds_ctx->gmainloop);
return;
}
@@ -929,13 +929,11 @@ static void tracker_query_cb(GObject *object,
slq->state = SLQ_STATE_ERROR;
DEBUG(1, ("Tracker query error: %s\n", error->message));
g_error_free(error);
- g_main_loop_quit(slq->mds_ctx->gmainloop);
return;
}
if (slq->state == SLQ_STATE_DONE) {
SLQ_DEBUG(10, slq, "done");
- g_main_loop_quit(slq->mds_ctx->gmainloop);
talloc_free(slq);
return;
}
@@ -1302,13 +1300,11 @@ static bool slrpc_open_query(struct mds_ctx *mds_ctx,
DEBUG(10, ("SPARQL query: \"%s\"\n", slq->sparql_query));
- g_main_context_push_thread_default(mds_ctx->gcontext);
tracker_sparql_connection_query_async(mds_ctx->tracker_con,
slq->sparql_query,
slq->gcancellable,
tracker_query_cb,
slq);
- g_main_context_pop_thread_default(mds_ctx->gcontext);
slq->state = SLQ_STATE_RUNNING;
sl_result = 0;
@@ -1401,13 +1397,11 @@ static bool slrpc_fetch_query_results(struct mds_ctx *mds_ctx,
}
if (slq->state == SLQ_STATE_FULL) {
slq->state = SLQ_STATE_RESULTS;
- g_main_context_push_thread_default(mds_ctx->gcontext);
tracker_sparql_cursor_next_async(
slq->tracker_cursor,
slq->gcancellable,
tracker_cursor_cb,
slq);
- g_main_context_pop_thread_default(mds_ctx->gcontext);
}
break;
@@ -1811,6 +1805,42 @@ static bool slrpc_close_query(struct mds_ctx *mds_ctx,
return true;
}
+static struct mdssvc_ctx *mdssvc_init(struct tevent_context *ev)
+{
+ if (mdssvc_ctx != NULL) {
+ return mdssvc_ctx;
+ }
+
+ mdssvc_ctx = talloc_zero(ev, struct mdssvc_ctx);
+ if (mdssvc_ctx == NULL) {
+ return NULL;
+ }
+
+ mdssvc_ctx->ev_ctx = ev;
+
+ mdssvc_ctx->gmain_ctx = g_main_context_new();
+ if (mdssvc_ctx->gmain_ctx == NULL) {
+ DBG_ERR("error from g_main_context_new\n");
+ return NULL;
+ }
+
+ /*
+ * This ensures all glib threads, especially gioi worker threads
+ * dispatch their async callbacks via our gmain_ctx.
+ */
+ g_main_context_push_thread_default(mdssvc_ctx->gmain_ctx);
+
+ mdssvc_ctx->glue = samba_tevent_glib_glue_create(ev,
+ mdssvc_ctx->ev_ctx,
+ mdssvc_ctx->gmain_ctx);
+ if (mdssvc_ctx->glue == NULL) {
+ DBG_ERR("samba_tevent_glib_glue_create failed\n");
+ return NULL;
+ }
+
+ return mdssvc_ctx;
+}
+
/**
* Init callbacks at startup
**/
@@ -1824,21 +1854,25 @@ bool mds_init(struct messaging_context *msg_ctx)
bool mds_shutdown(void)
{
- return true;
-}
+ if (mdssvc_ctx == NULL) {
+ return false;
+ }
-static gboolean gmainloop_timer(gpointer user_data)
-{
- struct mds_ctx *ctx = talloc_get_type_abort(user_data, struct mds_ctx);
+ samba_tevent_glib_glue_quit(mdssvc_ctx->glue);
+ TALLOC_FREE(mdssvc_ctx->glue);
- DEBUG(10,("%s\n", __func__));
- g_main_loop_quit(ctx->gmainloop);
+ g_main_context_pop_thread_default(mdssvc_ctx->gmain_ctx);
- return G_SOURCE_CONTINUE;
+ TALLOC_FREE(mdssvc_ctx);
+
+ return true;
}
/**
- * Initialise a context per share handle
+ * Initialise a context per RPC bind
+ *
+ * This ends up being called for every tcon, because the client does a
+ * RPC bind for every tcon, so this is acually a per tcon context.
**/
struct mds_ctx *mds_init_ctx(TALLOC_CTX *mem_ctx,
struct tevent_context *ev,
@@ -1853,6 +1887,11 @@ struct mds_ctx *mds_init_ctx(TALLOC_CTX *mem_ctx,
}
talloc_set_destructor(mds_ctx, mds_ctx_destructor_cb);
+ mds_ctx->mdssvc_ctx = mdssvc_init(ev);
+ if (mds_ctx->mdssvc_ctx == NULL) {
+ goto error;
+ }
+
mds_ctx->spath = talloc_strdup(mds_ctx, path);
if (mds_ctx->spath == NULL) {
goto error;
@@ -1872,22 +1911,8 @@ struct mds_ctx *mds_init_ctx(TALLOC_CTX *mem_ctx,
goto error;
}
- mds_ctx->gcontext = g_main_context_new();
- if (mds_ctx->gcontext == NULL) {
- DEBUG(1,("error from g_main_context_new\n"));
- goto error;
- }
-
- mds_ctx->gmainloop = g_main_loop_new(mds_ctx->gcontext, false);
- if (mds_ctx->gmainloop == NULL) {
- DEBUG(1,("error from g_main_loop_new\n"));
- goto error;
- }
-
- g_main_context_push_thread_default(mds_ctx->gcontext);
tracker_sparql_connection_get_async(mds_ctx->gcancellable,
tracker_con_cb, mds_ctx);
- g_main_context_pop_thread_default(mds_ctx->gcontext);
return mds_ctx;
@@ -1920,76 +1945,12 @@ int mds_ctx_destructor_cb(struct mds_ctx *mds_ctx)
g_cancellable_cancel(mds_ctx->gcancellable);
g_object_unref(mds_ctx->gcancellable);
}
- if (mds_ctx->gmainloop != NULL) {
- g_main_loop_unref(mds_ctx->gmainloop);
- }
- if (mds_ctx->gcontext != NULL) {
- g_main_context_unref(mds_ctx->gcontext);
- }
ZERO_STRUCTP(mds_ctx);
return 0;
}
-static bool mds_run_gmainloop(struct mds_ctx *mds_ctx, guint timeout)
-{
- guint timer_id;
- GSource *timer;
-
- /*
- * It seems the event processing of the libtracker-sparql
- * async subsystem defers callbacks until *all* events are
- * processes by the async subsystem main processing loop.
- *
- * g_main_context_iteration(may_block=FALSE) can't be used,
- * because a search that produces a few thousand matches
- * generates as many events that must be processed in either
- * g_main_context_iteration() or g_main_loop_run() before
- * callbacks are called.
- *
- * Unfortunately g_main_context_iteration() only processes a
- * small subset of these event (1-30) at a time when run in
- * mds_dispatch(), which happens once a second while the
- * client polls for results.
- *
- * Carefully using the blocking g_main_loop_run() fixes
- * this. It processes events until we exit from the loop at
- * defined exit points. By adding a 1 ms timeout we at least
- * try to get as close as possible to non-blocking behaviour.
- */
-
- if (!g_main_context_pending(mds_ctx->gcontext)) {
- return true;
- }
-
- g_main_context_push_thread_default(mds_ctx->gcontext);
-
- timer = g_timeout_source_new(timeout);
- if (timer == NULL) {
- DEBUG(1,("g_timeout_source_new_seconds\n"));
- g_main_context_pop_thread_default(mds_ctx->gcontext);
- return false;
- }
-
- timer_id = g_source_attach(timer, mds_ctx->gcontext);
- if (timer_id == 0) {
- DEBUG(1,("g_timeout_add failed\n"));
- g_source_destroy(timer);
- g_main_context_pop_thread_default(mds_ctx->gcontext);
- return false;
- }
-
- g_source_set_callback(timer, gmainloop_timer, mds_ctx, NULL);
-
- g_main_loop_run(mds_ctx->gmainloop);
-
- g_source_destroy(timer);
-
- g_main_context_pop_thread_default(mds_ctx->gcontext);
- return true;
-}
-
/**
* Dispatch a Spotlight RPC command
**/
@@ -2014,34 +1975,6 @@ bool mds_dispatch(struct mds_ctx *mds_ctx,
response_blob->length = 0;
- /*
- * Process finished glib events.
- *
- * FIXME: integrate with tevent instead of piggy packing it
- * onto the processing of new requests.
- *
- * mds_dispatch() is called by the client a few times in a row:
- *
- * - first in order to open/start a search query
- *
- * - later in order to fetch results asynchronously, typically
- * once a second. If no results have been retrieved from the
- * search store (Tracker) yet, we return no results.
- * The client asks for more results every second as long
- * as the "Search Window" in the client gui is open.
- *
- * - at some point the query is closed
- *
- * This means we try to iterate through the glib event loop
- * before processing the request in order to get result
- * from tracker which can be returned to the client.
- */
-
- ok = mds_run_gmainloop(mds_ctx, MDS_TRACKER_ASYNC_TIMEOUT_MS);
- if (!ok) {
- goto cleanup;
- }
-
DEBUG(10, ("share path: %s\n", mds_ctx->spath));
query = dalloc_new(mds_ctx);
@@ -2102,17 +2035,6 @@ bool mds_dispatch(struct mds_ctx *mds_ctx,
goto cleanup;
}
- /*
- * Run g_main_loop a second time in order to dispatch events
- * that may have been queued at the libtracker-sparql level.
- * As we only want to dispatch (write out requests) but not
- * wait for anything, we use a much shorter timeout here.
- */
- ok = mds_run_gmainloop(mds_ctx, MDS_TRACKER_ASYNC_TIMEOUT_MS / 10);
- if (!ok) {
- goto cleanup;
- }
-
response_blob->length = len;
cleanup:
diff --git a/source3/rpc_server/mdssvc/mdssvc.h b/source3/rpc_server/mdssvc/mdssvc.h
index e1b1ccfc34d..b9e3e56f785 100644
--- a/source3/rpc_server/mdssvc/mdssvc.h
+++ b/source3/rpc_server/mdssvc/mdssvc.h
@@ -96,15 +96,22 @@ struct sl_inode_path_map {
char *path;
};
+/* Per process state */
+struct mdssvc_ctx {
+ struct tevent_context *ev_ctx;
+ GMainContext *gmain_ctx;
+ struct tevent_glib_glue *glue;
+};
+
+/* Per tree connect state */
struct mds_ctx {
+ struct mdssvc_ctx *mdssvc_ctx;
struct auth_session_info *pipe_session_info;
struct dom_sid sid;
uid_t uid;
const char *spath;
GCancellable *gcancellable;
TrackerSparqlConnection *tracker_con;
- GMainContext *gcontext;
- GMainLoop *gmainloop;
struct sl_query *query_list; /* list of active queries */
struct db_context *ino_path_map; /* dbwrap rbt for storing inode->path mappings */
};
diff --git a/source3/rpc_server/wscript_build b/source3/rpc_server/wscript_build
index ae75e567ace..354f6050ff6 100644
--- a/source3/rpc_server/wscript_build
+++ b/source3/rpc_server/wscript_build
@@ -150,7 +150,7 @@ bld.SAMBA3_MODULE('rpc_mdssvc_module',
mdssvc/srv_mdssvc_nt.c
../../librpc/gen_ndr/srv_mdssvc.c''',
init_function='',
- deps='samba-util ' + bld.env['libtracker'],
+ deps='samba-util tevent-glib-glue ' + bld.env['libtracker'],
internal_module=bld.SAMBA3_IS_STATIC_MODULE('rpc_mdssvc_module'),
enabled=bld.SAMBA3_IS_ENABLED_MODULE('rpc_mdssvc_module'))
@@ -205,5 +205,5 @@ bld.SAMBA3_SUBSYSTEM('FSSD',
bld.SAMBA3_SUBSYSTEM('MDSSD',
source='mdssd.c',
- deps='RPC_SOCK_HELPER RPC_MODULES samba-util',
+ deps='RPC_SOCK_HELPER RPC_MODULES samba-util tevent-glib-glue',
enabled=bld.env.with_spotlight)
--
2.17.2
From f6c3ce9c7dedf807f848ce82446952ea297fb2ad Mon Sep 17 00:00:00 2001
From: Ralph Boehme <slow at samba.org>
Date: Mon, 25 Mar 2019 16:11:30 +0100
Subject: [PATCH 08/11] s3-mdssvc: use default g_main context
Way back when the module was developed it seemed to be necessary the use
a private context with push/pop as thread default. Maybe there was a bug
in libtracker-sparql dispatching callback in the wrong (global)
context. It's not necessary anymore with a recent libtracker-sparql
version.
Signed-off-by: Ralph Boehme <slow at samba.org>
---
source3/rpc_server/mdssvc/mdssvc.c | 10 +---------
1 file changed, 1 insertion(+), 9 deletions(-)
diff --git a/source3/rpc_server/mdssvc/mdssvc.c b/source3/rpc_server/mdssvc/mdssvc.c
index c8545a52e55..e297333ad16 100644
--- a/source3/rpc_server/mdssvc/mdssvc.c
+++ b/source3/rpc_server/mdssvc/mdssvc.c
@@ -1818,18 +1818,12 @@ static struct mdssvc_ctx *mdssvc_init(struct tevent_context *ev)
mdssvc_ctx->ev_ctx = ev;
- mdssvc_ctx->gmain_ctx = g_main_context_new();
+ mdssvc_ctx->gmain_ctx = g_main_context_default();
if (mdssvc_ctx->gmain_ctx == NULL) {
DBG_ERR("error from g_main_context_new\n");
return NULL;
}
- /*
- * This ensures all glib threads, especially gioi worker threads
- * dispatch their async callbacks via our gmain_ctx.
- */
- g_main_context_push_thread_default(mdssvc_ctx->gmain_ctx);
-
mdssvc_ctx->glue = samba_tevent_glib_glue_create(ev,
mdssvc_ctx->ev_ctx,
mdssvc_ctx->gmain_ctx);
@@ -1861,8 +1855,6 @@ bool mds_shutdown(void)
samba_tevent_glib_glue_quit(mdssvc_ctx->glue);
TALLOC_FREE(mdssvc_ctx->glue);
- g_main_context_pop_thread_default(mdssvc_ctx->gmain_ctx);
-
TALLOC_FREE(mdssvc_ctx);
return true;
--
2.17.2
From cb98babdf480ce496af2b9d42bebebeecd4e9527 Mon Sep 17 00:00:00 2001
From: Ralph Boehme <slow at samba.org>
Date: Tue, 12 Mar 2019 15:27:25 +0100
Subject: [PATCH 09/11] s3-mdssvc: add missing call to g_cancellable_new()
Signed-off-by: Ralph Boehme <slow at samba.org>
---
source3/rpc_server/mdssvc/mdssvc.c | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/source3/rpc_server/mdssvc/mdssvc.c b/source3/rpc_server/mdssvc/mdssvc.c
index e297333ad16..693675fff6f 100644
--- a/source3/rpc_server/mdssvc/mdssvc.c
+++ b/source3/rpc_server/mdssvc/mdssvc.c
@@ -1903,6 +1903,12 @@ struct mds_ctx *mds_init_ctx(TALLOC_CTX *mem_ctx,
goto error;
}
+ mds_ctx->gcancellable = g_cancellable_new();
+ if (mds_ctx->gcancellable == NULL) {
+ DBG_ERR("error from g_cancellable_new\n");
+ goto error;
+ }
+
tracker_sparql_connection_get_async(mds_ctx->gcancellable,
tracker_con_cb, mds_ctx);
--
2.17.2
From 5a8eec5a86341cfe8e326f777c9f7c89ca24ce08 Mon Sep 17 00:00:00 2001
From: Ralph Boehme <slow at samba.org>
Date: Tue, 12 Mar 2019 15:29:48 +0100
Subject: [PATCH 10/11] s3-mdssvc: make mds_ctx_destructor_cb static
This is only used in this compilation unit.
Signed-off-by: Ralph Boehme <slow at samba.org>
---
source3/rpc_server/mdssvc/mdssvc.c | 60 +++++++++++++++---------------
source3/rpc_server/mdssvc/mdssvc.h | 1 -
2 files changed, 30 insertions(+), 31 deletions(-)
diff --git a/source3/rpc_server/mdssvc/mdssvc.c b/source3/rpc_server/mdssvc/mdssvc.c
index 693675fff6f..7111f0e87ca 100644
--- a/source3/rpc_server/mdssvc/mdssvc.c
+++ b/source3/rpc_server/mdssvc/mdssvc.c
@@ -1860,6 +1860,36 @@ bool mds_shutdown(void)
return true;
}
+/**
+ * Tear down connections and free all resources
+ **/
+static int mds_ctx_destructor_cb(struct mds_ctx *mds_ctx)
+{
+ /*
+ * We need to free query_list before ino_path_map
+ */
+ while (mds_ctx->query_list != NULL) {
+ /*
+ * slq destructor removes element from list.
+ * Don't use TALLOC_FREE()!
+ */
+ talloc_free(mds_ctx->query_list);
+ }
+ TALLOC_FREE(mds_ctx->ino_path_map);
+
+ if (mds_ctx->tracker_con != NULL) {
+ g_object_unref(mds_ctx->tracker_con);
+ }
+ if (mds_ctx->gcancellable != NULL) {
+ g_cancellable_cancel(mds_ctx->gcancellable);
+ g_object_unref(mds_ctx->gcancellable);
+ }
+
+ ZERO_STRUCTP(mds_ctx);
+
+ return 0;
+}
+
/**
* Initialise a context per RPC bind
*
@@ -1919,36 +1949,6 @@ struct mds_ctx *mds_init_ctx(TALLOC_CTX *mem_ctx,
return NULL;
}
-/**
- * Tear down connections and free all resources
- **/
-int mds_ctx_destructor_cb(struct mds_ctx *mds_ctx)
-{
- /*
- * We need to free query_list before ino_path_map
- */
- while (mds_ctx->query_list != NULL) {
- /*
- * slq destructor removes element from list.
- * Don't use TALLOC_FREE()!
- */
- talloc_free(mds_ctx->query_list);
- }
- TALLOC_FREE(mds_ctx->ino_path_map);
-
- if (mds_ctx->tracker_con != NULL) {
- g_object_unref(mds_ctx->tracker_con);
- }
- if (mds_ctx->gcancellable != NULL) {
- g_cancellable_cancel(mds_ctx->gcancellable);
- g_object_unref(mds_ctx->gcancellable);
- }
-
- ZERO_STRUCTP(mds_ctx);
-
- return 0;
-}
-
/**
* Dispatch a Spotlight RPC command
**/
diff --git a/source3/rpc_server/mdssvc/mdssvc.h b/source3/rpc_server/mdssvc/mdssvc.h
index b9e3e56f785..baf0e92f8f2 100644
--- a/source3/rpc_server/mdssvc/mdssvc.h
+++ b/source3/rpc_server/mdssvc/mdssvc.h
@@ -129,7 +129,6 @@ extern struct mds_ctx *mds_init_ctx(TALLOC_CTX *mem_ctx,
struct tevent_context *ev,
struct auth_session_info *session_info,
const char *path);
-extern int mds_ctx_destructor_cb(struct mds_ctx *mds_ctx);
extern bool mds_dispatch(struct mds_ctx *query_ctx,
struct mdssvc_blob *request_blob,
struct mdssvc_blob *response_blob);
--
2.17.2
From 4b00f7aa80bd9d70d216ddc3abd38ef256dc7b64 Mon Sep 17 00:00:00 2001
From: Ralph Boehme <slow at samba.org>
Date: Tue, 12 Mar 2019 15:43:57 +0100
Subject: [PATCH 11/11] s3-mdssvc: add a comment to mds_init()
Signed-off-by: Ralph Boehme <slow at samba.org>
---
source3/rpc_server/mdssvc/mdssvc.c | 3 +++
1 file changed, 3 insertions(+)
diff --git a/source3/rpc_server/mdssvc/mdssvc.c b/source3/rpc_server/mdssvc/mdssvc.c
index 7111f0e87ca..58a219bedb0 100644
--- a/source3/rpc_server/mdssvc/mdssvc.c
+++ b/source3/rpc_server/mdssvc/mdssvc.c
@@ -1837,6 +1837,9 @@ static struct mdssvc_ctx *mdssvc_init(struct tevent_context *ev)
/**
* Init callbacks at startup
+ *
+ * This gets typically called in the main parent smbd which means we can't
+ * initialize our global state here.
**/
bool mds_init(struct messaging_context *msg_ctx)
{
--
2.17.2
More information about the samba-technical
mailing list