[PATCH] tevent-glib-glue
Ralph Böhme
slow at samba.org
Fri Mar 22 15:21:34 UTC 2019
Hi Noel,
On Fri, Mar 22, 2019 at 11:08:33AM +0000, Noel Power wrote:
>A little later than I expected. So still I haven't looked at the mdssvc
>parts. About the glue code at least I had a pretty good look.
Thanks a *lot*!
>+ in 'tevent_glib_glue_trace_callback' we currently are calling
>tevent_glib_prepare potentially every trace point so possibly 4 times
>per tevent loop iteration (depending on what fires)
ah, drat, retrofitting something is a recipe for disaster. :( Fixed in attached
patchset.
>+ 'g_main_context_acquire' & 'g_main_context_release' calls need to be
>matched (currently they are not) otherwise the gmain_ctx owner_counter
>will runaway making it impossible for a different thread acquire the context
Also fixed by using an explicit state flag in the glue.
>+ After calling tevent_glib_glue_cleanup we need to re-initialiase/re-alloc
> some glue context items to avoid using freed memory
Fixed.
>
>+ Similarly we need to initialise some additional glue context members,
>num_maps
> definitely needs to be reset to 0 (and probably gtimeout, gpriority etc_
Fixed.
>I haven't had a chance to run your latest patches (or my adjustments to
>those patches) on WSP yet (I will try to do that today)
>
>I tried to run make test TESTS=samba3.tevent_glib_glue with no luck,
>doesn't seem to think the test exists (I don't know what I did wrong :/)
>Note: I can run the test manually from test_tevent_glib_glue.sh
Hm, works here.
>Also I noticed that the tests should run regardless of whether glib_glue
>stuff is built (via fake tevent_glib_glue_test generated in the test) I
>don't love that approach as it looks like the glue test has run and
>passed, I mean the current CI looks like it is running with the
>glib_glue stuff but you need --enable-spotlight for the glib_glue stuff
>to be built and afaics none of the CI (or autobuild) test jobs enable that.
Yeah, good point. I've changed the test runner script to mark the test skipped
if not run.
Also, tevent_glib_glue is now only dependent on glib2-devel, not
--enable-spotlight. Hopefully we can get glib2-devel in the gitlab image, I've
requested this via the mailing list.
All changes are in the FIXUP commits, other commit are unchanged compared to the
previous iteration.
Thanks!
-slow
--
Ralph Boehme, Samba Team https://samba.org/
Samba Developer, SerNet GmbH https://sernet.de/en/samba/
GPG-Fingerprint FAE2C6088A24252051C559E4AA1E9B7126399E46
-------------- next part --------------
From 063df0b0e74f06c8dca88d51f92992bc754511c2 Mon Sep 17 00:00:00 2001
From: Ralph Boehme <slow at samba.org>
Date: Mon, 15 Feb 2016 10:42:52 +0100
Subject: [PATCH 1/9] s3: build: seperate out check for Gnome Tracker from
Spotlight
Signed-off-by: Ralph Boehme <slow at samba.org>
---
source3/wscript | 54 ++++++++++++++++++++++++++++++-------------------
1 file changed, 33 insertions(+), 21 deletions(-)
diff --git a/source3/wscript b/source3/wscript
index e0db9839795..2bc10b3d23e 100644
--- a/source3/wscript
+++ b/source3/wscript
@@ -1623,35 +1623,47 @@ main() {
conf.fatal('AFS headers not available, but --with-fake-kaserver was specified')
conf.env['libtracker']=''
+ tracker_versions = ['2.0', '1.0', '0.16', '0.14']
+
+ for version in tracker_versions:
+ testlib = 'tracker-sparql-' + version
+ if conf.CHECK_CFG(package=testlib,
+ args='--cflags --libs',
+ mandatory=False):
+ conf.SET_TARGET_TYPE(testlib, 'SYSLIB')
+ conf.env['libtracker'] = testlib
+ conf.DEFINE('HAVE_TRACKER', '1')
+ break
+
+ Logs.info("Checking for bison")
+ bison.configure(conf)
+ conf.CHECK_COMMAND('%s --version | head -n1' % conf.env['BISON'],
+ msg='Using bison version',
+ define=None,
+ on_target=False)
+
+ Logs.info("Checking for flex")
+ flex.configure(conf)
+ conf.CHECK_COMMAND('%s --version' % conf.env['FLEX'],
+ msg='Using flex version',
+ define=None,
+ on_target=False)
+
conf.env.with_spotlight = False
if Options.options.with_spotlight:
-
- Logs.info("Requested Spotlight support, checking for bison")
- bison.configure(conf)
if not conf.env['BISON']:
conf.fatal("Spotlight support requested but bison missing")
- conf.CHECK_COMMAND('%s --version | head -n1' % conf.env['BISON'], msg='Using bison version', define=None, on_target=False)
- Logs.info("Requested Spotlight support, checking for flex")
- flex.configure(conf)
+
if not conf.env['FLEX']:
conf.fatal("Spotlight support requested but flex missing")
- conf.CHECK_COMMAND('%s --version' % conf.env['FLEX'], msg='Using flex version', define=None, on_target=False)
- versions = ['2.0', '1.0', '0.16', '0.14']
- for version in versions:
- testlib = 'tracker-sparql-' + version
- if conf.CHECK_CFG(package=testlib,
- args='--cflags --libs',
- mandatory=False):
- conf.SET_TARGET_TYPE(testlib, 'SYSLIB')
- conf.env['libtracker'] = testlib
- conf.env.with_spotlight = True
- conf.DEFINE('WITH_SPOTLIGHT', '1')
- break
-
- if not conf.env.with_spotlight:
- conf.fatal("Spotlight support requested but tracker-sparql library missing")
+
+ if not conf.CONFIG_SET('HAVE_TRACKER'):
+ conf.fatal('Missing Gnome Tracker development files')
+
Logs.info("building with Spotlight support")
default_static_modules.extend(TO_LIST('rpc_mdssvc_module'))
+ conf.DEFINE('WITH_SPOTLIGHT', '1')
+ conf.env.with_spotlight = True
conf.CHECK_HEADERS('rpc/xdr.h', lib='tirpc')
--
2.17.2
From f6e007a58f990186f1eda7a333dc4de51105332b Mon Sep 17 00:00:00 2001
From: Ralph Boehme <slow at samba.org>
Date: Wed, 20 Jan 2016 15:08:31 +0100
Subject: [PATCH 2/9] s3/lib: new tevent_glib_glue subsystem
tevent_glib_glue_create() takes glib GMainContext and adds its event
sources to a tevent context. tevent will poll the sources and run
handlers for pending events as detailed in the glib documentation:
https://developer.gnome.org/glib/stable/glib-The-Main-Event-Loop.html
Signed-off-by: Ralph Boehme <slow at samba.org>
---
source3/lib/tevent_glib_glue.c | 841 +++++++++++++++++++++++++++++++++
source3/lib/tevent_glib_glue.h | 68 +++
source3/wscript | 15 +
source3/wscript_build | 6 +
4 files changed, 930 insertions(+)
create mode 100644 source3/lib/tevent_glib_glue.c
create mode 100644 source3/lib/tevent_glib_glue.h
diff --git a/source3/lib/tevent_glib_glue.c b/source3/lib/tevent_glib_glue.c
new file mode 100644
index 00000000000..97f3fce8a69
--- /dev/null
+++ b/source3/lib/tevent_glib_glue.c
@@ -0,0 +1,841 @@
+/*
+ Unix SMB/CIFS implementation.
+ Integration of a glib g_main_context into a tevent_context
+ Copyright (C) Stefan Metzmacher 2016
+ Copyright (C) Ralph Boehme 2016
+
+ ** NOTE! The following LGPL license applies to the tevent
+ ** library. This does NOT imply that all of Samba is released
+ ** under the LGPL
+
+ This library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 3 of the License, or (at your option) any later version.
+
+ This library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with this library; if not, see <http://www.gnu.org/licenses/>.
+*/
+
+#include "replace.h"
+#include "system/filesys.h"
+#include "lib/util/debug.h"
+#include "lib/util/select.h"
+#include <tevent.h>
+
+#undef DBGC_CLASS
+#define DBGC_CLASS DBGC_TEVENT
+
+#ifdef HAVE_GLIB
+#include <glib.h>
+#include "tevent_glib_glue.h"
+
+struct fd_map {
+ struct tevent_glib_glue *glue;
+ int fd;
+ struct tevent_fd *fd_event;
+};
+
+struct tevent_glib_glue {
+ /*
+ * The tevent context we're feeding.
+ */
+ struct tevent_context *ev;
+
+ /*
+ * The glib gmain context we're polling.
+ */
+ GMainContext *gmain_ctx;
+
+ /*
+ * Set by samba_tevent_glib_glue_quit().
+ */
+ bool quit;
+
+ /*
+ * tevent trace callback and data we got from tevent_get_trace_callback()
+ * before installing our own trace callback.
+ */
+ tevent_trace_callback_t prev_tevent_trace_cb;
+ void *prev_tevent_trace_data;
+
+ /*
+ * Don't call tevent_glib_prepare() in the tevent tracepoint handled if
+ * explicity told so. This is an optimisation for the case that glib
+ * event sources are created from glib event callbacks.
+ */
+ bool skip_glib_refresh;
+
+ /*
+ * Used when acquiring the glib gmain context failed.
+ */
+ struct tevent_timer *acquire_retry_timer;
+
+ /*
+ * glib gmain context timeout and priority for the current event look
+ * iteration. gtimeout is translated to a tevent timer event, unless it
+ * is 0 which signals some event source is pending. In that case we
+ * dispatch an immediate event. gpriority is ignored by us, just passed
+ * to the glib relevant functions.
+ */
+ gint gtimeout;
+ gint gpriority;
+ struct tevent_timer *timer;
+ struct tevent_immediate *im;
+ bool scheduled_im;
+
+ /*
+ * glib gmain context fds returned from g_main_context_query(). These
+ * get translated to tevent fd events.
+ */
+ GPollFD *gpollfds;
+ gint num_gpollfds;
+
+ /*
+ * A copy of gpollfds and num_gpollfds from the previous event loop
+ * iteration, used to detect changes in the set of fds.
+ */
+ GPollFD *prev_gpollfds;
+ gint num_prev_gpollfds;
+
+ /*
+ * An array of pointers to fd_map's. The fd_map'd contain the tevent
+ * event fd as well as a pointer to the corresponding glib GPollFD.
+ */
+ struct fd_map **fd_map;
+ size_t num_maps;
+};
+
+static bool tevent_glib_prepare(struct tevent_glib_glue *glue);
+static bool tevent_glib_process(struct tevent_glib_glue *glue);
+static void tevent_glib_glue_cleanup(struct tevent_glib_glue *glue);
+static void tevent_glib_fd_handler(struct tevent_context *ev,
+ struct tevent_fd *fde,
+ uint16_t flags,
+ void *private_data);
+
+typedef int (*gfds_cmp_cb)(const void *fd1, const void *fd2);
+typedef bool (*gfds_found_cb)(struct tevent_glib_glue *glue,
+ const GPollFD *new,
+ const GPollFD *old);
+typedef bool (*gfds_new_cb)(struct tevent_glib_glue *glue,
+ const GPollFD *fd);
+typedef bool (*gfds_removed_cb)(struct tevent_glib_glue *glue,
+ const GPollFD *fd);
+
+/**
+ * Compare two sorted GPollFD arrays
+ *
+ * For every element that exists in gfds and prev_gfds found_fn() is called.
+ * For every element in gfds but not in prev_gfds, new_fn() is called.
+ * For every element in prev_gfds but not in gfds removed_fn() is called.
+ **/
+static bool cmp_gfds(struct tevent_glib_glue *glue,
+ GPollFD *gfds,
+ GPollFD *prev_gfds,
+ size_t num_gfds,
+ size_t num_prev_gfds,
+ gfds_cmp_cb cmp_cb,
+ gfds_found_cb found_cb,
+ gfds_new_cb new_cb,
+ gfds_removed_cb removed_cb)
+{
+ bool ok;
+ size_t i = 0, j = 0;
+ int cmp;
+
+ while (i < num_gfds && j < num_prev_gfds) {
+ cmp = cmp_cb(&gfds[i], &prev_gfds[j]);
+ if (cmp == 0) {
+ ok = found_cb(glue, &gfds[i], &prev_gfds[j]);
+ if (!ok) {
+ return false;
+ }
+ i++;
+ j++;
+ } else if (cmp < 0) {
+ ok = new_cb(glue, &gfds[i]);
+ if (!ok) {
+ return false;
+ }
+ i++;
+ } else {
+ ok = removed_cb(glue, &prev_gfds[j]);
+ if (!ok) {
+ return false;
+ }
+ j++;
+ }
+ }
+
+ while (i < num_gfds) {
+ ok = new_cb(glue, &gfds[i++]);
+ if (!ok) {
+ return false;
+ }
+ }
+
+ while (j < num_prev_gfds) {
+ ok = removed_cb(glue, &prev_gfds[j++]);
+ if (!ok) {
+ return false;
+ }
+ }
+
+ return true;
+}
+
+static int glib_fd_cmp_func(const void *p1, const void *p2)
+{
+ const GPollFD *lhs = p1;
+ const GPollFD *rhs = p2;
+
+ if (lhs->fd < rhs->fd) {
+ return -1;
+ } else if (lhs->fd > rhs->fd) {
+ return 1;
+ }
+
+ return 0;
+}
+
+/*
+ * We already have a tevent fd event for the glib GPollFD, but we may have to
+ * update flags.
+ */
+static bool match_gfd_cb(struct tevent_glib_glue *glue,
+ const GPollFD *new_gfd,
+ const GPollFD *old_gfd)
+{
+ size_t i;
+ struct fd_map *fd_map = NULL;
+ struct tevent_fd *fd_event = NULL;
+
+ if (new_gfd->events == old_gfd->events) {
+ return true;
+ }
+
+ for (i = 0; i < glue->num_maps; i++) {
+ if (glue->fd_map[i]->fd == new_gfd->fd) {
+ break;
+ }
+ }
+
+ if (i == glue->num_maps) {
+ DBG_ERR("match_gfd_cb: glib fd %d not in map\n", new_gfd->fd);
+ return false;
+ }
+
+ fd_map = glue->fd_map[i];
+ if (fd_map == NULL) {
+ DBG_ERR("fd_map for fd %d is NULL\n", new_gfd->fd);
+ return false;
+ }
+
+ fd_event = fd_map->fd_event;
+ if (fd_event == NULL) {
+ DBG_ERR("fd_event for fd %d is NULL\n", new_gfd->fd);
+ return false;
+ }
+
+ tevent_fd_set_flags(fd_event, 0);
+
+ if (new_gfd->events & (G_IO_IN | G_IO_HUP | G_IO_ERR)) {
+ TEVENT_FD_READABLE(fd_event);
+ }
+ if (new_gfd->events & G_IO_OUT) {
+ TEVENT_FD_WRITEABLE(fd_event);
+ }
+
+ return true;
+}
+
+static bool new_gfd_cb(struct tevent_glib_glue *glue, const GPollFD *gfd)
+{
+ struct tevent_fd *fd_event = NULL;
+ struct fd_map *fd_map = NULL;
+ uint16_t events = 0;
+ bool revent;
+ bool wevent;
+
+ revent = (gfd->events & (G_IO_IN | G_IO_HUP | G_IO_ERR));
+ wevent = (gfd->events & G_IO_OUT);
+ if (revent) {
+ events |= TEVENT_FD_READ;
+ }
+ if (wevent) {
+ events |= TEVENT_FD_WRITE;
+ }
+
+ glue->fd_map = talloc_realloc(glue,
+ glue->fd_map,
+ struct fd_map *,
+ glue->num_maps + 1);
+ if (glue->fd_map == NULL) {
+ DBG_ERR("talloc_realloc failed\n");
+ return false;
+ }
+ fd_map = talloc_zero(glue->fd_map, struct fd_map);
+ if (fd_map == NULL) {
+ DBG_ERR("talloc_realloc failed\n");
+ return false;
+ }
+ glue->fd_map[glue->num_maps] = fd_map;
+ glue->num_maps++;
+
+ fd_event = tevent_add_fd(glue->ev,
+ glue->fd_map,
+ gfd->fd,
+ events,
+ tevent_glib_fd_handler,
+ fd_map);
+ if (fd_event == NULL) {
+ DBG_ERR("tevent_add_fd failed\n");
+ return false;
+ }
+
+ *fd_map = (struct fd_map) {
+ .glue = glue,
+ .fd = gfd->fd,
+ .fd_event = fd_event,
+ };
+
+ DBG_DEBUG("added tevent_fd for glib fd %d\n", gfd->fd);
+
+ return true;
+}
+
+static bool remove_gfd_cb(struct tevent_glib_glue *glue, const GPollFD *gfd)
+{
+ size_t i;
+
+ for (i = 0; i < glue->num_maps; i++) {
+ if (glue->fd_map[i]->fd == gfd->fd) {
+ break;
+ }
+ }
+
+ if (i == glue->num_maps) {
+ DBG_ERR("remove_gfd_cb: glib fd %d not in map\n", gfd->fd);
+ return false;
+ }
+
+ TALLOC_FREE(glue->fd_map[i]->fd_event);
+ TALLOC_FREE(glue->fd_map[i]);
+
+ if (i + 1 < glue->num_maps) {
+ memmove(&glue->fd_map[i],
+ &glue->fd_map[i+1],
+ (glue->num_maps - (i + 1)) * sizeof(struct fd_map *));
+ }
+
+ glue->fd_map = talloc_realloc(glue,
+ glue->fd_map,
+ struct fd_map *,
+ glue->num_maps - 1);
+ if (glue->num_maps > 0 && glue->fd_map == NULL) {
+ DBG_ERR("talloc_realloc failed\n");
+ return false;
+ }
+ glue->num_maps--;
+
+ return true;
+}
+
+static short gpoll_to_poll_event(gushort gevent)
+{
+ short pevent = 0;
+
+ if (gevent & G_IO_IN) {
+ pevent |= POLLIN;
+ }
+ if (gevent & G_IO_OUT) {
+ pevent |= POLLOUT;
+ }
+ if (gevent & G_IO_HUP) {
+ pevent |= POLLHUP;
+ }
+ if (gevent & G_IO_ERR) {
+ pevent |= POLLERR;
+ }
+
+ return pevent;
+}
+
+static gushort poll_to_gpoll_event(short pevent)
+{
+ gushort gevent = 0;
+
+ if (pevent & POLLIN) {
+ gevent |= G_IO_IN;
+ }
+ if (pevent & POLLOUT) {
+ gevent |= G_IO_OUT;
+ }
+ if (pevent & POLLHUP) {
+ gevent |= G_IO_HUP;
+ }
+ if (pevent & POLLERR) {
+ gevent |= G_IO_ERR;
+ }
+
+ return gevent;
+}
+
+static void tevent_glib_fd_handler(struct tevent_context *ev,
+ struct tevent_fd *fde,
+ uint16_t flags,
+ void *private_data)
+{
+ struct fd_map *fd_map = talloc_get_type_abort(
+ private_data, struct fd_map);
+ struct tevent_glib_glue *glue = NULL;
+ GPollFD *gpollfd = NULL;
+ struct pollfd fd;
+ int ret;
+ int i;
+
+ glue = fd_map->glue;
+
+ for (i = 0; i < glue->num_gpollfds; i++) {
+ if (glue->gpollfds[i].fd != fd_map->fd) {
+ continue;
+ }
+ gpollfd = &glue->gpollfds[i];
+ break;
+ }
+ if (gpollfd == NULL) {
+ DBG_ERR("No gpollfd for fd_map [%p] fd [%d]\n",
+ fd_map, fd_map->fd);
+ return;
+ }
+ /*
+ * We have to poll() the fd to get the correct fd event for glib. tevent
+ * only tells us about readable/writable in flags, but we need the full
+ * glory for glib.
+ */
+
+ fd = (struct pollfd) {
+ .fd = gpollfd->fd,
+ .events = gpoll_to_poll_event(gpollfd->events),
+ };
+
+ ret = sys_poll_intr(&fd, 1, 0);
+ if (ret == -1) {
+ DBG_ERR("poll: %s\n", strerror(errno));
+ return;
+ }
+ if (ret == 0) {
+ return;
+ }
+
+ gpollfd->revents = poll_to_gpoll_event(fd.revents);
+
+ tevent_glib_process(glue);
+
+ glue->skip_glib_refresh = true;
+ return;
+}
+
+static void tevent_glib_timer_handler(struct tevent_context *ev,
+ struct tevent_timer *te,
+ struct timeval current_time,
+ void *private_data)
+{
+ struct tevent_glib_glue *glue = talloc_get_type_abort(
+ private_data, struct tevent_glib_glue);
+
+ glue->timer = NULL;
+ tevent_glib_process(glue);
+
+ glue->skip_glib_refresh = true;
+ return;
+}
+
+static void tevent_glib_im_handler(struct tevent_context *ev,
+ struct tevent_immediate *im,
+ void *private_data)
+{
+ struct tevent_glib_glue *glue = talloc_get_type_abort(
+ private_data, struct tevent_glib_glue);
+
+ glue->scheduled_im = false;
+ tevent_glib_process(glue);
+
+ glue->skip_glib_refresh = true;
+ return;
+}
+
+static bool save_current_fdset(struct tevent_glib_glue *glue)
+{
+ /*
+ * Save old glib fds. We only grow the prev array.
+ */
+
+ if (glue->num_prev_gpollfds < glue->num_gpollfds) {
+ glue->prev_gpollfds = talloc_realloc(glue,
+ glue->prev_gpollfds,
+ GPollFD,
+ glue->num_gpollfds);
+ if (glue->prev_gpollfds == NULL) {
+ DBG_ERR("talloc_realloc failed\n");
+ return false;
+ }
+ }
+ glue->num_prev_gpollfds = glue->num_gpollfds;
+ if (glue->num_gpollfds > 0) {
+ memcpy(glue->prev_gpollfds, glue->gpollfds,
+ sizeof(GPollFD) * glue->num_gpollfds);
+ memset(glue->gpollfds, 0, sizeof(GPollFD) * glue->num_gpollfds);
+ }
+
+ return true;
+}
+
+static bool get_glib_fds_and_timeout(struct tevent_glib_glue *glue)
+{
+ bool ok;
+ gint num_fds;
+
+ ok = save_current_fdset(glue);
+ if (!ok) {
+ return false;
+ }
+
+ while (true) {
+ num_fds = g_main_context_query(glue->gmain_ctx,
+ glue->gpriority,
+ &glue->gtimeout,
+ glue->gpollfds,
+ glue->num_gpollfds);
+ if (num_fds == glue->num_gpollfds) {
+ break;
+ }
+ glue->gpollfds = talloc_realloc(glue,
+ glue->gpollfds,
+ GPollFD,
+ num_fds);
+ if (num_fds > 0 && glue->gpollfds == NULL) {
+ DBG_ERR("talloc_realloc failed\n");
+ return false;
+ }
+ glue->num_gpollfds = num_fds;
+ };
+
+ if (glue->num_gpollfds > 0) {
+ qsort(glue->gpollfds,
+ num_fds,
+ sizeof(GPollFD),
+ glib_fd_cmp_func);
+ }
+
+ DBG_DEBUG("num fds: %d, timeout: %d ms\n",
+ num_fds, glue->gtimeout);
+
+ return true;
+}
+
+static bool tevent_glib_update_events(struct tevent_glib_glue *glue)
+{
+ uint64_t microsec;
+ struct timeval tv;
+ bool ok;
+
+ ok = cmp_gfds(glue,
+ glue->gpollfds,
+ glue->prev_gpollfds,
+ glue->num_gpollfds,
+ glue->num_prev_gpollfds,
+ glib_fd_cmp_func,
+ match_gfd_cb,
+ new_gfd_cb,
+ remove_gfd_cb);
+ if (!ok) {
+ return false;
+ }
+
+ TALLOC_FREE(glue->timer);
+
+ if (glue->gtimeout == -1) {
+ return true;
+ }
+
+ if (glue->gtimeout == 0) {
+ /*
+ * glue->gtimeout is 0 if g_main_context_query() returned
+ * timeout=0. That means there are pending events ready to be
+ * dispatched. We only want to run one event handler per loop
+ * iteration, so we schedule an immediate event to run it in the
+ * next iteration.
+ */
+ if (glue->scheduled_im) {
+ return true;
+ }
+ tevent_schedule_immediate(glue->im,
+ glue->ev,
+ tevent_glib_im_handler,
+ glue);
+ glue->scheduled_im = true;
+ return true;
+ }
+
+ microsec = glue->gtimeout * 1000;
+ tv = tevent_timeval_current_ofs(microsec / 1000000,
+ microsec % 1000000);
+
+ glue->timer = tevent_add_timer(glue->ev,
+ glue,
+ tv,
+ tevent_glib_timer_handler,
+ glue);
+ if (glue->timer == NULL) {
+ DBG_ERR("tevent_add_timer failed\n");
+ return false;
+ }
+
+ return true;
+}
+
+static void tevent_glib_retry_timer(struct tevent_context *ev,
+ struct tevent_timer *te,
+ struct timeval current_time,
+ void *private_data)
+{
+ struct tevent_glib_glue *glue = talloc_get_type_abort(
+ private_data, struct tevent_glib_glue);
+
+ glue->acquire_retry_timer = NULL;
+ (void)tevent_glib_prepare(glue);
+}
+
+/**
+ * Fetch glib event sources and add them to tevent
+ *
+ * Fetch glib event sources and attach corresponding tevent events to our tevent
+ * context. get_glib_fds_and_timeout() gets the relevant glib event sources: the
+ * set of active fds and the next timer. tevent_glib_update_events() then
+ * translates those to tevent and creates tevent events.
+ *
+ * When called, the thread must NOT be the owner to the glib main
+ * context. tevent_glib_prepare() is either the first function when the
+ * tevent_glib_glue is created, or after tevent_glib_process() has been called
+ * to process pending event, which will have ceased ownership.
+ **/
+static bool tevent_glib_prepare(struct tevent_glib_glue *glue)
+{
+ bool ok;
+ gboolean gok;
+
+ if (glue->quit) {
+ /* Set via samba_tevent_glib_glue_quit() */
+ return true;
+ }
+
+ if (glue->acquire_retry_timer != NULL) {
+ /*
+ * We're still waiting on the below g_main_context_acquire() to
+ * succeed, just return.
+ */
+ return true;
+ }
+
+ gok = g_main_context_acquire(glue->gmain_ctx);
+ if (!gok) {
+ DBG_ERR("couldn't acquire g_main_context\n");
+
+ /*
+ * Ensure no tevent event fires while we're not the gmain
+ * context owner. The event handler would call
+ * tevent_glib_process() and that expects being the owner of the
+ * context.
+ */
+ tevent_glib_glue_cleanup(glue);
+
+ glue->acquire_retry_timer = tevent_add_timer(
+ glue->ev,
+ glue,
+ tevent_timeval_current_ofs(0, 1000),
+ tevent_glib_retry_timer,
+ glue);
+ if (glue->acquire_retry_timer == NULL) {
+ DBG_ERR("tevent_add_timer failed\n");
+ samba_tevent_glib_glue_quit(glue);
+ return false;
+ }
+ return true;
+ }
+
+ /*
+ * Discard "ready" return value from g_main_context_prepare(). We don't
+ * want to dispatch events here, thats only done in from the tevent loop.
+ */
+ (void)g_main_context_prepare(glue->gmain_ctx, &glue->gpriority);
+
+ ok = get_glib_fds_and_timeout(glue);
+ if (!ok) {
+ return false;
+ }
+
+ ok = tevent_glib_update_events(glue);
+ if (!ok) {
+ DBG_ERR("tevent_glib_update_events failed\n");
+ samba_tevent_glib_glue_quit(glue);
+ return false;
+ }
+
+ return true;
+}
+
+/**
+ * Process pending glib events
+ *
+ * tevent_glib_process() gets called to process pending glib events via
+ * g_main_context_check() and then g_main_context_dispatch().
+ *
+ * After pending event handlers are dispatched, we rearm the glib glue event
+ * handlers in tevent by calling tevent_glib_prepare().
+ *
+ * When tevent_glib_process() is called the thread must own the glib
+ * gmain_ctx. That is achieved by tevent_glib_prepare() being the only function
+ * that acuires context ownership.
+ *
+ * To give other threads that are blocked on g_main_context_acquire(gmain_ctx) a
+ * chance to acquire context ownership (eg needed to attach event sources), we
+ * release context ownership before calling tevent_glib_prepare() which will
+ * acquire it again.
+ */
+static bool tevent_glib_process(struct tevent_glib_glue *glue)
+{
+ bool ok;
+ bool sources_ready;
+
+ DBG_DEBUG("tevent_glib_process\n");
+
+ sources_ready = g_main_context_check(glue->gmain_ctx,
+ glue->gpriority,
+ glue->gpollfds,
+ glue->num_gpollfds);
+ if (sources_ready) {
+ g_main_context_dispatch(glue->gmain_ctx);
+ }
+
+ g_main_context_release(glue->gmain_ctx);
+
+ ok = tevent_glib_prepare(glue);
+ if (!ok) {
+ samba_tevent_glib_glue_quit(glue);
+ return false;
+ }
+
+ return true;
+}
+
+static void tevent_glib_glue_trace_callback(enum tevent_trace_point point,
+ void *private_data)
+{
+ struct tevent_glib_glue *glue = talloc_get_type_abort(
+ private_data, struct tevent_glib_glue);
+
+ /*
+ * Don't call tevent_glib_prepare() if explicity told so. This is an
+ * optimisation for the case that glib event sources are created from
+ * glib event callbacks.
+ */
+ if (!glue->skip_glib_refresh) {
+ tevent_glib_prepare(glue);
+ }
+ glue->skip_glib_refresh = false;
+
+ /* chain previous handler */
+ if (glue->prev_tevent_trace_cb != NULL) {
+ glue->prev_tevent_trace_cb(point, glue->prev_tevent_trace_data);
+ }
+}
+
+static void tevent_glib_glue_cleanup(struct tevent_glib_glue *glue)
+{
+ size_t n = talloc_array_length(glue->fd_map);
+ size_t i;
+
+ for (i = 0; i < n; i++) {
+ TALLOC_FREE(glue->fd_map[i]->fd_event);
+ TALLOC_FREE(glue->fd_map[i]);
+ }
+
+ tevent_set_trace_callback(glue->ev,
+ glue->prev_tevent_trace_cb,
+ glue->prev_tevent_trace_data);
+
+ TALLOC_FREE(glue->fd_map);
+ TALLOC_FREE(glue->gpollfds);
+ TALLOC_FREE(glue->prev_gpollfds);
+ TALLOC_FREE(glue->timer);
+ TALLOC_FREE(glue->acquire_retry_timer);
+ TALLOC_FREE(glue->im);
+ glue->num_gpollfds = 0;
+ glue->num_prev_gpollfds = 0;
+}
+
+void samba_tevent_glib_glue_quit(struct tevent_glib_glue *glue)
+{
+ tevent_glib_glue_cleanup(glue);
+ glue->quit = true;
+ return;
+}
+
+struct tevent_glib_glue *samba_tevent_glib_glue_create(TALLOC_CTX *mem_ctx,
+ struct tevent_context *ev,
+ GMainContext *gmain_ctx)
+{
+ bool ok;
+ struct tevent_glib_glue *glue = NULL;
+
+ glue = talloc_zero(mem_ctx, struct tevent_glib_glue);
+ if (glue == NULL) {
+ DBG_ERR("talloc_zero failed\n");
+ return NULL;
+ }
+
+ *glue = (struct tevent_glib_glue) {
+ .ev = ev,
+ .gmain_ctx = gmain_ctx,
+ };
+
+ glue->im = tevent_create_immediate(glue);
+
+ tevent_get_trace_callback(glue->ev,
+ &glue->prev_tevent_trace_cb,
+ &glue->prev_tevent_trace_data);
+ tevent_set_trace_callback(glue->ev,
+ tevent_glib_glue_trace_callback,
+ glue);
+
+ ok = tevent_glib_prepare(glue);
+ if (!ok) {
+ TALLOC_FREE(glue);
+ return NULL;
+ }
+
+ return glue;
+}
+
+#else /* HAVE_GLIB */
+
+struct tevent_glib_glue *samba_tevent_glib_glue_create(TALLOC_CTX *mem_ctx,
+ struct tevent_context *ev,
+ GMainContext *gmain_ctx)
+{
+ errno = ENOSYS;
+ return NULL;
+}
+
+void samba_tevent_glib_glue_quit(struct tevent_glib_glue *glue)
+{
+ return;
+}
+#endif /* HAVE_GLIB */
diff --git a/source3/lib/tevent_glib_glue.h b/source3/lib/tevent_glib_glue.h
new file mode 100644
index 00000000000..0d001fa8ce9
--- /dev/null
+++ b/source3/lib/tevent_glib_glue.h
@@ -0,0 +1,68 @@
+/*
+ Unix SMB/CIFS implementation.
+ Poll glib event loop from tevent
+
+ Copyright (C) Ralph Boehme 2016
+
+ ** NOTE! The following LGPL license applies to the tevent
+ ** library. This does NOT imply that all of Samba is released
+ ** under the LGPL
+
+ This library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 3 of the License, or (at your option) any later version.
+
+ This library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with this library; if not, see <http://www.gnu.org/licenses/>.
+*/
+
+#ifndef _TEVENT_GLIB_GLUE_H
+#define _TEVENT_GLIB_GLUE_H
+
+#include <talloc.h>
+#include <tevent.h>
+
+/**
+ * @brief Add a glib GmainContext to a tevent context
+ *
+ * tevent will poll the glib event sources and run handlers for
+ * pending events as detailed in the glib documentation:
+ *
+ * https://developer.gnome.org/glib/stable/glib-The-Main-Event-Loop.html
+ *
+ * If tevent was built without glib support, this function will always return
+ * NULL with an error number ENOSYS.
+ *
+ * @param[in] mem_ctx Memory context to use
+ *
+ * @param[in] ev Event context to use
+ *
+ * @param[in] gmain_ctx GMainContext that will be added to tevent
+ *
+ * @return A handle on the glue context that binds the
+ * the GMainContext to tevent. Pass the glue handle to
+ * tevent_glib_glue_quit() in a callback when you want
+ * stop processing glib events.
+ * You must not call talloc_free() on the handle while
+ * the loop is still in use and attached to tevent.
+ */
+struct tevent_glib_glue *samba_tevent_glib_glue_create(TALLOC_CTX *mem_ctx,
+ struct tevent_context *ev,
+ GMainContext *gmain_ctx);
+
+/**
+ * @brief Stop polling a GMainContext
+ *
+ * Used in a callback when you want to stop processing glib events.
+ *
+ * @param[in] glue And tevent_glib_glue handle
+ */
+void samba_tevent_glib_glue_quit(struct tevent_glib_glue *glue);
+
+#endif
diff --git a/source3/wscript b/source3/wscript
index 2bc10b3d23e..e2d38ac211f 100644
--- a/source3/wscript
+++ b/source3/wscript
@@ -1622,6 +1622,17 @@ main() {
else:
conf.fatal('AFS headers not available, but --with-fake-kaserver was specified')
+ if conf.CHECK_CFG(package='glib-2.0',
+ args='--cflags --libs',
+ msg='Checking for glib-2.0',
+ uselib_store="GLIB-2.0"):
+ if (conf.CHECK_HEADERS('glib.h', lib='glib-2.0') and conf.CHECK_LIB('glib-2.0', shlib=True)):
+ conf.DEFINE('HAVE_GLIB', 1)
+
+ if not conf.CONFIG_SET('HAVE_GLIB'):
+ # define an empty subsystem to allow it to be used as an empty dependency
+ conf.SET_TARGET_TYPE('glib-2.0', 'EMPTY')
+
conf.env['libtracker']=''
tracker_versions = ['2.0', '1.0', '0.16', '0.14']
@@ -1660,6 +1671,10 @@ main() {
if not conf.CONFIG_SET('HAVE_TRACKER'):
conf.fatal('Missing Gnome Tracker development files')
+ if not conf.CONFIG_SET('HAVE_GLIB'):
+ conf.fatal('Missing glib-2.0 development files')
+ conf.DEFINE('WITH_TEVENT_GLIB_GLUE', '1')
+
Logs.info("building with Spotlight support")
default_static_modules.extend(TO_LIST('rpc_mdssvc_module'))
conf.DEFINE('WITH_SPOTLIGHT', '1')
diff --git a/source3/wscript_build b/source3/wscript_build
index ed4de978fdc..c28b3e7621c 100644
--- a/source3/wscript_build
+++ b/source3/wscript_build
@@ -1073,6 +1073,12 @@ bld.SAMBA3_SUBSYSTEM('SPOOLSSD',
RPC_SOCK_HELPER
''')
+bld.SAMBA3_SUBSYSTEM('tevent-glib-glue',
+ source='lib/tevent_glib_glue.c',
+ deps='glib-2.0',
+ enabled=bld.CONFIG_SET('WITH_TEVENT_GLIB_GLUE'),
+)
+
########################## BINARIES #################################
bld.SAMBA3_BINARY('smbd/smbd',
--
2.17.2
From ed11b0869600f2d41aab09c931e3a6ea5d9f0e04 Mon Sep 17 00:00:00 2001
From: Ralph Boehme <slow at samba.org>
Date: Fri, 22 Mar 2019 13:23:55 +0100
Subject: [PATCH 3/9] FIXUP tevent_glib_glue.c
---
source3/lib/tevent_glib_glue.c | 89 +++++++++++++++++++++++++---------
source3/wscript | 6 +--
2 files changed, 69 insertions(+), 26 deletions(-)
diff --git a/source3/lib/tevent_glib_glue.c b/source3/lib/tevent_glib_glue.c
index 97f3fce8a69..625a38f3992 100644
--- a/source3/lib/tevent_glib_glue.c
+++ b/source3/lib/tevent_glib_glue.c
@@ -48,9 +48,11 @@ struct tevent_glib_glue {
struct tevent_context *ev;
/*
- * The glib gmain context we're polling.
+ * The glib gmain context we're polling and whether we're currently
+ * owning it by virtue of g_main_context_acquire().
*/
GMainContext *gmain_ctx;
+ bool gmain_owner;
/*
* Set by samba_tevent_glib_glue_quit().
@@ -65,7 +67,7 @@ struct tevent_glib_glue {
void *prev_tevent_trace_data;
/*
- * Don't call tevent_glib_prepare() in the tevent tracepoint handled if
+ * Don't call tevent_glib_prepare() in the tevent tracepoint handler if
* explicity told so. This is an optimisation for the case that glib
* event sources are created from glib event callbacks.
*/
@@ -113,7 +115,7 @@ struct tevent_glib_glue {
static bool tevent_glib_prepare(struct tevent_glib_glue *glue);
static bool tevent_glib_process(struct tevent_glib_glue *glue);
-static void tevent_glib_glue_cleanup(struct tevent_glib_glue *glue);
+static bool tevent_glib_glue_reinit(struct tevent_glib_glue *glue);
static void tevent_glib_fd_handler(struct tevent_context *ev,
struct tevent_fd *fde,
uint16_t flags,
@@ -437,8 +439,6 @@ static void tevent_glib_fd_handler(struct tevent_context *ev,
gpollfd->revents = poll_to_gpoll_event(fd.revents);
tevent_glib_process(glue);
-
- glue->skip_glib_refresh = true;
return;
}
@@ -452,8 +452,6 @@ static void tevent_glib_timer_handler(struct tevent_context *ev,
glue->timer = NULL;
tevent_glib_process(glue);
-
- glue->skip_glib_refresh = true;
return;
}
@@ -466,8 +464,6 @@ static void tevent_glib_im_handler(struct tevent_context *ev,
glue->scheduled_im = false;
tevent_glib_process(glue);
-
- glue->skip_glib_refresh = true;
return;
}
@@ -644,6 +640,11 @@ static bool tevent_glib_prepare(struct tevent_glib_glue *glue)
return true;
}
+ if (glue->gmain_owner) {
+ g_main_context_release(glue->gmain_ctx);
+ glue->gmain_owner = false;
+ }
+
gok = g_main_context_acquire(glue->gmain_ctx);
if (!gok) {
DBG_ERR("couldn't acquire g_main_context\n");
@@ -654,7 +655,12 @@ static bool tevent_glib_prepare(struct tevent_glib_glue *glue)
* tevent_glib_process() and that expects being the owner of the
* context.
*/
- tevent_glib_glue_cleanup(glue);
+ ok = tevent_glib_glue_reinit(glue);
+ if (!ok) {
+ DBG_ERR("tevent_glib_glue_reinit failed\n");
+ samba_tevent_glib_glue_quit(glue);
+ return false;
+ }
glue->acquire_retry_timer = tevent_add_timer(
glue->ev,
@@ -669,6 +675,7 @@ static bool tevent_glib_prepare(struct tevent_glib_glue *glue)
}
return true;
}
+ glue->gmain_owner = true;
/*
* Discard "ready" return value from g_main_context_prepare(). We don't
@@ -678,6 +685,8 @@ static bool tevent_glib_prepare(struct tevent_glib_glue *glue)
ok = get_glib_fds_and_timeout(glue);
if (!ok) {
+ DBG_ERR("get_glib_fds_and_timeout failed\n");
+ samba_tevent_glib_glue_quit(glue);
return false;
}
@@ -724,14 +733,11 @@ static bool tevent_glib_process(struct tevent_glib_glue *glue)
g_main_context_dispatch(glue->gmain_ctx);
}
- g_main_context_release(glue->gmain_ctx);
-
ok = tevent_glib_prepare(glue);
if (!ok) {
- samba_tevent_glib_glue_quit(glue);
return false;
}
-
+ glue->skip_glib_refresh = true;
return true;
}
@@ -740,13 +746,20 @@ static void tevent_glib_glue_trace_callback(enum tevent_trace_point point,
{
struct tevent_glib_glue *glue = talloc_get_type_abort(
private_data, struct tevent_glib_glue);
+ bool do_glib_refresh = false;
- /*
- * Don't call tevent_glib_prepare() if explicity told so. This is an
- * optimisation for the case that glib event sources are created from
- * glib event callbacks.
- */
- if (!glue->skip_glib_refresh) {
+ if (point == TEVENT_TRACE_AFTER_LOOP_ONCE) {
+ do_glib_refresh = true;
+ }
+ if (glue->skip_glib_refresh) {
+ /*
+ * Don't call tevent_glib_prepare() if explicity told so. This
+ * is an optimisation for the case that glib event sources are
+ * created from glib event callbacks.
+ */
+ do_glib_refresh = false;
+ }
+ if (do_glib_refresh) {
tevent_glib_prepare(glue);
}
glue->skip_glib_refresh = false;
@@ -770,15 +783,47 @@ static void tevent_glib_glue_cleanup(struct tevent_glib_glue *glue)
tevent_set_trace_callback(glue->ev,
glue->prev_tevent_trace_cb,
glue->prev_tevent_trace_data);
+ glue->prev_tevent_trace_cb = NULL;
+ glue->prev_tevent_trace_data = NULL;
TALLOC_FREE(glue->fd_map);
+ glue->num_maps = 0;
+
TALLOC_FREE(glue->gpollfds);
+ glue->num_gpollfds = 0;
+
TALLOC_FREE(glue->prev_gpollfds);
+ glue->num_prev_gpollfds = 0;
+
TALLOC_FREE(glue->timer);
TALLOC_FREE(glue->acquire_retry_timer);
TALLOC_FREE(glue->im);
- glue->num_gpollfds = 0;
- glue->num_prev_gpollfds = 0;
+
+ /*
+ * These are not really needed, but let's wipe the slate clean.
+ */
+ glue->skip_glib_refresh = false;
+ glue->gtimeout = 0;
+ glue->gpriority = 0;
+}
+
+static bool tevent_glib_glue_reinit(struct tevent_glib_glue *glue)
+{
+ tevent_glib_glue_cleanup(glue);
+
+ glue->im = tevent_create_immediate(glue);
+ if (glue->im == NULL) {
+ return false;
+ }
+
+ tevent_get_trace_callback(glue->ev,
+ &glue->prev_tevent_trace_cb,
+ &glue->prev_tevent_trace_data);
+ tevent_set_trace_callback(glue->ev,
+ tevent_glib_glue_trace_callback,
+ glue);
+
+ return true;
}
void samba_tevent_glib_glue_quit(struct tevent_glib_glue *glue)
diff --git a/source3/wscript b/source3/wscript
index e2d38ac211f..2a585966d90 100644
--- a/source3/wscript
+++ b/source3/wscript
@@ -1629,9 +1629,8 @@ main() {
if (conf.CHECK_HEADERS('glib.h', lib='glib-2.0') and conf.CHECK_LIB('glib-2.0', shlib=True)):
conf.DEFINE('HAVE_GLIB', 1)
- if not conf.CONFIG_SET('HAVE_GLIB'):
- # define an empty subsystem to allow it to be used as an empty dependency
- conf.SET_TARGET_TYPE('glib-2.0', 'EMPTY')
+ if conf.CONFIG_SET('HAVE_GLIB'):
+ conf.DEFINE('WITH_TEVENT_GLIB_GLUE', '1')
conf.env['libtracker']=''
tracker_versions = ['2.0', '1.0', '0.16', '0.14']
@@ -1673,7 +1672,6 @@ main() {
if not conf.CONFIG_SET('HAVE_GLIB'):
conf.fatal('Missing glib-2.0 development files')
- conf.DEFINE('WITH_TEVENT_GLIB_GLUE', '1')
Logs.info("building with Spotlight support")
default_static_modules.extend(TO_LIST('rpc_mdssvc_module'))
--
2.17.2
From cfd418584f348f9e76408addcf77958088b481cf Mon Sep 17 00:00:00 2001
From: Ralph Boehme <slow at samba.org>
Date: Thu, 28 Jan 2016 08:29:28 +0100
Subject: [PATCH 4/9] s3/lib: add a tevent_glib_glue subsystem test
Tests adapted from glib2 glib/tests/mainloop.c.
Signed-off-by: Ralph Boehme <slow at samba.org>
---
source3/lib/tevent_glib_glue_tests.c | 397 ++++++++++++++++++
source3/script/tests/test_tevent_glib_glue.sh | 20 +
source3/selftest/tests.py | 3 +
source3/wscript_build | 10 +
4 files changed, 430 insertions(+)
create mode 100644 source3/lib/tevent_glib_glue_tests.c
create mode 100755 source3/script/tests/test_tevent_glib_glue.sh
diff --git a/source3/lib/tevent_glib_glue_tests.c b/source3/lib/tevent_glib_glue_tests.c
new file mode 100644
index 00000000000..d6cf66462f6
--- /dev/null
+++ b/source3/lib/tevent_glib_glue_tests.c
@@ -0,0 +1,397 @@
+/*
+ Unix SMB/CIFS implementation.
+
+ testing of the tevent glib glue subsystem
+
+ Copyright (C) Ralph Boehme 2016
+
+ glib tests adapted from glib2 glib/tests/mainloop.c
+ Copyright (C) 2011 Red Hat Inc., Matthias Clasen
+
+ ** NOTE! The following LGPL license applies to the tevent
+ ** library. This does NOT imply that all of Samba is released
+ ** under the LGPL
+
+ This library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 3 of the License, or (at your option) any later version.
+
+ This library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with this library; if not, see <http://www.gnu.org/licenses/>.
+*/
+
+#include "replace.h"
+
+/*
+ * glib uses TRUE and FALSE which may have redefined by "includes.h" to be
+ * unusable. Unndefine so glib can establish its own working replacement.
+ */
+#undef TRUE
+#undef FALSE
+#include <glib.h>
+#include <glib-unix.h>
+#include "lib/tevent_glib_glue.h"
+
+/*
+ * Unfortunately the glib test suite runner doesn't pass args to tests
+ * so we must keep a few globals here.
+ */
+static struct tevent_context *ev;
+
+static gboolean count_calls(gpointer data)
+{
+ gint *i = (gint *)data;
+
+ (*i)++;
+
+ return TRUE;
+}
+
+static gboolean quit_loop(gpointer data)
+{
+ struct tevent_glib_glue *glue = talloc_get_type_abort(
+ data, struct tevent_glib_glue);
+
+ samba_tevent_glib_glue_quit(glue);
+
+ return G_SOURCE_REMOVE;
+}
+
+static void test_timeouts(void)
+{
+ GMainContext *ctx = NULL;
+ struct tevent_glib_glue *glue = NULL;
+ GSource *source = NULL;
+ gint a;
+ gint b;
+ gint c;
+
+ a = b = c = 0;
+
+ ctx = g_main_context_new();
+ glue = samba_tevent_glib_glue_create(ev, ev, ctx);
+ g_assert(glue != NULL);
+
+ source = g_timeout_source_new(100);
+ g_source_set_callback(source, count_calls, &a, NULL);
+ g_source_attach(source, ctx);
+ g_source_unref(source);
+
+ source = g_timeout_source_new(250);
+ g_source_set_callback(source, count_calls, &b, NULL);
+ g_source_attach(source, ctx);
+ g_source_unref(source);
+
+ source = g_timeout_source_new(330);
+ g_source_set_callback(source, count_calls, &c, NULL);
+ g_source_attach(source, ctx);
+ g_source_unref(source);
+
+ source = g_timeout_source_new(1050);
+ g_source_set_callback(source, quit_loop, glue, NULL);
+ g_source_attach(source, ctx);
+ g_source_unref(source);
+
+ g_assert(tevent_loop_wait(ev) == 0);
+
+ /* We may be delayed for an arbitrary amount of time - for example,
+ * it's possible for all timeouts to fire exactly once.
+ */
+ g_assert_cmpint(a, >, 0);
+ g_assert_cmpint(a, >=, b);
+ g_assert_cmpint(b, >=, c);
+
+ g_assert_cmpint(a, <=, 10);
+ g_assert_cmpint(b, <=, 4);
+ g_assert_cmpint(c, <=, 3);
+
+ samba_tevent_glib_glue_quit(glue);
+ TALLOC_FREE(glue);
+ g_main_context_unref(ctx);
+}
+
+struct test_glib_ev_source_data {
+ GMainContext *ctx;
+ struct tevent_glib_glue *glue;
+};
+
+static gboolean test_glib_ev_source_quit_loop(gpointer data);
+
+static gboolean test_glib_ev_source_timeout_cb(gpointer data)
+{
+ struct test_glib_ev_source_data *state = talloc_get_type_abort(
+ data, struct test_glib_ev_source_data);
+ GSource *source = NULL;
+
+ source = g_timeout_source_new(100);
+ g_source_set_callback(source,
+ test_glib_ev_source_quit_loop,
+ state,
+ NULL);
+ g_source_attach(source, state->ctx);
+ g_source_unref(source);
+
+ return TRUE;
+}
+
+static gboolean test_glib_ev_source_quit_loop(gpointer data)
+{
+ struct test_glib_ev_source_data *state = talloc_get_type_abort(
+ data, struct test_glib_ev_source_data);
+
+ samba_tevent_glib_glue_quit(state->glue);
+
+ return G_SOURCE_REMOVE;
+}
+
+static void test_glib_ev_source(void)
+{
+ GMainContext *ctx = NULL;
+ struct tevent_glib_glue *glue = NULL;
+ struct test_glib_ev_source_data *state = NULL;
+ GSource *source = NULL;
+
+ ctx = g_main_context_new();
+ g_assert(ctx != NULL);
+
+ glue = samba_tevent_glib_glue_create(ev, ev, ctx);
+ g_assert(glue != NULL);
+
+ state = talloc_zero(glue, struct test_glib_ev_source_data);
+ g_assert(state != NULL);
+
+ state->ctx = ctx;
+ state->glue = glue;
+
+ source = g_timeout_source_new(100);
+ g_source_set_callback(source,
+ test_glib_ev_source_timeout_cb,
+ state,
+ NULL);
+ g_source_attach(source, ctx);
+ g_source_unref(source);
+
+ g_assert(tevent_loop_wait(ev) == 0);
+
+ TALLOC_FREE(glue);
+ g_main_context_unref(ctx);
+}
+
+struct test_tevent_ev_source_data {
+ GMainContext *ctx;
+ struct tevent_glib_glue *glue;
+};
+
+static gboolean test_tevent_ev_source_quit_loop(gpointer data);
+
+static void test_tevent_ev_source_timeout_cb(struct tevent_context *_ev,
+ struct tevent_timer *te,
+ struct timeval current_time,
+ void *data)
+{
+ struct test_tevent_ev_source_data *state = talloc_get_type_abort(
+ data, struct test_tevent_ev_source_data);
+ GSource *source = NULL;
+
+ source = g_timeout_source_new(100);
+ g_source_set_callback(source,
+ test_tevent_ev_source_quit_loop,
+ state,
+ NULL);
+ g_source_attach(source, state->ctx);
+ g_source_unref(source);
+
+ return;
+}
+
+static gboolean test_tevent_ev_source_quit_loop(gpointer data)
+{
+ struct test_tevent_ev_source_data *state = talloc_get_type_abort(
+ data, struct test_tevent_ev_source_data);
+
+ samba_tevent_glib_glue_quit(state->glue);
+
+ return G_SOURCE_REMOVE;
+}
+
+static void test_tevent_ev_source(void)
+{
+ GMainContext *ctx = NULL;
+ struct tevent_glib_glue *glue = NULL;
+ struct test_tevent_ev_source_data *state = NULL;
+ struct tevent_timer *timer = NULL;
+
+ ctx = g_main_context_new();
+ g_assert(ctx != NULL);
+
+ glue = samba_tevent_glib_glue_create(ev, ev, ctx);
+ g_assert(glue != NULL);
+
+ state = talloc_zero(glue, struct test_tevent_ev_source_data);
+ g_assert(state != NULL);
+
+ state->ctx = ctx;
+ state->glue = glue;
+
+ timer = tevent_add_timer(ev,
+ state,
+ tevent_timeval_current_ofs(0, 1000),
+ test_tevent_ev_source_timeout_cb,
+ state);
+ g_assert(timer != NULL);
+
+ g_assert(tevent_loop_wait(ev) == 0);
+
+ TALLOC_FREE(glue);
+ g_main_context_unref(ctx);
+}
+
+static gchar zeros[1024];
+
+static gsize fill_a_pipe(gint fd)
+{
+ gsize written = 0;
+ GPollFD pfd;
+
+ pfd.fd = fd;
+ pfd.events = G_IO_OUT;
+ while (g_poll(&pfd, 1, 0) == 1)
+ /* we should never see -1 here */
+ written += write(fd, zeros, sizeof zeros);
+
+ return written;
+}
+
+static gboolean write_bytes(gint fd,
+ GIOCondition condition,
+ gpointer user_data)
+{
+ gssize *to_write = user_data;
+ gint limit;
+
+ if (*to_write == 0)
+ return FALSE;
+
+ /* Detect if we run before we should */
+ g_assert(*to_write >= 0);
+
+ limit = MIN(*to_write, sizeof zeros);
+ *to_write -= write(fd, zeros, limit);
+
+ return TRUE;
+}
+
+static gboolean read_bytes(gint fd,
+ GIOCondition condition,
+ gpointer user_data)
+{
+ static gchar buffer[1024];
+ gssize *to_read = user_data;
+
+ *to_read -= read(fd, buffer, sizeof buffer);
+
+ /* The loop will exit when there is nothing else to read, then we will
+ * use g_source_remove() to destroy this source.
+ */
+ return TRUE;
+}
+
+static void test_unix_fd(void)
+{
+ gssize to_write = -1;
+ gssize to_read;
+ gint fds[2];
+ gint a, b;
+ gint s;
+ GSource *source_a = NULL;
+ GSource *source_b = NULL;
+ struct tevent_glib_glue *glue = NULL;
+
+ glue = samba_tevent_glib_glue_create(ev, ev, g_main_context_default());
+ g_assert(glue != NULL);
+
+ s = pipe(fds);
+ g_assert(s == 0);
+
+ to_read = fill_a_pipe(fds[1]);
+ /* write at higher priority to keep the pipe full... */
+ a = g_unix_fd_add_full(G_PRIORITY_HIGH,
+ fds[1],
+ G_IO_OUT,
+ write_bytes,
+ &to_write,
+ NULL);
+ source_a = g_source_ref(g_main_context_find_source_by_id(NULL, a));
+ /* make sure no 'writes' get dispatched yet */
+ while (tevent_loop_once(ev));
+
+ to_read += 128 * 1024 * 1024;
+ to_write = 128 * 1024 * 1024;
+ b = g_unix_fd_add(fds[0], G_IO_IN, read_bytes, &to_read);
+ source_b = g_source_ref(g_main_context_find_source_by_id(NULL, b));
+
+ /* Assuming the kernel isn't internally 'laggy' then there will always
+ * be either data to read or room in which to write. That will keep
+ * the loop running until all data has been read and written.
+ */
+ while (to_write > 0 || to_read > 0)
+ {
+ gssize to_write_was = to_write;
+ gssize to_read_was = to_read;
+
+ if (tevent_loop_once(ev) != 0)
+ break;
+
+ /* Since the sources are at different priority, only one of them
+ * should possibly have run.
+ */
+ g_assert(to_write == to_write_was || to_read == to_read_was);
+ }
+
+ g_assert(to_write == 0);
+ g_assert(to_read == 0);
+
+ /* 'a' is already removed by itself */
+ g_assert(g_source_is_destroyed(source_a));
+ g_source_unref(source_a);
+ g_source_remove(b);
+ g_assert(g_source_is_destroyed(source_b));
+ g_source_unref(source_b);
+
+ samba_tevent_glib_glue_quit(glue);
+ TALLOC_FREE(glue);
+
+ close(fds[1]);
+ close(fds[0]);
+}
+
+int main(int argc, const char *argv[])
+{
+ int test_argc = 3;
+ char *test_argv[] = {
+ discard_const("test_glib_glue"),
+ discard_const("-m"),
+ discard_const("no-undefined")
+ };
+ char **argvp = test_argv;
+
+ g_test_init(&test_argc, &argvp, NULL);
+
+ ev = tevent_context_init(NULL);
+ if (ev == NULL) {
+ exit(1);
+ }
+
+ g_test_add_func("/mainloop/timeouts", test_timeouts);
+ g_test_add_func("/mainloop/glib_ev_source", test_glib_ev_source);
+ g_test_add_func("/mainloop/tevent_ev_source", test_tevent_ev_source);
+ g_test_add_func("/mainloop/unix-fd", test_unix_fd);
+
+ return g_test_run();
+}
diff --git a/source3/script/tests/test_tevent_glib_glue.sh b/source3/script/tests/test_tevent_glib_glue.sh
new file mode 100755
index 00000000000..67544946ec9
--- /dev/null
+++ b/source3/script/tests/test_tevent_glib_glue.sh
@@ -0,0 +1,20 @@
+#!/bin/sh
+
+incdir=`dirname $0`/../../../testprogs/blackbox
+. $incdir/subunit.sh
+
+if [ ! -x $BINDIR/tevent_glib_glue_test ] ; then
+ # Some machines don't have /bin/true, simulate it
+ cat >$BINDIR/tevent_glib_glue_test <<EOF
+#!/bin/sh
+exit 0
+EOF
+ chmod +x $BINDIR/tevent_glib_glue_test
+fi
+
+failed=0
+
+testit "tevent_glib_glue_test" $VALGRIND $BINDIR/tevent_glib_glue_test ||
+ failed=`expr $failed + 1`
+
+testok $0 $failed
diff --git a/source3/selftest/tests.py b/source3/selftest/tests.py
index e8d516573dd..c6917219ab0 100755
--- a/source3/selftest/tests.py
+++ b/source3/selftest/tests.py
@@ -415,6 +415,9 @@ plantestsuite(
"samba3.resolvconf", "none",
[os.path.join(samba3srcdir, "script/tests/test_resolvconf.sh")])
+plantestsuite("samba3.tevent_glib_glue", "none",
+ [os.path.join(samba3srcdir, "script/tests/test_tevent_glib_glue.sh")])
+
plantestsuite("samba3.async_req", "nt4_dc",
[os.path.join(samba3srcdir, "script/tests/test_async_req.sh")])
diff --git a/source3/wscript_build b/source3/wscript_build
index c28b3e7621c..0a4e858d28e 100644
--- a/source3/wscript_build
+++ b/source3/wscript_build
@@ -1345,6 +1345,16 @@ bld.SAMBA3_BINARY('spotlight2sparql',
enabled=bld.env.with_spotlight,
install=False)
+bld.SAMBA3_BINARY('tevent_glib_glue_test',
+ source='lib/tevent_glib_glue_tests.c',
+ deps='''
+ talloc
+ libsmb
+ popt_samba3
+ tevent-glib-glue''',
+ enabled=bld.CONFIG_SET('WITH_TEVENT_GLIB_GLUE'),
+ install=False)
+
########################## INCLUDES #################################
bld.RECURSE('auth')
--
2.17.2
From 0b91fa3f2e5a7a8a46ce351e5b87f8a695715a6f Mon Sep 17 00:00:00 2001
From: Ralph Boehme <slow at samba.org>
Date: Fri, 22 Mar 2019 16:00:42 +0100
Subject: [PATCH 5/9] FIXUP: skip test if glib2-devel is missing
---
source3/script/tests/test_tevent_glib_glue.sh | 15 ++++++++-------
1 file changed, 8 insertions(+), 7 deletions(-)
diff --git a/source3/script/tests/test_tevent_glib_glue.sh b/source3/script/tests/test_tevent_glib_glue.sh
index 67544946ec9..ce8cb3db0c1 100755
--- a/source3/script/tests/test_tevent_glib_glue.sh
+++ b/source3/script/tests/test_tevent_glib_glue.sh
@@ -3,18 +3,19 @@
incdir=`dirname $0`/../../../testprogs/blackbox
. $incdir/subunit.sh
+failed=0
+TESTNAME="tevent_glib_glue_test"
+
if [ ! -x $BINDIR/tevent_glib_glue_test ] ; then
- # Some machines don't have /bin/true, simulate it
- cat >$BINDIR/tevent_glib_glue_test <<EOF
-#!/bin/sh
-exit 0
+ subunit_start_test "$TESTNAME"
+ subunit_skip_test "$TESTNAME" <<EOF
+Test needs glib2-devel
EOF
- chmod +x $BINDIR/tevent_glib_glue_test
+ testok $0 $failed
fi
-failed=0
-testit "tevent_glib_glue_test" $VALGRIND $BINDIR/tevent_glib_glue_test ||
+testit "$TESTNAME" $VALGRIND $BINDIR/tevent_glib_glue_test ||
failed=`expr $failed + 1`
testok $0 $failed
--
2.17.2
From c5269ea041f0b4b3f26a23cc0c639c6e23528400 Mon Sep 17 00:00:00 2001
From: Ralph Boehme <slow at samba.org>
Date: Fri, 22 Jan 2016 15:38:39 +0100
Subject: [PATCH 6/9] s3/lib: tevent-glib-glue test utiltity with Tracker
A small utilitly useful for tesing the tevent_glib_glue code. It runs a
tracker-sparql search query against your local tracker store that must
be setup and running.
Signed-off-by: Ralph Boehme <slow at samba.org>
---
source3/utils/async-tracker.c | 288 ++++++++++++++++++++++++++++++++++
source3/wscript_build | 10 ++
2 files changed, 298 insertions(+)
create mode 100644 source3/utils/async-tracker.c
diff --git a/source3/utils/async-tracker.c b/source3/utils/async-tracker.c
new file mode 100644
index 00000000000..4ccf26f907e
--- /dev/null
+++ b/source3/utils/async-tracker.c
@@ -0,0 +1,288 @@
+/*
+ * Copyright (C) 2011, Nokia <ivan.frade at nokia.com>
+ * Copyright (C) 2015, Noel Power <nopower at suse.com>
+ * Copyright (C) 2016, Ralph Boehme <slow at samba.org.>
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public
+ * License along with this library; if not, write to the
+ * Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor,
+ * Boston, MA 02110-1301, USA.
+ */
+
+#include "includes.h"
+#include "lib/util/debug.h"
+#include "popt_common.h"
+#include "param.h"
+/*
+ * glib uses TRUE and FALSE which was redefined by "includes.h" to be
+ * unusable, undefine so glib can establish its own working
+ * replacement.
+ */
+#undef TRUE
+#undef FALSE
+#include <glib.h>
+#include <libtracker-sparql/tracker-sparql.h>
+#include "lib/tevent_glib_glue.h"
+
+enum loop_type {TEVENT_LOOP, GLIB_LOOP};
+
+struct test_state {
+ enum loop_type loop_type;
+ TrackerSparqlConnection *connection;
+ GCancellable *cancellable;
+ GTimer *timer;
+ GMainLoop *loop;
+ struct tevent_context *ev;
+ struct tevent_glib_glue *glue;
+};
+
+static void cleanup(struct test_state *state)
+{
+ g_cancellable_cancel(state->cancellable);
+ g_object_unref(state->cancellable);
+ g_timer_destroy(state->timer);
+ if (state->connection != NULL) {
+ g_object_unref(state->connection);
+ state->connection = NULL;
+ }
+ if (state->loop_type == GLIB_LOOP) {
+ g_main_loop_quit(state->loop);
+ } else {
+ samba_tevent_glib_glue_quit(state->glue);
+ }
+}
+
+static void cursor_cb(GObject *object,
+ GAsyncResult *res,
+ gpointer user_data)
+{
+ struct test_state *state = talloc_get_type_abort(
+ user_data, struct test_state);
+ TrackerSparqlCursor *cursor = NULL;
+ GError *error = NULL;
+ gboolean more_results;
+ static gint i = 0;
+
+ cursor = TRACKER_SPARQL_CURSOR(object);
+ more_results = tracker_sparql_cursor_next_finish(cursor,
+ res,
+ &error);
+ if (error) {
+ g_critical("Could not run cursor next: %s", error->message);
+
+ if (cursor != NULL) {
+ g_object_unref(cursor);
+ }
+
+ g_error_free(error);
+ cleanup(state);
+ return;
+ }
+
+ if (!more_results) {
+ g_print("\n");
+ g_print("\nAsync cursor next took: %.6f (for all %d results)\n",
+ g_timer_elapsed (state->timer, NULL), i);
+
+ g_object_unref(cursor);
+ cleanup(state);
+ return;
+ }
+
+ if (i++ < 5) {
+ int num_cols = tracker_sparql_cursor_get_n_columns(cursor);
+ int col;
+
+ if (i == 1) {
+ g_print("Printing first 5 results:\n");
+ }
+ for (col = 0; col < num_cols; col++) {
+ g_print(" %s ", tracker_sparql_cursor_get_string(
+ cursor, col, NULL));
+ if (col == num_cols -1 ) {
+ g_print("\n");
+ }
+ }
+
+ if (i == 5) {
+ g_print(" ...\n");
+ g_print(" Printing nothing for remaining results\n");
+ }
+ }
+
+ tracker_sparql_cursor_next_async(cursor,
+ state->cancellable,
+ cursor_cb,
+ state);
+}
+
+static void query_cb(GObject *object,
+ GAsyncResult *res,
+ gpointer user_data)
+{
+ struct test_state *state = talloc_get_type_abort(
+ user_data, struct test_state);
+ TrackerSparqlCursor *cursor = NULL;
+ GError *error = NULL;
+
+ g_print("Async query took: %.6f\n", g_timer_elapsed(state->timer, NULL));
+
+ cursor = tracker_sparql_connection_query_finish(
+ TRACKER_SPARQL_CONNECTION(object),
+ res,
+ &error);
+ if (error) {
+ g_critical("Could not run query: %s", error->message);
+
+ if (cursor) {
+ g_object_unref(cursor);
+ }
+
+ g_error_free(error);
+ cleanup(state);
+ return;
+ }
+
+ g_timer_start(state->timer);
+
+ tracker_sparql_cursor_next_async(cursor,
+ state->cancellable,
+ cursor_cb,
+ state);
+}
+
+static void connection_cb(GObject *object,
+ GAsyncResult *res,
+ gpointer user_data)
+{
+ struct test_state *state = talloc_get_type_abort(
+ user_data, struct test_state);
+ GError *error = NULL;
+
+ g_print("Async connection took: %.6f\n",
+ g_timer_elapsed(state->timer, NULL));
+
+ state->connection = tracker_sparql_connection_get_finish(res, &error);
+ if (error) {
+ g_critical("Could not connect: %s", error->message);
+ g_error_free(error);
+ cleanup(state);
+ return;
+ }
+
+ g_timer_start(state->timer);
+
+ tracker_sparql_connection_query_async(
+ state->connection,
+ "SELECT ?name nie:mimeType(?s) nfo:fileName(?s) "
+ "WHERE { {?s nie:url ?name}}",
+ state->cancellable,
+ query_cb,
+ state);
+}
+
+static void debug_fn(void *private_data,
+ enum tevent_debug_level level,
+ const char *fmt,
+ va_list ap)
+{
+ dbgtext_va(fmt, ap);
+}
+
+int main(int argc, const char **argv)
+{
+ TALLOC_CTX *mem_ctx = NULL;
+ struct test_state *state = NULL;
+ int c;
+ poptContext pc;
+ struct poptOption long_options[] = {
+ POPT_AUTOHELP
+ {"tevent", 't', POPT_ARG_NONE, NULL, 't', "Use tevent loop" },
+ {"glib", 'g', POPT_ARG_NONE, NULL, 'g', "Use glib loop" },
+ POPT_COMMON_SAMBA
+ POPT_TABLEEND
+ };
+
+ mem_ctx = talloc_new(NULL);
+ if (mem_ctx == NULL) {
+ exit(1);
+ }
+
+ state = talloc_zero(mem_ctx, struct test_state);
+ if (state == NULL) {
+ exit(1);
+ }
+
+ state->loop_type = TEVENT_LOOP;
+
+ setup_logging(argv[0], DEBUG_STDERR);
+ smb_init_locale();
+
+ if (!lp_load_client(get_dyn_CONFIGFILE())) {
+ fprintf(stderr, "ERROR: Can't load %s\n",
+ get_dyn_CONFIGFILE());
+ exit(1);
+ }
+
+ pc = poptGetContext(NULL, argc, argv, long_options,
+ POPT_CONTEXT_KEEP_FIRST);
+
+ while ((c = poptGetNextOpt(pc)) != -1) {
+ switch (c) {
+ case 'g':
+ state->loop_type = GLIB_LOOP;
+ break;
+ case 't':
+ state->loop_type = TEVENT_LOOP;
+ break;
+ }
+ }
+
+ if (state->loop_type == GLIB_LOOP) {
+ state->loop = g_main_loop_new(NULL, false);
+ } else {
+ state->ev = tevent_context_init(mem_ctx);
+ if (CHECK_DEBUGLVL(10)) {
+ tevent_set_debug(state->ev, debug_fn, NULL);
+ }
+ state->glue = samba_tevent_glib_glue_create(
+ mem_ctx, state->ev, g_main_context_default());
+ if (state->glue == NULL) {
+ printf("tevent_glib_glue_create failed\n");
+ exit(1);
+ }
+ }
+
+ state->timer = g_timer_new();
+ state->cancellable = g_cancellable_new();
+
+ tracker_sparql_connection_get_async(state->cancellable,
+ connection_cb,
+ state);
+
+ if (state->loop_type == GLIB_LOOP) {
+ printf("entering g_main_loop_run\n");
+ g_main_loop_run(state->loop);
+ } else {
+ printf("entering tevent_loop_wait\n");
+ tevent_loop_wait(state->ev);
+
+ TALLOC_FREE(state->glue);
+ TALLOC_FREE(state->ev);
+ }
+
+ TALLOC_FREE(mem_ctx);
+ poptFreeContext(pc);
+
+ return 0;
+}
diff --git a/source3/wscript_build b/source3/wscript_build
index 0a4e858d28e..347ed0f8a87 100644
--- a/source3/wscript_build
+++ b/source3/wscript_build
@@ -1355,6 +1355,16 @@ bld.SAMBA3_BINARY('tevent_glib_glue_test',
enabled=bld.CONFIG_SET('WITH_TEVENT_GLIB_GLUE'),
install=False)
+bld.SAMBA3_BINARY('tevent_glib_tracker',
+ source='utils/async-tracker.c',
+ deps='''
+ talloc
+ libsmb
+ popt_samba3
+ tevent-glib-glue ''' + bld.env['libtracker'],
+ enabled=bld.CONFIG_SET('HAVE_TRACKER') and bld.CONFIG_SET('WITH_TEVENT_GLIB_GLUE'),
+ install=False)
+
########################## INCLUDES #################################
bld.RECURSE('auth')
--
2.17.2
From cab05b609d4a8dc9fb513b34858b42622bc83799 Mon Sep 17 00:00:00 2001
From: Ralph Boehme <slow at samba.org>
Date: Wed, 27 Jan 2016 13:17:04 +0100
Subject: [PATCH 7/9] s3-mdssvc: add tevent context arg to mds_init_ctx
This is needed later when adding tevent_glib_glue support, not used for now.
Signed-off-by: Ralph Boehme <slow at samba.org>
---
source3/rpc_server/mdssvc/mdssvc.c | 1 +
source3/rpc_server/mdssvc/mdssvc.h | 1 +
source3/rpc_server/mdssvc/srv_mdssvc_nt.c | 6 +++++-
3 files changed, 7 insertions(+), 1 deletion(-)
diff --git a/source3/rpc_server/mdssvc/mdssvc.c b/source3/rpc_server/mdssvc/mdssvc.c
index 7c996941cfa..ea676f48681 100644
--- a/source3/rpc_server/mdssvc/mdssvc.c
+++ b/source3/rpc_server/mdssvc/mdssvc.c
@@ -1822,6 +1822,7 @@ static gboolean gmainloop_timer(gpointer user_data)
* Initialise a context per share handle
**/
struct mds_ctx *mds_init_ctx(TALLOC_CTX *mem_ctx,
+ struct tevent_context *ev,
const struct auth_session_info *session_info,
const char *path)
{
diff --git a/source3/rpc_server/mdssvc/mdssvc.h b/source3/rpc_server/mdssvc/mdssvc.h
index 4e08fe18667..310e59582ed 100644
--- a/source3/rpc_server/mdssvc/mdssvc.h
+++ b/source3/rpc_server/mdssvc/mdssvc.h
@@ -118,6 +118,7 @@ struct mds_ctx {
extern bool mds_init(struct messaging_context *msg_ctx);
extern bool mds_shutdown(void);
extern struct mds_ctx *mds_init_ctx(TALLOC_CTX *mem_ctx,
+ struct tevent_context *ev,
const struct auth_session_info *session_info,
const char *path);
extern int mds_ctx_destructor_cb(struct mds_ctx *mds_ctx);
diff --git a/source3/rpc_server/mdssvc/srv_mdssvc_nt.c b/source3/rpc_server/mdssvc/srv_mdssvc_nt.c
index ab64423d0df..c4bf995ce14 100644
--- a/source3/rpc_server/mdssvc/srv_mdssvc_nt.c
+++ b/source3/rpc_server/mdssvc/srv_mdssvc_nt.c
@@ -18,6 +18,7 @@
*/
#include "includes.h"
+#include "messages.h"
#include "ntdomain.h"
#include "rpc_server/rpc_service_setup.h"
#include "rpc_server/rpc_config.h"
@@ -114,7 +115,10 @@ static NTSTATUS create_mdssvc_policy_handle(TALLOC_CTX *mem_ctx,
ZERO_STRUCTP(handle);
- mds_ctx = mds_init_ctx(mem_ctx, p->session_info, path);
+ mds_ctx = mds_init_ctx(mem_ctx,
+ messaging_tevent_context(p->msg_ctx),
+ p->session_info,
+ path);
if (mds_ctx == NULL) {
DEBUG(1, ("error in mds_init_ctx for: %s\n", path));
return NT_STATUS_UNSUCCESSFUL;
--
2.17.2
From 1df08bc48e7987c01546332b6c782bd270bb113a Mon Sep 17 00:00:00 2001
From: Ralph Boehme <slow at samba.org>
Date: Mon, 11 Mar 2019 18:11:04 +0100
Subject: [PATCH 8/9] s3-mdssvc: call [un]become_authenticated_pipe_user()
This ensures we're running as the authenticated user int the tevent
callback which might be running in an arbitrary impersonation context.
Signed-off-by: Ralph Boehme <slow at samba.org>
---
source3/rpc_server/mdssvc/mdssvc.c | 23 ++++++++++++++++++++++-
source3/rpc_server/mdssvc/mdssvc.h | 3 ++-
2 files changed, 24 insertions(+), 2 deletions(-)
diff --git a/source3/rpc_server/mdssvc/mdssvc.c b/source3/rpc_server/mdssvc/mdssvc.c
index ea676f48681..f03cb9d7ebd 100644
--- a/source3/rpc_server/mdssvc/mdssvc.c
+++ b/source3/rpc_server/mdssvc/mdssvc.c
@@ -810,20 +810,39 @@ static void tracker_cursor_cb(GObject *object,
return;
}
+ /*
+ * We're in a tevent callback which means in the case of
+ * running as external RPC service we're running as root and
+ * not as the user.
+ */
+ if (!become_authenticated_pipe_user(slq->mds_ctx->pipe_session_info)) {
+ DBG_ERR("can't become authenticated user: %d\n", slq->mds_ctx->uid);
+ smb_panic("can't become authenticated user");
+ }
+
if (geteuid() != slq->mds_ctx->uid) {
DEBUG(0, ("uid mismatch: %d/%d\n", geteuid(), slq->mds_ctx->uid));
smb_panic("uid mismatch");
}
+ /*
+ * We've changed identity to the authenticated pipe user, so
+ * any function exit below must ensure we switch back
+ */
+
result = sys_stat(path, &sb, false);
if (result != 0) {
+ unbecome_authenticated_pipe_user();
goto done;
}
result = access(path, R_OK);
if (result != 0) {
+ unbecome_authenticated_pipe_user();
goto done;
}
+ unbecome_authenticated_pipe_user();
+
ino64 = sb.st_ex_ino;
if (slq->cnids) {
/*
@@ -1823,7 +1842,7 @@ static gboolean gmainloop_timer(gpointer user_data)
**/
struct mds_ctx *mds_init_ctx(TALLOC_CTX *mem_ctx,
struct tevent_context *ev,
- const struct auth_session_info *session_info,
+ struct auth_session_info *session_info,
const char *path)
{
struct mds_ctx *mds_ctx;
@@ -1839,6 +1858,8 @@ struct mds_ctx *mds_init_ctx(TALLOC_CTX *mem_ctx,
goto error;
}
+ mds_ctx->pipe_session_info = session_info;
+
if (session_info->security_token->num_sids < 1) {
goto error;
}
diff --git a/source3/rpc_server/mdssvc/mdssvc.h b/source3/rpc_server/mdssvc/mdssvc.h
index 310e59582ed..e1b1ccfc34d 100644
--- a/source3/rpc_server/mdssvc/mdssvc.h
+++ b/source3/rpc_server/mdssvc/mdssvc.h
@@ -97,6 +97,7 @@ struct sl_inode_path_map {
};
struct mds_ctx {
+ struct auth_session_info *pipe_session_info;
struct dom_sid sid;
uid_t uid;
const char *spath;
@@ -119,7 +120,7 @@ extern bool mds_init(struct messaging_context *msg_ctx);
extern bool mds_shutdown(void);
extern struct mds_ctx *mds_init_ctx(TALLOC_CTX *mem_ctx,
struct tevent_context *ev,
- const struct auth_session_info *session_info,
+ struct auth_session_info *session_info,
const char *path);
extern int mds_ctx_destructor_cb(struct mds_ctx *mds_ctx);
extern bool mds_dispatch(struct mds_ctx *query_ctx,
--
2.17.2
From b5ac0a5b4bf85f0e37cfd6976a2a4ad2c57f577f Mon Sep 17 00:00:00 2001
From: Ralph Boehme <slow at samba.org>
Date: Wed, 27 Jan 2016 13:23:51 +0100
Subject: [PATCH 9/9] s3-mdssvc: use tevent_glib_glue in mdssvc RPC service
Signed-off-by: Ralph Boehme <slow at samba.org>
---
source3/rpc_server/mdssvc/mdssvc.c | 206 +++++++++--------------------
source3/rpc_server/mdssvc/mdssvc.h | 11 +-
source3/rpc_server/wscript_build | 4 +-
3 files changed, 75 insertions(+), 146 deletions(-)
diff --git a/source3/rpc_server/mdssvc/mdssvc.c b/source3/rpc_server/mdssvc/mdssvc.c
index f03cb9d7ebd..c8545a52e55 100644
--- a/source3/rpc_server/mdssvc/mdssvc.c
+++ b/source3/rpc_server/mdssvc/mdssvc.c
@@ -28,6 +28,7 @@
#include "libcli/security/dom_sid.h"
#include "mdssvc.h"
#include "rpc_server/mdssvc/sparql_parser.tab.h"
+#include "lib/tevent_glib_glue.h"
#undef DBGC_CLASS
#define DBGC_CLASS DBGC_RPC_SRV
@@ -64,6 +65,15 @@ struct slq_destroy_state {
struct sl_query *slq;
};
+/*
+ * This is a static global because we may be called multiple times and
+ * we only want one mdssvc_ctx per connection to Tracker.
+ *
+ * The client will bind multiple times to the mdssvc RPC service, once
+ * for every tree connect.
+ */
+static struct mdssvc_ctx *mdssvc_ctx = NULL;
+
/*
* If these functions return an error, they hit something like a non
* recoverable talloc error. Most errors are dealt with by returning
@@ -731,7 +741,6 @@ static void tracker_con_cb(GObject *object,
}
DEBUG(10, ("connected to Tracker\n"));
- g_main_loop_quit(mds_ctx->gmainloop);
}
static void tracker_cursor_cb_destroy_done(struct tevent_req *subreq);
@@ -770,7 +779,6 @@ static void tracker_cursor_cb(GObject *object,
* we return.
*/
SLQ_DEBUG(10, slq, "closed");
- g_main_loop_quit(slq->mds_ctx->gmainloop);
req = slq_destroy_send(slq, global_event_context(), &slq);
if (req == NULL) {
@@ -785,13 +793,11 @@ static void tracker_cursor_cb(GObject *object,
DEBUG(1, ("Tracker cursor: %s\n", error->message));
g_error_free(error);
slq->state = SLQ_STATE_ERROR;
- g_main_loop_quit(slq->mds_ctx->gmainloop);
return;
}
if (!more_results) {
slq->state = SLQ_STATE_DONE;
- g_main_loop_quit(slq->mds_ctx->gmainloop);
return;
}
@@ -799,14 +805,12 @@ static void tracker_cursor_cb(GObject *object,
if (uri == NULL) {
DEBUG(1, ("error fetching Tracker URI\n"));
slq->state = SLQ_STATE_ERROR;
- g_main_loop_quit(slq->mds_ctx->gmainloop);
return;
}
path = tracker_to_unix_path(slq->query_results, uri);
if (path == NULL) {
DEBUG(1, ("error converting Tracker URI to path: %s\n", uri));
slq->state = SLQ_STATE_ERROR;
- g_main_loop_quit(slq->mds_ctx->gmainloop);
return;
}
@@ -866,7 +870,6 @@ static void tracker_cursor_cb(GObject *object,
if (result != 0) {
DEBUG(1, ("dalloc error\n"));
slq->state = SLQ_STATE_ERROR;
- g_main_loop_quit(slq->mds_ctx->gmainloop);
return;
}
ok = add_filemeta(slq->reqinfo, slq->query_results->fm_array,
@@ -874,7 +877,6 @@ static void tracker_cursor_cb(GObject *object,
if (!ok) {
DEBUG(1, ("add_filemeta error\n"));
slq->state = SLQ_STATE_ERROR;
- g_main_loop_quit(slq->mds_ctx->gmainloop);
return;
}
@@ -882,7 +884,6 @@ static void tracker_cursor_cb(GObject *object,
if (!ok) {
DEBUG(1, ("inode_map_add error\n"));
slq->state = SLQ_STATE_ERROR;
- g_main_loop_quit(slq->mds_ctx->gmainloop);
return;
}
@@ -892,7 +893,6 @@ static void tracker_cursor_cb(GObject *object,
if (slq->query_results->num_results >= MAX_SL_RESULTS) {
slq->state = SLQ_STATE_FULL;
SLQ_DEBUG(10, slq, "full");
- g_main_loop_quit(slq->mds_ctx->gmainloop);
return;
}
@@ -929,13 +929,11 @@ static void tracker_query_cb(GObject *object,
slq->state = SLQ_STATE_ERROR;
DEBUG(1, ("Tracker query error: %s\n", error->message));
g_error_free(error);
- g_main_loop_quit(slq->mds_ctx->gmainloop);
return;
}
if (slq->state == SLQ_STATE_DONE) {
SLQ_DEBUG(10, slq, "done");
- g_main_loop_quit(slq->mds_ctx->gmainloop);
talloc_free(slq);
return;
}
@@ -1302,13 +1300,11 @@ static bool slrpc_open_query(struct mds_ctx *mds_ctx,
DEBUG(10, ("SPARQL query: \"%s\"\n", slq->sparql_query));
- g_main_context_push_thread_default(mds_ctx->gcontext);
tracker_sparql_connection_query_async(mds_ctx->tracker_con,
slq->sparql_query,
slq->gcancellable,
tracker_query_cb,
slq);
- g_main_context_pop_thread_default(mds_ctx->gcontext);
slq->state = SLQ_STATE_RUNNING;
sl_result = 0;
@@ -1401,13 +1397,11 @@ static bool slrpc_fetch_query_results(struct mds_ctx *mds_ctx,
}
if (slq->state == SLQ_STATE_FULL) {
slq->state = SLQ_STATE_RESULTS;
- g_main_context_push_thread_default(mds_ctx->gcontext);
tracker_sparql_cursor_next_async(
slq->tracker_cursor,
slq->gcancellable,
tracker_cursor_cb,
slq);
- g_main_context_pop_thread_default(mds_ctx->gcontext);
}
break;
@@ -1811,6 +1805,42 @@ static bool slrpc_close_query(struct mds_ctx *mds_ctx,
return true;
}
+static struct mdssvc_ctx *mdssvc_init(struct tevent_context *ev)
+{
+ if (mdssvc_ctx != NULL) {
+ return mdssvc_ctx;
+ }
+
+ mdssvc_ctx = talloc_zero(ev, struct mdssvc_ctx);
+ if (mdssvc_ctx == NULL) {
+ return NULL;
+ }
+
+ mdssvc_ctx->ev_ctx = ev;
+
+ mdssvc_ctx->gmain_ctx = g_main_context_new();
+ if (mdssvc_ctx->gmain_ctx == NULL) {
+ DBG_ERR("error from g_main_context_new\n");
+ return NULL;
+ }
+
+ /*
+ * This ensures all glib threads, especially gioi worker threads
+ * dispatch their async callbacks via our gmain_ctx.
+ */
+ g_main_context_push_thread_default(mdssvc_ctx->gmain_ctx);
+
+ mdssvc_ctx->glue = samba_tevent_glib_glue_create(ev,
+ mdssvc_ctx->ev_ctx,
+ mdssvc_ctx->gmain_ctx);
+ if (mdssvc_ctx->glue == NULL) {
+ DBG_ERR("samba_tevent_glib_glue_create failed\n");
+ return NULL;
+ }
+
+ return mdssvc_ctx;
+}
+
/**
* Init callbacks at startup
**/
@@ -1824,21 +1854,25 @@ bool mds_init(struct messaging_context *msg_ctx)
bool mds_shutdown(void)
{
- return true;
-}
+ if (mdssvc_ctx == NULL) {
+ return false;
+ }
-static gboolean gmainloop_timer(gpointer user_data)
-{
- struct mds_ctx *ctx = talloc_get_type_abort(user_data, struct mds_ctx);
+ samba_tevent_glib_glue_quit(mdssvc_ctx->glue);
+ TALLOC_FREE(mdssvc_ctx->glue);
- DEBUG(10,("%s\n", __func__));
- g_main_loop_quit(ctx->gmainloop);
+ g_main_context_pop_thread_default(mdssvc_ctx->gmain_ctx);
- return G_SOURCE_CONTINUE;
+ TALLOC_FREE(mdssvc_ctx);
+
+ return true;
}
/**
- * Initialise a context per share handle
+ * Initialise a context per RPC bind
+ *
+ * This ends up being called for every tcon, because the client does a
+ * RPC bind for every tcon, so this is acually a per tcon context.
**/
struct mds_ctx *mds_init_ctx(TALLOC_CTX *mem_ctx,
struct tevent_context *ev,
@@ -1853,6 +1887,11 @@ struct mds_ctx *mds_init_ctx(TALLOC_CTX *mem_ctx,
}
talloc_set_destructor(mds_ctx, mds_ctx_destructor_cb);
+ mds_ctx->mdssvc_ctx = mdssvc_init(ev);
+ if (mds_ctx->mdssvc_ctx == NULL) {
+ goto error;
+ }
+
mds_ctx->spath = talloc_strdup(mds_ctx, path);
if (mds_ctx->spath == NULL) {
goto error;
@@ -1872,22 +1911,8 @@ struct mds_ctx *mds_init_ctx(TALLOC_CTX *mem_ctx,
goto error;
}
- mds_ctx->gcontext = g_main_context_new();
- if (mds_ctx->gcontext == NULL) {
- DEBUG(1,("error from g_main_context_new\n"));
- goto error;
- }
-
- mds_ctx->gmainloop = g_main_loop_new(mds_ctx->gcontext, false);
- if (mds_ctx->gmainloop == NULL) {
- DEBUG(1,("error from g_main_loop_new\n"));
- goto error;
- }
-
- g_main_context_push_thread_default(mds_ctx->gcontext);
tracker_sparql_connection_get_async(mds_ctx->gcancellable,
tracker_con_cb, mds_ctx);
- g_main_context_pop_thread_default(mds_ctx->gcontext);
return mds_ctx;
@@ -1920,76 +1945,12 @@ int mds_ctx_destructor_cb(struct mds_ctx *mds_ctx)
g_cancellable_cancel(mds_ctx->gcancellable);
g_object_unref(mds_ctx->gcancellable);
}
- if (mds_ctx->gmainloop != NULL) {
- g_main_loop_unref(mds_ctx->gmainloop);
- }
- if (mds_ctx->gcontext != NULL) {
- g_main_context_unref(mds_ctx->gcontext);
- }
ZERO_STRUCTP(mds_ctx);
return 0;
}
-static bool mds_run_gmainloop(struct mds_ctx *mds_ctx, guint timeout)
-{
- guint timer_id;
- GSource *timer;
-
- /*
- * It seems the event processing of the libtracker-sparql
- * async subsystem defers callbacks until *all* events are
- * processes by the async subsystem main processing loop.
- *
- * g_main_context_iteration(may_block=FALSE) can't be used,
- * because a search that produces a few thousand matches
- * generates as many events that must be processed in either
- * g_main_context_iteration() or g_main_loop_run() before
- * callbacks are called.
- *
- * Unfortunately g_main_context_iteration() only processes a
- * small subset of these event (1-30) at a time when run in
- * mds_dispatch(), which happens once a second while the
- * client polls for results.
- *
- * Carefully using the blocking g_main_loop_run() fixes
- * this. It processes events until we exit from the loop at
- * defined exit points. By adding a 1 ms timeout we at least
- * try to get as close as possible to non-blocking behaviour.
- */
-
- if (!g_main_context_pending(mds_ctx->gcontext)) {
- return true;
- }
-
- g_main_context_push_thread_default(mds_ctx->gcontext);
-
- timer = g_timeout_source_new(timeout);
- if (timer == NULL) {
- DEBUG(1,("g_timeout_source_new_seconds\n"));
- g_main_context_pop_thread_default(mds_ctx->gcontext);
- return false;
- }
-
- timer_id = g_source_attach(timer, mds_ctx->gcontext);
- if (timer_id == 0) {
- DEBUG(1,("g_timeout_add failed\n"));
- g_source_destroy(timer);
- g_main_context_pop_thread_default(mds_ctx->gcontext);
- return false;
- }
-
- g_source_set_callback(timer, gmainloop_timer, mds_ctx, NULL);
-
- g_main_loop_run(mds_ctx->gmainloop);
-
- g_source_destroy(timer);
-
- g_main_context_pop_thread_default(mds_ctx->gcontext);
- return true;
-}
-
/**
* Dispatch a Spotlight RPC command
**/
@@ -2014,34 +1975,6 @@ bool mds_dispatch(struct mds_ctx *mds_ctx,
response_blob->length = 0;
- /*
- * Process finished glib events.
- *
- * FIXME: integrate with tevent instead of piggy packing it
- * onto the processing of new requests.
- *
- * mds_dispatch() is called by the client a few times in a row:
- *
- * - first in order to open/start a search query
- *
- * - later in order to fetch results asynchronously, typically
- * once a second. If no results have been retrieved from the
- * search store (Tracker) yet, we return no results.
- * The client asks for more results every second as long
- * as the "Search Window" in the client gui is open.
- *
- * - at some point the query is closed
- *
- * This means we try to iterate through the glib event loop
- * before processing the request in order to get result
- * from tracker which can be returned to the client.
- */
-
- ok = mds_run_gmainloop(mds_ctx, MDS_TRACKER_ASYNC_TIMEOUT_MS);
- if (!ok) {
- goto cleanup;
- }
-
DEBUG(10, ("share path: %s\n", mds_ctx->spath));
query = dalloc_new(mds_ctx);
@@ -2102,17 +2035,6 @@ bool mds_dispatch(struct mds_ctx *mds_ctx,
goto cleanup;
}
- /*
- * Run g_main_loop a second time in order to dispatch events
- * that may have been queued at the libtracker-sparql level.
- * As we only want to dispatch (write out requests) but not
- * wait for anything, we use a much shorter timeout here.
- */
- ok = mds_run_gmainloop(mds_ctx, MDS_TRACKER_ASYNC_TIMEOUT_MS / 10);
- if (!ok) {
- goto cleanup;
- }
-
response_blob->length = len;
cleanup:
diff --git a/source3/rpc_server/mdssvc/mdssvc.h b/source3/rpc_server/mdssvc/mdssvc.h
index e1b1ccfc34d..b9e3e56f785 100644
--- a/source3/rpc_server/mdssvc/mdssvc.h
+++ b/source3/rpc_server/mdssvc/mdssvc.h
@@ -96,15 +96,22 @@ struct sl_inode_path_map {
char *path;
};
+/* Per process state */
+struct mdssvc_ctx {
+ struct tevent_context *ev_ctx;
+ GMainContext *gmain_ctx;
+ struct tevent_glib_glue *glue;
+};
+
+/* Per tree connect state */
struct mds_ctx {
+ struct mdssvc_ctx *mdssvc_ctx;
struct auth_session_info *pipe_session_info;
struct dom_sid sid;
uid_t uid;
const char *spath;
GCancellable *gcancellable;
TrackerSparqlConnection *tracker_con;
- GMainContext *gcontext;
- GMainLoop *gmainloop;
struct sl_query *query_list; /* list of active queries */
struct db_context *ino_path_map; /* dbwrap rbt for storing inode->path mappings */
};
diff --git a/source3/rpc_server/wscript_build b/source3/rpc_server/wscript_build
index ae75e567ace..354f6050ff6 100644
--- a/source3/rpc_server/wscript_build
+++ b/source3/rpc_server/wscript_build
@@ -150,7 +150,7 @@ bld.SAMBA3_MODULE('rpc_mdssvc_module',
mdssvc/srv_mdssvc_nt.c
../../librpc/gen_ndr/srv_mdssvc.c''',
init_function='',
- deps='samba-util ' + bld.env['libtracker'],
+ deps='samba-util tevent-glib-glue ' + bld.env['libtracker'],
internal_module=bld.SAMBA3_IS_STATIC_MODULE('rpc_mdssvc_module'),
enabled=bld.SAMBA3_IS_ENABLED_MODULE('rpc_mdssvc_module'))
@@ -205,5 +205,5 @@ bld.SAMBA3_SUBSYSTEM('FSSD',
bld.SAMBA3_SUBSYSTEM('MDSSD',
source='mdssd.c',
- deps='RPC_SOCK_HELPER RPC_MODULES samba-util',
+ deps='RPC_SOCK_HELPER RPC_MODULES samba-util tevent-glib-glue',
enabled=bld.env.with_spotlight)
--
2.17.2
More information about the samba-technical
mailing list