[RFC] [WIP] tevent/glib integration
Noel Power
nopower at suse.com
Wed Dec 16 12:44:16 UTC 2015
Hi
I started experimenting with trying to integrate glib with tevent,
specifically my goal is to be able to use tracker-sparql asynchronously
without the need for funky polling or iterating mechanisms. Generally
it's not a good idea to run multiple event loops simultaneously, it's
best if one loop is 'master' and the other event loop's events are
somehow integrated into the 'master' loop. Also I don't really want (if
possible) to have a solution specifically tailored for tracker-sparql
but rather have something that would allow other GLib/GObject library
based functionality to be used in a similar way. GLib provides the
following functions for interation with an external event loops
g_main_context_prepare(), g_main_context_query(),
g_main_context_check() and g_main_context_dispatch()
To effectively use these functions you need to be able to access the
underlying 'poll/select/epoll' functionality to pass foreign fds and
timeouts, Why stop at glib, there are other event loops out there too so
I tried to work on a generic solution.
attached and available at
http://cgit.freedesktop.org/~noelp/noelp-samba/log/?h=glib-integration-wip&qt=range&q=17bc0fab767e59976c6360739d31aa9ad2febc5b...eae9982000eaf1bd4345900e561fd09de64d96b0
are a set of patches showing what I am doing (well trying anyway)
The idea is quite simple, allow 2 callback functions to be registered
'prepare_poll' & 'check_poll'
+ 'prepare_poll' allows a set of file descriptors and associated
timeout to be provided to the underlying poll/epoll/select function used
by tevent
+ 'check_poll' tells the results of the poll
Note: the 'foreign' file descriptors although passed to the same 'poll'
function call as any existing tevent file descriptors are processed
separately
prepare_poll (ask for fds to monitor and timeout for poll)
|
poll/select/epoll
|
process foriegn fds (examine results for monitored events etc.)
|
check_poll (tell the polll is complete passing some result)
|
normal tevent fd processing (continue normal tevent processing)
Each tevent backend needs to be modified to call the 'prepare_poll' &
'check_poll' functions
All the required glue code needed to provide the specific event loop
integration should be performed in 'prepare-poll' & 'check_poll'
e.g for glib;
+ 'prepare_poll' calls ' g_main_context_prepare()' &
'g_main_context_query()'
+ 'check_poll' calls 'g_main_context_check()' and
'g_main_context_dispatch()'
The set of patches provides
+ 2 modified tevent backends
a) the 'poll' backend in the tevent library, I only modified one
for demonstration and testing, without discussion seemed no point in
doing the others at this point
b) the 's3' backend in samba
+ Standalone example 'source3/utils/async-tracker.c it produces a test
binary ./bin/default/source3/tevent_glib
./bin/default/source3/tevent_glib -h
tevent_glib [-t|-h|-g]
-t use tevent event loop
-g use glib event loop (default)
-h help
+ a single function 'tevent_integrate_glib' which can be used with
processes that use the 's3' tevent backend to enable glib integration
if you want to try it out apply the patches (they should apply to
current master) you need to a configure with '--enable-spotlight' in
order to be able to run the example above *and* enable the 's3' glib
event integration, it should build without it (but will only provide the
foreign file descriptor 'poll' integration and no glib event integration)
If you have got this far, well done
some things to consider
a) this is experimental and *very* rough and ready still
b) I only discovered the 's3' backend after doing the work in the tevent
library, so it would be possible to not expose any of this in the tevent
library at all but instead just implement glib integration (in the same
way) internally in samba (would that be preferred ?)
thanks and looking forward to some comments
Noel
p.s. Ralph I *really* hope you are interested in this and would be
willing to test this against your spotlight stuff
-------------- next part --------------
From f809b67c130bebfae051e0133ed8493fca7d00ef Mon Sep 17 00:00:00 2001
From: Noel Power <noel.power at suse.com>
Date: Tue, 15 Dec 2015 12:19:45 +0000
Subject: [PATCH 1/9] add hooks and stubs for integrating foreign fds into
poll/select/epoll backends
Although tevent provides a mechanism (tevent_add_fd) to add a foreign
file descriptor into it's event loop there are some limitations
a) you can't influence the timeout of the underlying poll/select/epoll
mechanism used
b) you can only monitor READ/WRITE events for fds under the control of
tevent, ERROR and HUP events are either converted, ignored or treated
as internal errors
c) you are informed of the events on the fds one at a time (and possibly
interspersed with normal tevent fd events, timer or signal events)
There are situations where an application using tevent might want to monitor
fds without the limitations above and specifically be gauranteed the result
of a set of fds passed to the underlying 'poll/select/epoll' is returned
(immediately or as close to after the poll/select/epoll). One such situation
where this behaviour is desireable is when an application want's to integrate
a foreign event loop (such as glib)
---
lib/tevent/pytevent.c | 10 ++++++++++
lib/tevent/tevent.c | 8 ++++++++
lib/tevent/tevent.h | 45 ++++++++++++++++++++++++++++++++++++++++++++
lib/tevent/tevent_epoll.c | 10 ++++++++++
lib/tevent/tevent_internal.h | 7 +++++++
lib/tevent/tevent_poll.c | 11 +++++++++++
lib/tevent/tevent_port.c | 8 ++++++++
lib/tevent/tevent_select.c | 10 ++++++++++
lib/tevent/tevent_standard.c | 1 +
9 files changed, 110 insertions(+)
diff --git a/lib/tevent/pytevent.c b/lib/tevent/pytevent.c
index 10d8a22..7d0593c 100644
--- a/lib/tevent/pytevent.c
+++ b/lib/tevent/pytevent.c
@@ -162,6 +162,15 @@ static int py_loop_wait(struct tevent_context *ev, const char *location)
return 0;
}
+static bool py_add_foreign_fd_handler(struct tevent_context *ev,
+ struct foreign_loop_handler *integrator,
+ void *private_data,
+ const char *location)
+{
+ /* FIXME */
+ return false;
+}
+
const static struct tevent_ops py_tevent_ops = {
.context_init = py_context_init,
.add_fd = py_add_fd,
@@ -173,6 +182,7 @@ const static struct tevent_ops py_tevent_ops = {
.add_signal = py_add_signal,
.loop_wait = py_loop_wait,
.loop_once = py_loop_once,
+ .add_foreign_fd_handler = py_add_foreign_fd_handler,
};
static PyObject *py_register_backend(PyObject *self, PyObject *args)
diff --git a/lib/tevent/tevent.c b/lib/tevent/tevent.c
index 843cf05..71cf0c1 100644
--- a/lib/tevent/tevent.c
+++ b/lib/tevent/tevent.c
@@ -656,6 +656,14 @@ int _tevent_loop_wait(struct tevent_context *ev, const char *location)
return ev->ops->loop_wait(ev, location);
}
+bool _tevent_add_foreign_fd_handler(struct tevent_context *ev,
+ struct foreign_loop_handler *integrator,
+ void *private_data,
+ const char *location)
+{
+ return ev->ops->add_foreign_fd_handler(ev, integrator, private_data,
+ location);
+}
/*
re-initialise a tevent context. This leaves you with the same
diff --git a/lib/tevent/tevent.h b/lib/tevent/tevent.h
index cb95507..e521daa 100644
--- a/lib/tevent/tevent.h
+++ b/lib/tevent/tevent.h
@@ -408,7 +408,47 @@ int _tevent_loop_wait(struct tevent_context *ev, const char *location);
_tevent_loop_wait(ev, __location__)
#endif
+#define FOREIGN_FD_READ 1
+#define FOREIGN_FD_WRITE 2
+#define FOREIGN_FD_HUP 4
+#define FOREIGN_FD_ERR 8
+
+struct foreign_pollfd
+{
+ int fd;
+ uint16_t events;
+ uint16_t revents;
+};
+
+struct foreign_loop_handler
+{
+ /*
+ * callback which provides the fds (and timeout) to be added to the
+ * tevent wait loop. Note: fd set provided here is only valid for a
+ * single loop iteration, each loop iteration requests a new
+ * set of fds - callback is fired before
+ * underlying poll/select/epoll is called
+ *
+ */
+ struct timeval *(*prepare_poll)(struct foreign_pollfd **pollfds,
+ uint64_t*numfds, void *private_data);
+ /*
+ * callback which fires 'after' the foreign fds are
+ * polled/epolled/selected, it runs 'before' normal tevent fds are
+ * processed thus you are gauranteed the prepare/poll/check sequence
+ * (and any foreign callback called as a result) happen before any
+ * normal tevent 'fds' are processed.
+ */
+ void (*check_poll)(uint64_t npollfd, void *private_data);
+};
+bool _tevent_add_foreign_fd_handler(struct tevent_context *ev,
+ struct foreign_loop_handler *integrator,
+ void *private_data,
+ const char *location);
+
+#define tevent_add_foreign_fd_handler(ev, handler, private_data) \
+ _tevent_add_foreign_fd_handler(ev, handler, private_data, __location__)
/**
* Assign a function to run when a tevent_fd is freed
*
@@ -1830,6 +1870,11 @@ struct tevent_ops {
/* loop functions */
int (*loop_once)(struct tevent_context *ev, const char *location);
int (*loop_wait)(struct tevent_context *ev, const char *location);
+ /* returns true if supported */
+ bool (*add_foreign_fd_handler)(struct tevent_context *ev,
+ struct foreign_loop_handler *handler,
+ void *private_data,
+ const char *location);
};
bool tevent_register_backend(const char *name, const struct tevent_ops *ops);
diff --git a/lib/tevent/tevent_epoll.c b/lib/tevent/tevent_epoll.c
index 507ea5c..4bb73e9 100644
--- a/lib/tevent/tevent_epoll.c
+++ b/lib/tevent/tevent_epoll.c
@@ -926,6 +926,15 @@ static int epoll_event_loop_once(struct tevent_context *ev, const char *location
return epoll_event_loop(epoll_ev, &tval);
}
+static bool epoll_add_foreign_fd_handler(struct tevent_context *ev,
+ struct foreign_loop_handler *integrator,
+ void *private_data,
+ const char *location)
+{
+ /* not supported #FIXME */
+ return false;
+}
+
static const struct tevent_ops epoll_event_ops = {
.context_init = epoll_event_context_init,
.add_fd = epoll_event_add_fd,
@@ -937,6 +946,7 @@ static const struct tevent_ops epoll_event_ops = {
.add_signal = tevent_common_add_signal,
.loop_once = epoll_event_loop_once,
.loop_wait = tevent_common_loop_wait,
+ .add_foreign_fd_handler = epoll_add_foreign_fd_handler,
};
_PRIVATE_ bool tevent_epoll_init(void)
diff --git a/lib/tevent/tevent_internal.h b/lib/tevent/tevent_internal.h
index 10cc4a4..784634b 100644
--- a/lib/tevent/tevent_internal.h
+++ b/lib/tevent/tevent_internal.h
@@ -282,6 +282,13 @@ struct tevent_context {
* tevent_common_add_timer_v2()
*/
struct tevent_timer *last_zero_timer;
+
+ struct {
+ struct foreign_loop_handler *handler;
+ void *private_data;
+ struct timeval timeout;
+ bool events_valid;
+ } foreign_poll;
};
const struct tevent_ops *tevent_find_ops_byname(const char *name);
diff --git a/lib/tevent/tevent_poll.c b/lib/tevent/tevent_poll.c
index 9b1781f..2f93c1a 100644
--- a/lib/tevent/tevent_poll.c
+++ b/lib/tevent/tevent_poll.c
@@ -688,6 +688,15 @@ static int poll_event_loop_wait(struct tevent_context *ev,
return 0;
}
+static bool poll_add_foreign_fd_handler(struct tevent_context *ev,
+ struct foreign_loop_handler *integrator,
+ void *private_data,
+ const char *location)
+{
+ /* #FIXME */
+ return false;
+}
+
static const struct tevent_ops poll_event_ops = {
.context_init = poll_event_context_init,
.add_fd = poll_event_add_fd,
@@ -699,6 +708,7 @@ static const struct tevent_ops poll_event_ops = {
.add_signal = tevent_common_add_signal,
.loop_once = poll_event_loop_once,
.loop_wait = poll_event_loop_wait,
+ .add_foreign_fd_handler = poll_add_foreign_fd_handler,
};
_PRIVATE_ bool tevent_poll_init(void)
@@ -717,6 +727,7 @@ static const struct tevent_ops poll_event_mt_ops = {
.add_signal = tevent_common_add_signal,
.loop_once = poll_event_loop_once,
.loop_wait = poll_event_loop_wait,
+ .add_foreign_fd_handler = poll_add_foreign_fd_handler,
};
_PRIVATE_ bool tevent_poll_mt_init(void)
diff --git a/lib/tevent/tevent_port.c b/lib/tevent/tevent_port.c
index 5b487d7..b409d3f 100644
--- a/lib/tevent/tevent_port.c
+++ b/lib/tevent/tevent_port.c
@@ -763,6 +763,13 @@ static int port_event_loop_once(struct tevent_context *ev, const char *location)
return port_event_loop(port_ev, &tval);
}
+static port_add_foreign_fd_handler(struct tevent_context *ev,
+ struct foreign_loop_handler *integrator,
+ const char *location)
+{
+ /* #FIXME not supported */
+ return false;
+}
static const struct tevent_ops port_event_ops = {
.context_init = port_event_context_init,
.add_fd = port_event_add_fd,
@@ -774,6 +781,7 @@ static const struct tevent_ops port_event_ops = {
.add_signal = tevent_common_add_signal,
.loop_once = port_event_loop_once,
.loop_wait = tevent_common_loop_wait,
+ .add_foreign_fd_handler = port_add_foreign_fd_handler,
};
_PRIVATE_ bool tevent_port_init(void)
diff --git a/lib/tevent/tevent_select.c b/lib/tevent/tevent_select.c
index 40a4dc0..6aa3a65 100644
--- a/lib/tevent/tevent_select.c
+++ b/lib/tevent/tevent_select.c
@@ -257,6 +257,15 @@ static int select_event_loop_once(struct tevent_context *ev, const char *locatio
return select_event_loop_select(select_ev, &tval);
}
+static bool select_add_foreign_fd_handler(struct tevent_context *ev,
+ struct foreign_loop_handler *integrator,
+ void *private_data,
+ const char *location)
+{
+ /* not supported #FIXME */
+ return false;
+}
+
static const struct tevent_ops select_event_ops = {
.context_init = select_event_context_init,
.add_fd = select_event_add_fd,
@@ -268,6 +277,7 @@ static const struct tevent_ops select_event_ops = {
.add_signal = tevent_common_add_signal,
.loop_once = select_event_loop_once,
.loop_wait = tevent_common_loop_wait,
+ .add_foreign_fd_handler = select_add_foreign_fd_handler,
};
_PRIVATE_ bool tevent_select_init(void)
diff --git a/lib/tevent/tevent_standard.c b/lib/tevent/tevent_standard.c
index a050901..4ad271e 100644
--- a/lib/tevent/tevent_standard.c
+++ b/lib/tevent/tevent_standard.c
@@ -145,6 +145,7 @@ static int std_event_loop_wait(struct tevent_context *ev, const char *location)
return glue->poll_ops->loop_wait(ev, location);
}
+
/*
Initialize the epoll backend and allow it to call a
switch function if epoll fails at runtime.
--
2.1.4
From 44306ff517694cc36cac625a657b20697c724a1d Mon Sep 17 00:00:00 2001
From: Noel Power <noel.power at suse.com>
Date: Wed, 9 Dec 2015 11:47:23 +0000
Subject: [PATCH 2/9] free up fde->additional_flags for internal use in
tevent_poll.c
I want to use fde->additional_flags but it is already used to store an
index into the file descriptors in use for the poll function (and associated
tevent_fd list) Create a new structure that can contain the index and the
data previously stored in fde->additional_data and modify code to use that
---
lib/tevent/tevent_poll.c | 43 +++++++++++++++++++++++++++++--------------
1 file changed, 29 insertions(+), 14 deletions(-)
diff --git a/lib/tevent/tevent_poll.c b/lib/tevent/tevent_poll.c
index 2f93c1a..f8235ad 100644
--- a/lib/tevent/tevent_poll.c
+++ b/lib/tevent/tevent_poll.c
@@ -29,6 +29,11 @@
#include "tevent_util.h"
#include "tevent_internal.h"
+struct fde_extra_data {
+ uint64_t idx;
+ struct tevent_fd **list;
+};
+
struct poll_event_context {
/* a pointer back to the generic event_context */
struct tevent_context *ev;
@@ -220,7 +225,9 @@ static int poll_event_fd_destructor(struct tevent_fd *fde)
{
struct tevent_context *ev = fde->event_ctx;
struct poll_event_context *poll_ev;
- uint64_t del_idx = fde->additional_flags;
+ struct fde_extra_data *extra_data =
+ (struct fde_extra_data *)fde->additional_data;
+ uint64_t del_idx = extra_data->idx;
if (ev == NULL) {
goto done;
@@ -230,8 +237,7 @@ static int poll_event_fd_destructor(struct tevent_fd *fde)
ev->additional_data, struct poll_event_context);
if (del_idx == UINT64_MAX) {
- struct tevent_fd **listp =
- (struct tevent_fd **)fde->additional_data;
+ struct tevent_fd **listp = extra_data->list;
DLIST_REMOVE((*listp), fde);
goto done;
@@ -269,6 +275,8 @@ _PRIVATE_ void tevent_poll_event_add_fd_internal(struct tevent_context *ev,
struct poll_event_context *poll_ev = talloc_get_type_abort(
ev->additional_data, struct poll_event_context);
struct tevent_fd **listp;
+ struct fde_extra_data *extra_data =
+ (struct fde_extra_data *)fde->additional_data;
if (fde->flags != 0) {
listp = &poll_ev->fresh;
@@ -276,8 +284,8 @@ _PRIVATE_ void tevent_poll_event_add_fd_internal(struct tevent_context *ev,
listp = &poll_ev->disabled;
}
- fde->additional_flags = UINT64_MAX;
- fde->additional_data = listp;
+ extra_data->idx = UINT64_MAX;
+ extra_data->list = listp;
DLIST_ADD((*listp), fde);
talloc_set_destructor(fde, poll_event_fd_destructor);
@@ -315,8 +323,9 @@ static struct tevent_fd *poll_event_add_fd(struct tevent_context *ev,
fde->private_data = private_data;
fde->handler_name = handler_name;
fde->location = location;
- fde->additional_flags = UINT64_MAX;
- fde->additional_data = NULL;
+ fde->additional_flags = 0;
+ fde->additional_data = talloc_zero(fde, struct fde_extra_data);
+ ((struct fde_extra_data*)fde->additional_data)->idx = UINT64_MAX;
tevent_poll_event_add_fd_internal(ev, fde);
poll_event_wake_pollthread(poll_ev);
@@ -335,7 +344,9 @@ static void poll_event_set_fd_flags(struct tevent_fd *fde, uint16_t flags)
{
struct tevent_context *ev = fde->event_ctx;
struct poll_event_context *poll_ev;
- uint64_t idx = fde->additional_flags;
+ struct fde_extra_data *extra_data =
+ (struct fde_extra_data *)fde->additional_data;
+ uint64_t idx = extra_data->idx;
uint16_t pollflags;
if (ev == NULL) {
@@ -347,8 +358,7 @@ static void poll_event_set_fd_flags(struct tevent_fd *fde, uint16_t flags)
fde->flags = flags;
if (idx == UINT64_MAX) {
- struct tevent_fd **listp =
- (struct tevent_fd **)fde->additional_data;
+ struct tevent_fd **listp = extra_data->list;
/*
* We move it between the fresh and disabled lists.
@@ -413,7 +423,9 @@ static bool poll_event_setup_fresh(struct tevent_context *ev,
poll_ev->fds[i] = poll_ev->fds[poll_ev->num_fds];
poll_ev->fdes[i] = poll_ev->fdes[poll_ev->num_fds];
if (poll_ev->fdes[i] != NULL) {
- poll_ev->fdes[i]->additional_flags = i;
+ struct fde_extra_data *extra_data =
+ (struct fde_extra_data *)poll_ev->fdes[i]->additional_data;
+ extra_data->idx = i;
}
}
poll_ev->deleted = false;
@@ -460,8 +472,9 @@ static bool poll_event_setup_fresh(struct tevent_context *ev,
for (fde = poll_ev->fresh; fde; fde = next) {
struct pollfd *pfd;
-
+ struct fde_extra_data *extra_data;
pfd = &poll_ev->fds[poll_ev->num_fds];
+ extra_data = (struct fde_extra_data *)fde->additional_data;
pfd->fd = fde->fd;
pfd->events = 0;
@@ -474,7 +487,7 @@ static bool poll_event_setup_fresh(struct tevent_context *ev,
pfd->events |= (POLLOUT);
}
- fde->additional_flags = poll_ev->num_fds;
+ extra_data->idx = poll_ev->num_fds;
poll_ev->fdes[poll_ev->num_fds] = fde;
next = fde->next;
@@ -544,7 +557,9 @@ static int poll_event_loop_poll(struct tevent_context *ev,
the handler to remove itself when called */
for (fde = ev->fd_events; fde; fde = next) {
- uint64_t idx = fde->additional_flags;
+ struct fde_extra_data *extra_data =
+ (struct fde_extra_data *)fde->additional_data;
+ uint64_t idx = extra_data->idx;
struct pollfd *pfd;
uint16_t flags = 0;
--
2.1.4
From 9cd36684a23ff14148af642f4522b13d2bc4da3d Mon Sep 17 00:00:00 2001
From: Noel Power <noel.power at suse.com>
Date: Tue, 15 Dec 2015 13:51:42 +0000
Subject: [PATCH 3/9] implement poll implementation that allows foreign fds to
be integrated
Modify the poll tevent backend to accept foreign fds to be monitored via
a callback. Foreign fds are fds that are not handled by tevent as such but are
inserted into the poll/select/epoll that tevent uses and the results for
any events to be monitored posted back to the client via callback. The
main purpose of this functionality is to allow integratiion with foreign
event loops.
---
lib/tevent/tevent_internal.h | 2 +
lib/tevent/tevent_poll.c | 267 +++++++++++++++++++++++++++++++++++++++----
2 files changed, 247 insertions(+), 22 deletions(-)
diff --git a/lib/tevent/tevent_internal.h b/lib/tevent/tevent_internal.h
index 784634b..f584b31 100644
--- a/lib/tevent/tevent_internal.h
+++ b/lib/tevent/tevent_internal.h
@@ -27,6 +27,8 @@
License along with this library; if not, see <http://www.gnu.org/licenses/>.
*/
+#define FOREIGN_FD 0x1000
+
struct tevent_req {
/**
* @brief What to do on completion
diff --git a/lib/tevent/tevent_poll.c b/lib/tevent/tevent_poll.c
index f8235ad..72d9fc0 100644
--- a/lib/tevent/tevent_poll.c
+++ b/lib/tevent/tevent_poll.c
@@ -471,20 +471,42 @@ static bool poll_event_setup_fresh(struct tevent_context *ev,
}
for (fde = poll_ev->fresh; fde; fde = next) {
- struct pollfd *pfd;
- struct fde_extra_data *extra_data;
- pfd = &poll_ev->fds[poll_ev->num_fds];
- extra_data = (struct fde_extra_data *)fde->additional_data;
-
- pfd->fd = fde->fd;
- pfd->events = 0;
- pfd->revents = 0;
- if (fde->flags & TEVENT_FD_READ) {
- pfd->events |= (POLLIN|POLLHUP);
- }
- if (fde->flags & TEVENT_FD_WRITE) {
- pfd->events |= (POLLOUT);
+ struct pollfd *pfd;
+ struct fde_extra_data *extra_data;
+ pfd = &poll_ev->fds[poll_ev->num_fds];
+ extra_data = (struct fde_extra_data *)fde->additional_data;
+
+ pfd->fd = fde->fd;
+ pfd->events = 0;
+ pfd->revents = 0;
+
+ if (fde->flags & FOREIGN_FD) {
+ struct foreign_pollfd *ffd =
+ (struct foreign_pollfd *)fde->private_data;
+ if (ffd->events & FOREIGN_FD_READ) {
+ pfd->events |= POLLIN;
+ //tevent_debug(poll_ev->ev, TEVENT_DEBUG_WARNING,"adding read event fo foreign fd %d\n", fde->fd);
+ }
+ if (ffd->events & FOREIGN_FD_WRITE) {
+ pfd->events |= POLLOUT;
+ //tevent_debug(poll_ev->ev, TEVENT_DEBUG_WARNING,"adding write event fo foreign fd %d\n", fde->fd);
+ }
+ if (ffd->events & FOREIGN_FD_ERR) {
+ pfd->events |= POLLERR;
+ }
+ if (ffd->events & FOREIGN_FD_HUP) {
+ pfd->events |= POLLHUP;
+ }
+ ffd->revents = 0;
+
+ } else {
+ if (fde->flags & TEVENT_FD_READ) {
+ pfd->events |= (POLLIN|POLLHUP);
+ }
+ if (fde->flags & TEVENT_FD_WRITE) {
+ pfd->events |= (POLLOUT);
+ }
}
extra_data->idx = poll_ev->num_fds;
@@ -496,9 +518,93 @@ static bool poll_event_setup_fresh(struct tevent_context *ev,
poll_ev->num_fds += 1;
}
+// tevent_debug(poll_ev->ev, TEVENT_DEBUG_WARNING,"number of fds %d\n",poll_ev->num_fds );
return true;
}
+static void remove_foreign_fds(struct tevent_context *ev)
+{
+ struct tevent_fd *fde = NULL;
+ struct tevent_fd *next = NULL;
+ struct foreign_loop_handler *integrator = ev->foreign_poll.handler;
+ if (!integrator) {
+ return;
+ }
+ for (fde = ev->fd_events; fde; fde = next) {
+ bool is_foreign_fd =
+ (fde->flags & FOREIGN_FD);
+ next = fde->next;
+ if (is_foreign_fd) {
+ TALLOC_FREE(fde);
+ }
+ }
+ /*
+ * mark that there are no foreign events to poll, however we still have
+ * the actual fd(s) in fd_events, they are there so we can keep
+ * looping with loop_wait, each iteratation of the loop we remove the
+ * old and will request a completely 'new' set of fds. So if 'zero'
+ * foreign fds are returned from the callback then the foreign fds
+ * will be removed if necessary in that iteration and no new ones
+ * added.
+ *
+ */
+ ev->foreign_poll.events_valid = false;
+}
+
+static void process_foreign_fds(struct tevent_context *ev, bool report_only)
+{
+ struct tevent_fd *fde = NULL;
+ struct tevent_fd *next = NULL;
+ struct poll_event_context *poll_ev = talloc_get_type_abort(
+ ev->additional_data, struct poll_event_context);
+ struct foreign_loop_handler *integrator = ev->foreign_poll.handler;
+ void *priv_data = ev->foreign_poll.private_data;
+ int i = 0;
+// tevent_debug(ev, TEVENT_DEBUG_WARNING,"enter check_and_rm_foreign_fds\n");
+ if (!integrator) {
+ return;
+ }
+ for (fde = ev->fd_events; !report_only && fde; fde = next) {
+ struct fde_extra_data *extra_data =
+ (struct fde_extra_data *)fde->additional_data;
+ struct pollfd *pfd;
+ bool is_foreign_fd =
+ (fde->flags & FOREIGN_FD);
+
+ next = fde->next;
+ if (is_foreign_fd) {
+ uint64_t idx = extra_data->idx;
+
+ pfd = &poll_ev->fds[idx];
+ if (pfd->revents & (POLLHUP|POLLERR)) {
+ fde->additional_flags |=
+ ((pfd->revents & (POLLHUP|POLLERR)));
+ }
+ if (pfd->revents & POLLIN) {
+ fde->additional_flags |= POLLIN;
+ }
+ if (pfd->revents & POLLOUT)
+ fde->additional_flags |= POLLOUT;
+ if (pfd->revents) {
+ i++;
+ }
+ fde->handler(ev, fde, fde->flags, fde->private_data);
+ }
+ }
+// tevent_debug(ev, TEVENT_DEBUG_WARNING,"leaving check_and_rm_foreign_fds and about to call check_poll\n");
+ integrator->check_poll(i, priv_data);
+
+ /*
+ * ensure while in outer loop_wait will terminate if we dont have
+ * any fds to read.
+ */
+ ev->foreign_poll.events_valid = false;
+}
+
+
+static void add_foreignfds_to_poll(struct tevent_context *ev,
+ struct foreign_pollfd *pollfds,
+ uint64_t numfds);
/*
event loop handling using poll()
*/
@@ -507,13 +613,14 @@ static int poll_event_loop_poll(struct tevent_context *ev,
{
struct poll_event_context *poll_ev = talloc_get_type_abort(
ev->additional_data, struct poll_event_context);
+ struct foreign_loop_handler *poll_handler = ev->foreign_poll.handler;
int pollrtn;
int timeout = -1;
int poll_errno;
struct tevent_fd *fde = NULL;
struct tevent_fd *next = NULL;
unsigned i;
-
+ int ret;
if (ev->signal_events && tevent_common_check_signal(ev)) {
return 0;
}
@@ -523,10 +630,38 @@ static int poll_event_loop_poll(struct tevent_context *ev,
timeout += (tvalp->tv_usec + 999) / 1000;
}
+// tevent_debug(ev, TEVENT_DEBUG_WARNING,"timeout is %d\n", timeout);
poll_event_drain_signal_fd(poll_ev);
+ if (poll_handler) {
+ struct timeval *foreign_tvalp = NULL;
+ struct foreign_pollfd *pollfds;
+ uint64_t numfds;
+ void *priv_data = ev->foreign_poll.private_data;
+ /* remove previous fds */
+ remove_foreign_fds(ev);
+ foreign_tvalp = poll_handler->prepare_poll(&pollfds, &numfds,
+ priv_data);
+ /*
+ * We may need to to override the timeout, e.g. glib is very
+ * dependant on having the correct (and often even '0')
+ * timeout, failure to honour that results in unacceptable
+ * performance from glib triggered functionality.
+ */
+ if (foreign_tvalp) {
+ int new_timeout = foreign_tvalp->tv_sec * 1000;
+ new_timeout += (foreign_tvalp->tv_usec + 999) / 1000;
+ /* use the smallest timeout */
+ if (new_timeout < timeout) {
+ timeout = new_timeout;
+ }
+ }
+ add_foreignfds_to_poll(ev, pollfds, numfds);
+ }
+
if (!poll_event_setup_fresh(ev, poll_ev)) {
- return -1;
+ ret = -1;
+ goto early_poll_out;
}
tevent_trace_point_callback(poll_ev->ev, TEVENT_TRACE_BEFORE_WAIT);
@@ -536,20 +671,43 @@ static int poll_event_loop_poll(struct tevent_context *ev,
if (pollrtn == -1 && poll_errno == EINTR && ev->signal_events) {
tevent_common_check_signal(ev);
- return 0;
+ ret = 0;
}
if (pollrtn == 0 && tvalp) {
/* we don't care about a possible delay here */
tevent_common_loop_timer_delay(ev);
- return 0;
+ ret = 0;
+ goto early_poll_out;
}
if (pollrtn <= 0) {
/*
* No fd's ready
*/
- return 0;
+ ret = 0;
+ goto early_poll_out;
+ }
+
+ /* process any polled foreign fds together, foreign loops expect similar
+ * e.g. all fds with events can be processed at once (the tevent poll loop
+ * processes events one at a time)
+ * Also, we do this here so we have a clean and short route to
+ * go through the following phases
+ * + prepare
+ * + poll
+ * + check
+ * This also ensures we process the foreign fds together, no
+ * interleaving with tevent events. For glib this allows
+ * the context to be acquired and held for all of the phases above.
+ * Additionally it should allow (if necessary) a mutex to be held
+ * from the point where the fds are passed to us (from prepare_poll)
+ * through to when the foreign event loop 'does it's thing' in
+ * check_poll. I see there is also some mt epoll backend, maybe
+ * that would require some mutex magic.. not sure
+ */
+ if (poll_handler) {
+ process_foreign_fds(ev, false);
}
/* at least one file descriptor is ready - check
@@ -565,6 +723,11 @@ static int poll_event_loop_poll(struct tevent_context *ev,
next = fde->next;
+ bool is_foreign_fd =
+ (fde->flags & FOREIGN_FD);
+ if (is_foreign_fd) {
+ continue;
+ }
if (idx == UINT64_MAX) {
continue;
}
@@ -617,6 +780,7 @@ static int poll_event_loop_poll(struct tevent_context *ev,
flags &= fde->flags;
if (flags != 0) {
DLIST_DEMOTE(ev->fd_events, fde, struct tevent_fd);
+// tevent_debug(ev, TEVENT_DEBUG_WARNING,"firing fd %d (in normal loop) is_foreign_fd 0x%x\n", fde->fd, (int)fde->additional_flags );
fde->handler(ev, fde, flags, fde->private_data);
return 0;
}
@@ -645,6 +809,54 @@ static int poll_event_loop_poll(struct tevent_context *ev,
}
return 0;
+early_poll_out:
+ /* just call check_poll, don't bother examining the fds */
+ process_foreign_fds(ev, true);
+ return ret;
+}
+
+static void foreign_fd_handler(struct tevent_context *ev,
+ struct tevent_fd *fde,
+ uint16_t flags,
+ void *private_data)
+{
+ struct foreign_pollfd *ffd =
+ (struct foreign_pollfd *)fde->private_data;
+
+// tevent_debug(ev, TEVENT_DEBUG_WARNING,"handler fired for foreign fd %d\n", fde->fd);
+ /* convert poll event to FOREIGN_FD_XYZ event */
+ if (fde->additional_flags & POLLIN) {
+// tevent_debug(ev, TEVENT_DEBUG_WARNING,"handler fired for foreign fd READ%d\n", fde->fd);
+ ffd->revents |= FOREIGN_FD_READ;
+ }
+ if (fde->additional_flags & POLLOUT) {
+ ffd->revents |= FOREIGN_FD_WRITE;
+// tevent_debug(ev, TEVENT_DEBUG_WARNING,"handler fired for foreign fd WRITE%d\n", fde->fd);
+ }
+ if (fde->additional_flags & POLLHUP) {
+ ffd->revents |= FOREIGN_FD_HUP;
+ }
+ if (fde->additional_flags & POLLERR) {
+ ffd->revents |= FOREIGN_FD_ERR;
+ }
+}
+
+static void add_foreignfds_to_poll(struct tevent_context *ev,
+ struct foreign_pollfd *pollfds,
+ uint64_t numfds)
+{
+ struct tevent_fd *fde;
+ uint64_t i;
+ for (i = 0; i < numfds; i++) {
+ /* set flags so the fd is on theactive list */
+ fde = poll_event_add_fd(ev, NULL, pollfds[i].fd,
+ TEVENT_FD_READ | TEVENT_FD_WRITE,
+ foreign_fd_handler, &pollfds[i],
+ "foreign_fd_handler", __location__);
+// tevent_debug(ev, TEVENT_DEBUG_WARNING,"adding foreign fd %d\n", fde->fd);
+ fde->flags |= FOREIGN_FD;
+ ev->foreign_poll.events_valid = true;
+ }
}
/*
@@ -678,16 +890,26 @@ static int poll_event_loop_wait(struct tevent_context *ev,
{
struct poll_event_context *poll_ev = talloc_get_type_abort(
ev->additional_data, struct poll_event_context);
-
+ struct foreign_loop_handler *poll_handler = ev->foreign_poll.handler;
/*
* loop as long as we have events pending
*/
+ /* see if there are any foreign events to add */
+ if (poll_handler) {
+ /* if we have a poll_handler it's pretty safe
+ * to assume we want to pass into the wait loop
+ * and call out the 'check_poll' handler at least
+ * once
+ */
+ ev->foreign_poll.events_valid = true;
+ }
while (ev->fd_events ||
ev->timer_events ||
ev->immediate_events ||
ev->signal_events ||
poll_ev->fresh ||
- poll_ev->disabled) {
+ poll_ev->disabled ||
+ ev->foreign_poll.events_valid) {
int ret;
ret = _tevent_loop_once(ev, location);
if (ret != 0) {
@@ -708,8 +930,9 @@ static bool poll_add_foreign_fd_handler(struct tevent_context *ev,
void *private_data,
const char *location)
{
- /* #FIXME */
- return false;
+ ev->foreign_poll.handler = integrator;
+ ev->foreign_poll.private_data = private_data;
+ return true;
}
static const struct tevent_ops poll_event_ops = {
--
2.1.4
From ce8d2dedbbda214cad05dc8922faf46a8fd0563d Mon Sep 17 00:00:00 2001
From: Noel Power <noel.power at suse.com>
Date: Tue, 15 Dec 2015 14:30:33 +0000
Subject: [PATCH 4/9] separate out some common functionality
---
lib/tevent/tevent.c | 69 ++++++++++++++++++++++++++++++++++++++++++++
lib/tevent/tevent_internal.h | 13 +++++++++
2 files changed, 82 insertions(+)
diff --git a/lib/tevent/tevent.c b/lib/tevent/tevent.c
index 71cf0c1..0686a31 100644
--- a/lib/tevent/tevent.c
+++ b/lib/tevent/tevent.c
@@ -678,3 +678,72 @@ int tevent_re_initialise(struct tevent_context *ev)
return ev->ops->context_init(ev);
}
+
+/* remove foreign fds from fd_events */
+void common_remove_foreign_fds(struct tevent_context *ev)
+{
+ struct tevent_fd *fde = NULL;
+ struct tevent_fd *next = NULL;
+ struct foreign_loop_handler *integrator = ev->foreign_poll.handler;
+ if (!integrator) {
+ return;
+ }
+ for (fde = ev->fd_events; fde; fde = next) {
+ bool is_foreign_fd =
+ (fde->flags & FOREIGN_FD);
+ next = fde->next;
+ if (is_foreign_fd) {
+ TALLOC_FREE(fde);
+ }
+ }
+ /*
+ * mark that there are no foreign events to poll, however we still have
+ * the actual fd(s) in fd_events, they are there so we can keep
+ * looping with loop_wait, each iteratation of the loop we remove the
+ * old and will request a completely 'new' set of fds. So if 'zero'
+ * foreign fds are returned from the callback then the foreign fds
+ * will be removed if necessary in that iteration and no new ones
+ * added.
+ *
+ */
+ ev->foreign_poll.events_valid = false;
+}
+
+/*
+ * add callbacks to allow foreign fds to be integrated and checked
+ * returns true if foreign fd handling is supported
+ */
+bool common_add_foreign_fd_handler(struct tevent_context *ev,
+ struct foreign_loop_handler *integrator,
+ void *private_data,
+ const char *location)
+{
+ ev->foreign_poll.handler = integrator;
+ ev->foreign_poll.private_data = private_data;
+ return true;
+}
+
+/*
+ * add foreign pollfds, the adds tevent_fds for each foreign_pollfd
+ * a special handler poll_handler is also added so that the foreign_pollfd
+ * revents can be converted from the specific poll/select/epoll etc. backend
+ */
+
+void common_add_foreignfds(struct tevent_context *ev,
+ tevent_fd_handler_t convert_revents,
+ struct foreign_pollfd *pollfds,
+ uint64_t numfds)
+{
+ struct tevent_fd *fde;
+ uint64_t i;
+ for (i = 0; i < numfds; i++) {
+ /* set flags so the fd is on theactive list */
+ fde = ev->ops->add_fd(ev, NULL, pollfds[i].fd,
+ TEVENT_FD_READ | TEVENT_FD_WRITE,
+ convert_revents, &pollfds[i],
+ "add_foreignfds_to_poll", __location__);
+// tevent_debug(ev, TEVENT_DEBUG_WARNING,"adding foreign fd %d\n", fde->fd);
+ fde->flags |= FOREIGN_FD;
+ ev->foreign_poll.events_valid = true;
+ }
+}
diff --git a/lib/tevent/tevent_internal.h b/lib/tevent/tevent_internal.h
index f584b31..800da54 100644
--- a/lib/tevent/tevent_internal.h
+++ b/lib/tevent/tevent_internal.h
@@ -367,3 +367,16 @@ bool tevent_port_init(void);
void tevent_trace_point_callback(struct tevent_context *ev,
enum tevent_trace_point);
+
+
+bool common_add_foreign_fd_handler(struct tevent_context *ev,
+ struct foreign_loop_handler *integrator,
+ void *private_data,
+ const char *location);
+
+void common_add_foreignfds(struct tevent_context *ev,
+ tevent_fd_handler_t convert_revents,
+ struct foreign_pollfd *pollfds,
+ uint64_t numfds);
+
+void common_remove_foreign_fds(struct tevent_context *ev);
--
2.1.4
From ff0db888c6a01ae598e060bf35da067ae0a7bcc6 Mon Sep 17 00:00:00 2001
From: Noel Power <noel.power at suse.com>
Date: Tue, 15 Dec 2015 15:29:16 +0000
Subject: [PATCH 5/9] bump tevent version and add some extra sigs for helper
functions
---
lib/tevent/ABI/tevent-0.9.27.sigs | 94 +++++++++++++++++++++++++++++++++++++++
lib/tevent/wscript | 2 +-
2 files changed, 95 insertions(+), 1 deletion(-)
create mode 100644 lib/tevent/ABI/tevent-0.9.27.sigs
diff --git a/lib/tevent/ABI/tevent-0.9.27.sigs b/lib/tevent/ABI/tevent-0.9.27.sigs
new file mode 100644
index 0000000..70441a7
--- /dev/null
+++ b/lib/tevent/ABI/tevent-0.9.27.sigs
@@ -0,0 +1,94 @@
+_tevent_add_fd: struct tevent_fd *(struct tevent_context *, TALLOC_CTX *, int, uint16_t, tevent_fd_handler_t, void *, const char *, const char *)
+_tevent_add_foreign_fd_handler: bool (struct tevent_context *, struct foreign_loop_handler *, void *, const char *)
+_tevent_add_signal: struct tevent_signal *(struct tevent_context *, TALLOC_CTX *, int, int, tevent_signal_handler_t, void *, const char *, const char *)
+_tevent_add_timer: struct tevent_timer *(struct tevent_context *, TALLOC_CTX *, struct timeval, tevent_timer_handler_t, void *, const char *, const char *)
+_tevent_create_immediate: struct tevent_immediate *(TALLOC_CTX *, const char *)
+_tevent_loop_once: int (struct tevent_context *, const char *)
+_tevent_loop_until: int (struct tevent_context *, bool (*)(void *), void *, const char *)
+_tevent_loop_wait: int (struct tevent_context *, const char *)
+_tevent_queue_create: struct tevent_queue *(TALLOC_CTX *, const char *, const char *)
+_tevent_req_callback_data: void *(struct tevent_req *)
+_tevent_req_cancel: bool (struct tevent_req *, const char *)
+_tevent_req_create: struct tevent_req *(TALLOC_CTX *, void *, size_t, const char *, const char *)
+_tevent_req_data: void *(struct tevent_req *)
+_tevent_req_done: void (struct tevent_req *, const char *)
+_tevent_req_error: bool (struct tevent_req *, uint64_t, const char *)
+_tevent_req_nomem: bool (const void *, struct tevent_req *, const char *)
+_tevent_req_notify_callback: void (struct tevent_req *, const char *)
+_tevent_req_oom: void (struct tevent_req *, const char *)
+_tevent_schedule_immediate: void (struct tevent_immediate *, struct tevent_context *, tevent_immediate_handler_t, void *, const char *, const char *)
+common_add_foreign_fd_handler: bool (struct tevent_context *, struct foreign_loop_handler *, void *, const char *)
+common_add_foreignfds: void (struct tevent_context *, tevent_fd_handler_t, struct foreign_pollfd *, uint64_t)
+common_remove_foreign_fds: void (struct tevent_context *)
+tevent_backend_list: const char **(TALLOC_CTX *)
+tevent_cleanup_pending_signal_handlers: void (struct tevent_signal *)
+tevent_common_add_fd: struct tevent_fd *(struct tevent_context *, TALLOC_CTX *, int, uint16_t, tevent_fd_handler_t, void *, const char *, const char *)
+tevent_common_add_signal: struct tevent_signal *(struct tevent_context *, TALLOC_CTX *, int, int, tevent_signal_handler_t, void *, const char *, const char *)
+tevent_common_add_timer: struct tevent_timer *(struct tevent_context *, TALLOC_CTX *, struct timeval, tevent_timer_handler_t, void *, const char *, const char *)
+tevent_common_add_timer_v2: struct tevent_timer *(struct tevent_context *, TALLOC_CTX *, struct timeval, tevent_timer_handler_t, void *, const char *, const char *)
+tevent_common_check_signal: int (struct tevent_context *)
+tevent_common_context_destructor: int (struct tevent_context *)
+tevent_common_fd_destructor: int (struct tevent_fd *)
+tevent_common_fd_get_flags: uint16_t (struct tevent_fd *)
+tevent_common_fd_set_close_fn: void (struct tevent_fd *, tevent_fd_close_fn_t)
+tevent_common_fd_set_flags: void (struct tevent_fd *, uint16_t)
+tevent_common_loop_immediate: bool (struct tevent_context *)
+tevent_common_loop_timer_delay: struct timeval (struct tevent_context *)
+tevent_common_loop_wait: int (struct tevent_context *, const char *)
+tevent_common_schedule_immediate: void (struct tevent_immediate *, struct tevent_context *, tevent_immediate_handler_t, void *, const char *, const char *)
+tevent_context_init: struct tevent_context *(TALLOC_CTX *)
+tevent_context_init_byname: struct tevent_context *(TALLOC_CTX *, const char *)
+tevent_context_init_ops: struct tevent_context *(TALLOC_CTX *, const struct tevent_ops *, void *)
+tevent_debug: void (struct tevent_context *, enum tevent_debug_level, const char *, ...)
+tevent_fd_get_flags: uint16_t (struct tevent_fd *)
+tevent_fd_set_auto_close: void (struct tevent_fd *)
+tevent_fd_set_close_fn: void (struct tevent_fd *, tevent_fd_close_fn_t)
+tevent_fd_set_flags: void (struct tevent_fd *, uint16_t)
+tevent_get_trace_callback: void (struct tevent_context *, tevent_trace_callback_t *, void *)
+tevent_loop_allow_nesting: void (struct tevent_context *)
+tevent_loop_set_nesting_hook: void (struct tevent_context *, tevent_nesting_hook, void *)
+tevent_num_signals: size_t (void)
+tevent_queue_add: bool (struct tevent_queue *, struct tevent_context *, struct tevent_req *, tevent_queue_trigger_fn_t, void *)
+tevent_queue_add_entry: struct tevent_queue_entry *(struct tevent_queue *, struct tevent_context *, struct tevent_req *, tevent_queue_trigger_fn_t, void *)
+tevent_queue_add_optimize_empty: struct tevent_queue_entry *(struct tevent_queue *, struct tevent_context *, struct tevent_req *, tevent_queue_trigger_fn_t, void *)
+tevent_queue_length: size_t (struct tevent_queue *)
+tevent_queue_running: bool (struct tevent_queue *)
+tevent_queue_start: void (struct tevent_queue *)
+tevent_queue_stop: void (struct tevent_queue *)
+tevent_queue_wait_recv: bool (struct tevent_req *)
+tevent_queue_wait_send: struct tevent_req *(TALLOC_CTX *, struct tevent_context *, struct tevent_queue *)
+tevent_re_initialise: int (struct tevent_context *)
+tevent_register_backend: bool (const char *, const struct tevent_ops *)
+tevent_req_default_print: char *(struct tevent_req *, TALLOC_CTX *)
+tevent_req_defer_callback: void (struct tevent_req *, struct tevent_context *)
+tevent_req_is_error: bool (struct tevent_req *, enum tevent_req_state *, uint64_t *)
+tevent_req_is_in_progress: bool (struct tevent_req *)
+tevent_req_poll: bool (struct tevent_req *, struct tevent_context *)
+tevent_req_post: struct tevent_req *(struct tevent_req *, struct tevent_context *)
+tevent_req_print: char *(TALLOC_CTX *, struct tevent_req *)
+tevent_req_received: void (struct tevent_req *)
+tevent_req_set_callback: void (struct tevent_req *, tevent_req_fn, void *)
+tevent_req_set_cancel_fn: void (struct tevent_req *, tevent_req_cancel_fn)
+tevent_req_set_cleanup_fn: void (struct tevent_req *, tevent_req_cleanup_fn)
+tevent_req_set_endtime: bool (struct tevent_req *, struct tevent_context *, struct timeval)
+tevent_req_set_print_fn: void (struct tevent_req *, tevent_req_print_fn)
+tevent_sa_info_queue_count: size_t (void)
+tevent_set_abort_fn: void (void (*)(const char *))
+tevent_set_debug: int (struct tevent_context *, void (*)(void *, enum tevent_debug_level, const char *, va_list), void *)
+tevent_set_debug_stderr: int (struct tevent_context *)
+tevent_set_default_backend: void (const char *)
+tevent_set_trace_callback: void (struct tevent_context *, tevent_trace_callback_t, void *)
+tevent_signal_support: bool (struct tevent_context *)
+tevent_thread_proxy_create: struct tevent_thread_proxy *(struct tevent_context *)
+tevent_thread_proxy_schedule: void (struct tevent_thread_proxy *, struct tevent_immediate **, tevent_immediate_handler_t, void *)
+tevent_timeval_add: struct timeval (const struct timeval *, uint32_t, uint32_t)
+tevent_timeval_compare: int (const struct timeval *, const struct timeval *)
+tevent_timeval_current: struct timeval (void)
+tevent_timeval_current_ofs: struct timeval (uint32_t, uint32_t)
+tevent_timeval_is_zero: bool (const struct timeval *)
+tevent_timeval_set: struct timeval (uint32_t, uint32_t)
+tevent_timeval_until: struct timeval (const struct timeval *, const struct timeval *)
+tevent_timeval_zero: struct timeval (void)
+tevent_trace_point_callback: void (struct tevent_context *, enum tevent_trace_point)
+tevent_wakeup_recv: bool (struct tevent_req *)
+tevent_wakeup_send: struct tevent_req *(TALLOC_CTX *, struct tevent_context *, struct timeval)
diff --git a/lib/tevent/wscript b/lib/tevent/wscript
index 73871d8..113c590 100755
--- a/lib/tevent/wscript
+++ b/lib/tevent/wscript
@@ -1,7 +1,7 @@
#!/usr/bin/env python
APPNAME = 'tevent'
-VERSION = '0.9.26'
+VERSION = '0.9.27'
blddir = 'bin'
--
2.1.4
From 66b27be26d7d5841ec21ecdf397b3ad374cee47e Mon Sep 17 00:00:00 2001
From: Noel Power <noel.power at suse.com>
Date: Tue, 15 Dec 2015 15:30:18 +0000
Subject: [PATCH 6/9] modify poll fd implementation to use some 'common' helper
functions
---
lib/tevent/tevent_poll.c | 70 ++++++++----------------------------------------
1 file changed, 11 insertions(+), 59 deletions(-)
diff --git a/lib/tevent/tevent_poll.c b/lib/tevent/tevent_poll.c
index 72d9fc0..c0fb9cf 100644
--- a/lib/tevent/tevent_poll.c
+++ b/lib/tevent/tevent_poll.c
@@ -522,35 +522,6 @@ static bool poll_event_setup_fresh(struct tevent_context *ev,
return true;
}
-static void remove_foreign_fds(struct tevent_context *ev)
-{
- struct tevent_fd *fde = NULL;
- struct tevent_fd *next = NULL;
- struct foreign_loop_handler *integrator = ev->foreign_poll.handler;
- if (!integrator) {
- return;
- }
- for (fde = ev->fd_events; fde; fde = next) {
- bool is_foreign_fd =
- (fde->flags & FOREIGN_FD);
- next = fde->next;
- if (is_foreign_fd) {
- TALLOC_FREE(fde);
- }
- }
- /*
- * mark that there are no foreign events to poll, however we still have
- * the actual fd(s) in fd_events, they are there so we can keep
- * looping with loop_wait, each iteratation of the loop we remove the
- * old and will request a completely 'new' set of fds. So if 'zero'
- * foreign fds are returned from the callback then the foreign fds
- * will be removed if necessary in that iteration and no new ones
- * added.
- *
- */
- ev->foreign_poll.events_valid = false;
-}
-
static void process_foreign_fds(struct tevent_context *ev, bool report_only)
{
struct tevent_fd *fde = NULL;
@@ -601,10 +572,10 @@ static void process_foreign_fds(struct tevent_context *ev, bool report_only)
ev->foreign_poll.events_valid = false;
}
-
-static void add_foreignfds_to_poll(struct tevent_context *ev,
- struct foreign_pollfd *pollfds,
- uint64_t numfds);
+static void foreign_fd_handler(struct tevent_context *ev,
+ struct tevent_fd *fde,
+ uint16_t flags,
+ void *private_data);
/*
event loop handling using poll()
*/
@@ -639,7 +610,7 @@ static int poll_event_loop_poll(struct tevent_context *ev,
uint64_t numfds;
void *priv_data = ev->foreign_poll.private_data;
/* remove previous fds */
- remove_foreign_fds(ev);
+ common_remove_foreign_fds(ev);
foreign_tvalp = poll_handler->prepare_poll(&pollfds, &numfds,
priv_data);
/*
@@ -656,7 +627,7 @@ static int poll_event_loop_poll(struct tevent_context *ev,
timeout = new_timeout;
}
}
- add_foreignfds_to_poll(ev, pollfds, numfds);
+ common_add_foreignfds(ev, foreign_fd_handler, pollfds, numfds);
}
if (!poll_event_setup_fresh(ev, poll_ev)) {
@@ -841,24 +812,6 @@ static void foreign_fd_handler(struct tevent_context *ev,
}
}
-static void add_foreignfds_to_poll(struct tevent_context *ev,
- struct foreign_pollfd *pollfds,
- uint64_t numfds)
-{
- struct tevent_fd *fde;
- uint64_t i;
- for (i = 0; i < numfds; i++) {
- /* set flags so the fd is on theactive list */
- fde = poll_event_add_fd(ev, NULL, pollfds[i].fd,
- TEVENT_FD_READ | TEVENT_FD_WRITE,
- foreign_fd_handler, &pollfds[i],
- "foreign_fd_handler", __location__);
-// tevent_debug(ev, TEVENT_DEBUG_WARNING,"adding foreign fd %d\n", fde->fd);
- fde->flags |= FOREIGN_FD;
- ev->foreign_poll.events_valid = true;
- }
-}
-
/*
do a single event loop using the events defined in ev
*/
@@ -925,14 +878,13 @@ static int poll_event_loop_wait(struct tevent_context *ev,
return 0;
}
-static bool poll_add_foreign_fd_handler(struct tevent_context *ev,
+static bool poll_add_foreign_fd_handler_mt(struct tevent_context *ev,
struct foreign_loop_handler *integrator,
void *private_data,
const char *location)
{
- ev->foreign_poll.handler = integrator;
- ev->foreign_poll.private_data = private_data;
- return true;
+ /* FIXME */
+ return false;
}
static const struct tevent_ops poll_event_ops = {
@@ -946,7 +898,7 @@ static const struct tevent_ops poll_event_ops = {
.add_signal = tevent_common_add_signal,
.loop_once = poll_event_loop_once,
.loop_wait = poll_event_loop_wait,
- .add_foreign_fd_handler = poll_add_foreign_fd_handler,
+ .add_foreign_fd_handler = common_add_foreign_fd_handler,
};
_PRIVATE_ bool tevent_poll_init(void)
@@ -965,7 +917,7 @@ static const struct tevent_ops poll_event_mt_ops = {
.add_signal = tevent_common_add_signal,
.loop_once = poll_event_loop_once,
.loop_wait = poll_event_loop_wait,
- .add_foreign_fd_handler = poll_add_foreign_fd_handler,
+ .add_foreign_fd_handler = poll_add_foreign_fd_handler_mt,
};
_PRIVATE_ bool tevent_poll_mt_init(void)
--
2.1.4
From 828297e1e1fec03482884981edec7f097db517af Mon Sep 17 00:00:00 2001
From: Noel Power <noel.power at suse.com>
Date: Tue, 15 Dec 2015 15:35:20 +0000
Subject: [PATCH 7/9] add a sample glib/tevent integration example
Adding some simple tracker-sparql async code that can use either
tevent or glib for the main event loop, the glib integration piggybacks
on newly added tevent functionallity for integrating foreign fds into the
underlying poll/select/epoll backends
---
source3/utils/async-tracker.c | 454 ++++++++++++++++++++++++++++++++++++++++++
source3/wscript_build | 7 +
2 files changed, 461 insertions(+)
create mode 100644 source3/utils/async-tracker.c
diff --git a/source3/utils/async-tracker.c b/source3/utils/async-tracker.c
new file mode 100644
index 0000000..be68267
--- /dev/null
+++ b/source3/utils/async-tracker.c
@@ -0,0 +1,454 @@
+/*
+ * Copyright (C) 2011, Nokia <ivan.frade at nokia.com>
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public
+ * License as published by the Free Software Foundation; either
+ * version 2 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public
+ * License along with this library; if not, write to the
+ * Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor,
+ * Boston, MA 02110-1301, USA.
+ */
+
+#include <stdlib.h>
+#include <tevent.h>
+#include "tevent_util.h"
+#include "tevent_internal.h"
+#include <libtracker-sparql/tracker-sparql.h>
+
+#include <errno.h>
+
+//#define DEBUG(x,y) printf y
+#define DEBUG(x,y)
+/*
+ * loop_type = 0 (default) example runs with glib event loop
+ * loop_type = 1 example (-t) runs with tevent loop
+ */
+static int loop_type = 0;
+
+typedef struct {
+ TrackerSparqlConnection *connection;
+ GCancellable *cancellable;
+ GTimer *timer;
+ GMainLoop *loop;
+ struct tevent_context *ev;
+} MyData;
+
+static void cleanup(MyData *md)
+{
+ g_cancellable_cancel (md->cancellable);
+ g_object_unref (md->cancellable);
+ g_timer_destroy (md->timer);
+ if (loop_type == 0) {
+ g_main_loop_quit (md->loop);
+ g_main_loop_unref (md->loop);
+ } else {
+ /*
+ * with glib there are always events, the
+ * normal g_main_loop_run runs forever (until someone
+ * tells it to stop processing events) thus when using
+ * tevent glib always gives us a fd to listen to.
+ */
+ common_remove_foreign_fds(md->ev);
+ }
+
+ DEBUG(0,("Async CLEANUP!!\n"));
+}
+
+static void
+cursor_cb (GObject *object,
+ GAsyncResult *res,
+ gpointer user_data)
+{
+ TrackerSparqlCursor *cursor;
+ GError *error = NULL;
+ MyData *md = user_data;
+ gboolean more_results;
+
+ cursor = TRACKER_SPARQL_CURSOR (object);
+ more_results = tracker_sparql_cursor_next_finish (cursor,
+ res,
+ &error);
+
+ if (!error) {
+ static gint i = 0;
+
+ if (more_results) {
+ if (i++ < 5) {
+ int num_cols = tracker_sparql_cursor_get_n_columns (cursor);
+ int col;
+ if (i == 1) {
+ g_print ("Printing first 5 results:\n");
+ }
+ for (col = 0; col < num_cols; col++) {
+ g_print (" %s ", tracker_sparql_cursor_get_string (cursor, col, NULL));
+ if (col == num_cols -1 ) {
+ g_print("\n");
+ }
+ }
+
+ if (i == 5) {
+ g_print (" ...\n");
+ g_print (" Printing nothing for remaining results\n");
+ }
+ }
+
+ tracker_sparql_cursor_next_async (cursor,
+ md->cancellable,
+ cursor_cb,
+ md);
+ } else {
+ g_print ("\n");
+ g_print ("Async cursor next took: %.6f (for all %d results)\n",
+ g_timer_elapsed (md->timer, NULL), i);
+
+ g_object_unref (cursor);
+ cleanup(md);
+ }
+ } else {
+ g_critical ("Could not run cursor next: %s", error->message);
+
+ if (cursor) {
+ g_object_unref (cursor);
+ }
+
+ g_error_free (error);
+ cleanup(md);
+ }
+}
+
+static void
+query_cb (GObject *object,
+ GAsyncResult *res,
+ gpointer user_data)
+{
+ TrackerSparqlCursor *cursor;
+ GError *error = NULL;
+ MyData *md = user_data;
+
+ cursor = tracker_sparql_connection_query_finish (TRACKER_SPARQL_CONNECTION (object),
+ res,
+ &error);
+ g_print ("Async query took: %.6f\n", g_timer_elapsed (md->timer, NULL));
+
+ g_timer_start (md->timer);
+
+ if (!error) {
+ tracker_sparql_cursor_next_async (cursor,
+ md->cancellable,
+ cursor_cb,
+ md);
+ } else {
+ g_critical ("Could not run query: %s", error->message);
+
+ if (cursor) {
+ g_object_unref (cursor);
+ }
+
+ g_error_free (error);
+ cleanup(md);
+ }
+}
+
+static void
+connection_cb (GObject *object,
+ GAsyncResult *res,
+ gpointer user_data)
+{
+ MyData *md = user_data;
+ GError *error = NULL;
+
+ md->connection = tracker_sparql_connection_get_finish (res, &error);
+ g_print ("Async connection took: %.6f\n", g_timer_elapsed (md->timer, NULL));
+
+ g_timer_start (md->timer);
+
+ if (!error) {
+ tracker_sparql_connection_query_async (md->connection,
+ "SELECT ?name nie:mimeType(?s) nfo:fileName(?s) WHERE { {?s nie:url ?name}}",
+ md->cancellable,
+ query_cb,
+ md);
+ } else {
+ g_critical ("Could not connect: %s", error->message);
+ g_error_free (error);
+ cleanup(md);
+ }
+}
+
+/*
+ * glib/tevent glue specific data
+ */
+struct loop_integration_data
+{
+ GPollFD *glib_fds;
+ size_t glib_fds_size;
+
+ struct foreign_pollfd *tevent_fds;
+ size_t tevent_fds_size;
+
+ struct timeval current_timeout;
+ struct timeval default_timeout;
+
+ gint priority;
+ gint num_active_fds;
+};
+
+static int glib_query(GMainContext *ctx,
+ struct loop_integration_data *glib_data,
+ int *p_timer)
+{
+ gint num_glib_fds = 0;
+ if (glib_data->glib_fds_size == 0)
+ {
+ glib_data->glib_fds =
+ talloc_zero_array(glib_data, GPollFD, 20);
+ glib_data->glib_fds_size = 20;
+ glib_data->tevent_fds =
+ talloc_zero_array(glib_data,
+ struct foreign_pollfd,
+ glib_data->glib_fds_size);
+ glib_data->tevent_fds_size = glib_data->glib_fds_size;
+ }
+
+ while (1)
+ {
+ num_glib_fds =
+ g_main_context_query(ctx,
+ glib_data->priority,
+ p_timer,
+ glib_data->glib_fds,
+ glib_data->glib_fds_size);
+ DEBUG(0,("num_glib_fds = %d timer = %d\n", num_glib_fds, *p_timer));
+ if (num_glib_fds <= glib_data->glib_fds_size) {
+ break;
+ }
+ glib_data->glib_fds = talloc_realloc(glib_data,
+ glib_data->glib_fds,
+ GPollFD, num_glib_fds);
+ glib_data->glib_fds_size = num_glib_fds;
+ /* #FIXME check for error talloc errors above */
+ }
+ if (glib_data->tevent_fds_size != glib_data->glib_fds_size) {
+ glib_data->tevent_fds = talloc_realloc(glib_data,
+ glib_data->tevent_fds,
+ struct foreign_pollfd,
+ num_glib_fds);
+ glib_data->tevent_fds_size = glib_data->glib_fds_size;
+
+ }
+ return num_glib_fds;
+}
+
+static bool glib_poll_prepare(GPollFD *pfds,
+ struct foreign_pollfd *ffds,
+ int count)
+{
+ int i;
+ for (i = 0; i < count; i++) {
+ ffds[i].fd = pfds[i].fd;
+ if (pfds[i].events & G_IO_IN) {
+ ffds[i].events |= FOREIGN_FD_READ;
+ }
+ if (pfds[i].events & G_IO_OUT) {
+ ffds[i].events |= FOREIGN_FD_WRITE;
+ }
+ if (pfds[i].events & G_IO_HUP) {
+ ffds[i].events |= FOREIGN_FD_HUP;
+ }
+ if (pfds[i].events & G_IO_ERR) {
+ ffds[i].events |= FOREIGN_FD_ERR;
+ }
+ pfds[i].revents = 0;
+ ffds[i].revents = 0;
+ }
+ return true;
+}
+
+static struct timeval * prepare_poll(struct foreign_pollfd **pollfds,
+ uint64_t *numfds, void *private_data)
+{
+ int glib_timeout;
+ bool source_ready = false;
+ struct loop_integration_data *glib_data = private_data;
+
+ GMainContext *ctx = g_main_context_default();
+
+ /*
+ * this looks particually dodgy, is it? just looks like there is
+ * potential for some holdup/spinning here due to waiting for the
+ * context, guess there is nothing that can be done
+ */
+ while (!g_main_context_acquire(ctx));
+
+ // PREPARE
+ source_ready = g_main_context_prepare(ctx, &glib_data->priority);
+ // QUERY
+ *numfds = glib_query(ctx, glib_data, &glib_timeout);
+ glib_data->num_active_fds = *numfds;
+ *pollfds = glib_data->tevent_fds;
+ DEBUG(0,("##### priority %d, timeout %d source_read = %s for ctx %p\n",
+ glib_data->priority, glib_timeout, source_ready ? "true" : "false", ctx ));
+ if (glib_timeout == -1) {
+ glib_data->current_timeout = glib_data->default_timeout;
+ } else {
+ glib_data->current_timeout.tv_sec = glib_timeout / 1000;
+ glib_data->current_timeout.tv_usec = (glib_timeout % 1000) * 1000;
+ }
+ glib_poll_prepare(glib_data->glib_fds, glib_data->tevent_fds, *numfds);
+ /*
+ * for some reason if don't do the entire;
+ * prepare/query/poll/check/dispatch set of phases with the
+ * the context acquired even a simple glib application seems to take
+ * a significantly longer time to complete.
+ * note: epoll/poll/select should be performed immediately by
+ * tevent when we return from here.
+ */
+ return &glib_data->current_timeout;
+}
+
+static void check_poll(uint64_t nfds, void *private_data)
+{
+ GMainContext *ctx = g_main_context_default();
+ int i;
+ struct loop_integration_data *glib_data = private_data;
+
+ DEBUG(0,("#### checkpoll for ctx %p nfds %ld\n", ctx, nfds));
+
+
+ for (i = 0; nfds && i < glib_data->num_active_fds; i++) {
+ if (glib_data->tevent_fds[i].revents & FOREIGN_FD_READ) {
+ glib_data->glib_fds[i].revents |= G_IO_IN;
+ }
+ if (glib_data->tevent_fds[i].revents & FOREIGN_FD_WRITE) {
+ glib_data->glib_fds[i].revents |= G_IO_OUT;
+ }
+ if (glib_data->tevent_fds[i].revents & FOREIGN_FD_HUP) {
+ glib_data->glib_fds[i].revents |= G_IO_HUP;
+ }
+ if (glib_data->tevent_fds[i].revents & FOREIGN_FD_ERR) {
+ glib_data->glib_fds[i].revents |= G_IO_ERR;
+ }
+ }
+ // CHECK
+ if (g_main_context_check(ctx, glib_data->priority, glib_data->glib_fds,
+ glib_data->num_active_fds)) {
+ // DISPATCH
+ DEBUG(0,("dispatching...\n"));
+ g_main_context_dispatch(ctx);
+ }
+ g_main_context_release(ctx);
+}
+
+/*
+ * this functionality could be moved from client code to library code
+ * either a samba util library or even maybe expose it in tevent itself
+ */
+static struct tevent_context *tevent_integrate_glib(TALLOC_CTX *mem_ctx,
+ struct tevent_context *ev)
+{
+ struct tevent_context *event_ctx = ev;
+ struct loop_integration_data *glib_data =
+ talloc_zero(mem_ctx, struct loop_integration_data);
+ struct foreign_loop_handler *handler =
+ talloc_zero(mem_ctx, struct foreign_loop_handler);
+
+ if (!event_ctx) {
+ /*
+ * currently only 'poll' backend supports poll/select
+ * of foreign fd(s)
+ */
+ event_ctx = tevent_context_init_byname(mem_ctx, "poll");
+ }
+
+ /*
+ * #FIXME
+ * default_timeout = tevent_common_loop_timer_delay(event_ctx);
+ * is there a better way to get at the timout ? if we include
+ * tevent_internal.h we can get it but...
+ */
+ glib_data->default_timeout = tevent_common_loop_timer_delay(event_ctx);
+
+ handler->prepare_poll = prepare_poll;
+ handler->check_poll = check_poll;
+ if (!tevent_add_foreign_fd_handler(event_ctx, handler, glib_data)) {
+ DEBUG(0,("failed to add foreign poll handler\n"));
+ if (!ev) {
+ TALLOC_FREE(event_ctx);
+ return NULL;
+ }
+ }
+ tevent_set_debug_stderr(event_ctx);
+ return event_ctx;
+}
+
+static void usage(void)
+{
+ printf("tevent_glib [-t|-h|-g]\n"
+ "\t -t use tevent event loop\n"
+ "\t -g use glib event loop (default)\n"
+ "\t -h help\n");
+ exit(1);
+}
+
+gint
+main (gint argc, gchar *argv[])
+{
+ MyData *md;
+ TALLOC_CTX *mem_ctx = talloc_new(NULL); //parent
+ struct tevent_context *event_ctx;
+ int i;
+ md = g_new0 (MyData, 1);
+ /*
+ * hacky cmd line processing
+ * -t = use tevent event loop
+ * -g = use glib event loop
+ * -h = usage
+ */
+ if (argc > 1) {
+ for (i = 1; i < argc; i++) {
+ if (strcmp(argv[i],"-t") == 0) {
+ loop_type = 1;
+ } else if (strcmp(argv[i],"-g") == 0) {
+ loop_type = 0;
+ } else if (strcmp(argv[i],"-h") == 0) {
+ usage();
+ } else {
+ usage();
+ }
+ }
+ }
+ if (loop_type == 0) {
+ md->loop = g_main_loop_new (NULL, FALSE);
+ }
+ md->timer = g_timer_new ();
+ md->cancellable = g_cancellable_new ();
+ tracker_sparql_connection_get_async (md->cancellable,
+ connection_cb,
+ md);
+ //while (!tevent_loop_once(event_ctx));
+ if (loop_type == 0) {
+ printf("entering g_main_loop_run\n");
+ g_main_loop_run (md->loop);
+ } else {
+ event_ctx = tevent_integrate_glib(mem_ctx, NULL);
+ md->ev = event_ctx;
+ printf("entering tevent_loop_wait\n");
+ tevent_loop_wait(event_ctx);
+ }
+ talloc_free(mem_ctx);
+
+ if (md->connection) {
+ g_object_unref (md->connection);
+ }
+ g_free (md);
+
+ return EXIT_SUCCESS;
+}
diff --git a/source3/wscript_build b/source3/wscript_build
index e66e89f..adf210d 100755
--- a/source3/wscript_build
+++ b/source3/wscript_build
@@ -1364,6 +1364,13 @@ bld.SAMBA3_BINARY('eventlogadm',
param
LIBEVENTLOG''')
+bld.SAMBA3_BINARY('tevent_glib',
+ source='utils/async-tracker.c',
+ deps='''
+ talloc
+ tevent ''' + bld.env['libtracker'],
+ enabled=bld.env.with_spotlight) #FIXME should be conditional with something tracker and/or glib related
+
bld.SAMBA3_BINARY('sharesec',
source='utils/sharesec.c lib/util_sd.c',
deps='''
--
2.1.4
From 00e4d73e861830372f7355dbeac7cbe54c70142f Mon Sep 17 00:00:00 2001
From: Noel Power <noel.power at suse.com>
Date: Tue, 15 Dec 2015 16:07:52 +0000
Subject: [PATCH 8/9] add support for integrating foreign poll fds into 's3'
tevent backend
---
source3/lib/events.c | 164 ++++++++++++++++++++++++++++++++++++++++++++++++---
1 file changed, 157 insertions(+), 7 deletions(-)
diff --git a/source3/lib/events.c b/source3/lib/events.c
index 0bc56e4..f5b783c 100644
--- a/source3/lib/events.c
+++ b/source3/lib/events.c
@@ -145,12 +145,33 @@ bool event_add_to_poll_args(struct tevent_context *ev, TALLOC_CTX *mem_ctx,
}
pfd->fd = fde->fd;
+ if (fde->flags & FOREIGN_FD) {
+ struct foreign_pollfd *ffd =
+ (struct foreign_pollfd *)fde->private_data;
+ if (ffd->events & FOREIGN_FD_READ) {
+ pfd->events |= POLLIN;
+ DEBUG(0,("adding read event fo foreign fd %d\n", fde->fd));
+ }
+ if (ffd->events & FOREIGN_FD_WRITE) {
+ pfd->events |= POLLOUT;
+ DEBUG(0,("adding write event fo foreign fd %d\n", fde->fd));
+ }
+ if (ffd->events & FOREIGN_FD_ERR) {
+ pfd->events |= POLLERR;
+ }
+ if (ffd->events & FOREIGN_FD_HUP) {
+ pfd->events |= POLLHUP;
+ }
+ ffd->revents = 0;
+
+ } else {
- if (fde->flags & TEVENT_FD_READ) {
- pfd->events |= (POLLIN|POLLHUP);
- }
- if (fde->flags & TEVENT_FD_WRITE) {
- pfd->events |= POLLOUT;
+ if (fde->flags & TEVENT_FD_READ) {
+ pfd->events |= (POLLIN|POLLHUP);
+ }
+ if (fde->flags & TEVENT_FD_WRITE) {
+ pfd->events |= POLLOUT;
+ }
}
}
*pfds = fds;
@@ -213,7 +234,11 @@ bool run_events_poll(struct tevent_context *ev, int pollrtn,
for (fde = ev->fd_events; fde; fde = fde->next) {
struct pollfd *pfd;
uint16_t flags = 0;
-
+ bool is_foreign_fd =
+ (fde->flags & FOREIGN_FD);
+ if (is_foreign_fd) {
+ continue;
+ }
if ((fde->flags & (TEVENT_FD_READ|TEVENT_FD_WRITE)) == 0) {
continue;
}
@@ -284,6 +309,90 @@ struct timeval *get_timed_events_timeout(struct tevent_context *ev,
return to_ret;
}
+static void foreign_fd_handler(struct tevent_context *ev,
+ struct tevent_fd *fde,
+ uint16_t flags,
+ void *private_data)
+{
+ struct foreign_pollfd *ffd =
+ (struct foreign_pollfd *)fde->private_data;
+
+ DEBUG(0,("handler fired for foreign fd %d\n", fde->fd));
+ /* convert poll event to FOREIGN_FD_XYZ event */
+ if (fde->additional_flags & POLLIN) {
+ DEBUG(0,("handler fired for foreign fd READ%d\n", fde->fd));
+ ffd->revents |= FOREIGN_FD_READ;
+ }
+ if (fde->additional_flags & POLLOUT) {
+ ffd->revents |= FOREIGN_FD_WRITE;
+ DEBUG(0,("handler fired for foreign fd WRITE%d\n", fde->fd));
+ }
+ if (fde->additional_flags & POLLHUP) {
+ ffd->revents |= FOREIGN_FD_HUP;
+ }
+ if (fde->additional_flags & POLLERR) {
+ ffd->revents |= FOREIGN_FD_ERR;
+ }
+}
+
+static void process_foreign_fds(struct tevent_context *ev, int num_pfds, bool report_only)
+{
+ struct tevent_fd *fde = NULL;
+ struct tevent_fd *next = NULL;
+ struct tevent_poll_private *state = tevent_get_poll_private(ev);
+ struct foreign_loop_handler *integrator = ev->foreign_poll.handler;
+ void *priv_data = ev->foreign_poll.private_data;
+ int *pollfd_idx;
+ int i = 0;
+ DEBUG(0,("enter check_and_rm_foreign_fds\n"));
+ if (!integrator) {
+ return;
+ }
+ pollfd_idx = state->pollfd_idx;
+
+ for (fde = ev->fd_events; !report_only && fde; fde = next) {
+ struct pollfd *pfd;
+ bool is_foreign_fd =
+ (fde->flags & FOREIGN_FD);
+
+ next = fde->next;
+ if (!is_foreign_fd) {
+ continue;
+ }
+
+ if (pollfd_idx[fde->fd] >= num_pfds) {
+ /* need to error out or something */
+ DEBUG(1, ("internal error: pollfd_idx[fde->fd] (%d) "
+ ">= num_pfds (%d)\n", pollfd_idx[fde->fd],
+ num_pfds));
+ i = 0;
+ break;
+ }
+ pfd = &state->pfds[pollfd_idx[fde->fd]];
+ if (pfd->revents & (POLLHUP|POLLERR)) {
+ fde->additional_flags |=
+ ((pfd->revents & (POLLHUP|POLLERR)));
+ }
+ if (pfd->revents & POLLIN) {
+ fde->additional_flags |= POLLIN;
+ }
+ if (pfd->revents & POLLOUT)
+ fde->additional_flags |= POLLOUT;
+ if (pfd->revents) {
+ i++;
+ }
+ fde->handler(ev, fde, fde->flags, fde->private_data);
+ }
+ DEBUG(0,("leaving check_and_rm_foreign_fds and about to call check_poll\n"));
+ integrator->check_poll(i, priv_data);
+
+ /*
+ * ensure while in outer loop_wait will terminate if we dont have
+ * any fds to read.
+ */
+ ev->foreign_poll.events_valid = false;
+}
+
static int s3_event_loop_once(struct tevent_context *ev, const char *location)
{
struct tevent_poll_private *state;
@@ -291,7 +400,7 @@ static int s3_event_loop_once(struct tevent_context *ev, const char *location)
int num_pfds;
int ret;
int poll_errno;
-
+ struct foreign_loop_handler *poll_handler = NULL;
timeout = INT_MAX;
state = tevent_get_poll_private(ev);
@@ -300,16 +409,49 @@ static int s3_event_loop_once(struct tevent_context *ev, const char *location)
return -1;
}
+ poll_handler = ev->foreign_poll.handler;
if (run_events_poll(ev, 0, NULL, 0)) {
return 0;
}
num_pfds = 0;
+
+ if (poll_handler) {
+ struct timeval *foreign_tvalp = NULL;
+ struct foreign_pollfd *pollfds;
+ uint64_t numfds;
+ void *priv_data = ev->foreign_poll.private_data;
+ /* remove previous fds */
+ common_remove_foreign_fds(ev);
+ foreign_tvalp = poll_handler->prepare_poll(&pollfds, &numfds,
+ priv_data);
+ /*
+ * We may need to to override the timeout, e.g. glib is very
+ * dependant on having the correct (and often even '0')
+ * timeout, failure to honour that results in unacceptable
+ * performance from glib triggered functionality.
+ */
+ if (foreign_tvalp) {
+ /*
+ * smallest timeout should be used.
+ */
+ int new_timeout = foreign_tvalp->tv_sec * 1000;
+ new_timeout += (foreign_tvalp->tv_usec + 999) / 1000;
+ if (new_timeout < timeout) {
+ timeout = new_timeout;
+ }
+ }
+ common_add_foreignfds(ev, foreign_fd_handler, pollfds, numfds);
+ }
if (!event_add_to_poll_args(ev, state,
&state->pfds, &num_pfds, &timeout)) {
+ if (poll_handler) {
+ process_foreign_fds(ev, num_pfds, true);
+ }
return -1;
}
+ DEBUG(0,("event_add_to_poll_args says %d fds to process\n", num_pfds));
tevent_trace_point_callback(ev, TEVENT_TRACE_BEFORE_WAIT);
ret = poll(state->pfds, num_pfds, timeout);
poll_errno = errno;
@@ -320,9 +462,15 @@ static int s3_event_loop_once(struct tevent_context *ev, const char *location)
tevent_debug(ev, TEVENT_DEBUG_FATAL,
"poll() failed: %d:%s\n",
errno, strerror(errno));
+ if (poll_handler) {
+ process_foreign_fds(ev, num_pfds, true);
+ }
return -1;
}
+ if (poll_handler) {
+ process_foreign_fds(ev, num_pfds, false);
+ }
run_events_poll(ev, ret, state->pfds, num_pfds);
return 0;
}
@@ -377,6 +525,7 @@ static const struct tevent_ops s3_event_ops = {
.add_signal = tevent_common_add_signal,
.loop_once = s3_event_loop_once,
.loop_wait = tevent_common_loop_wait,
+ .add_foreign_fd_handler = common_add_foreign_fd_handler,
};
static bool s3_tevent_init(void)
@@ -484,3 +633,4 @@ struct idle_event *event_add_idle(struct tevent_context *event_ctx,
return result;
}
+
--
2.1.4
From eae9982000eaf1bd4345900e561fd09de64d96b0 Mon Sep 17 00:00:00 2001
From: Noel Power <noel.power at suse.com>
Date: Tue, 15 Dec 2015 16:25:20 +0000
Subject: [PATCH 9/9] add tevent_integrate_glib function for s3 tevent backend
with a simple call to tevent_integrate_glib the 's3' backend tevent event loop
can additionally service glib events
---
source3/include/event.h | 5 ++
source3/lib/events.c | 194 ++++++++++++++++++++++++++++++++++++++++++++++++
source3/wscript_build | 2 +-
3 files changed, 200 insertions(+), 1 deletion(-)
diff --git a/source3/include/event.h b/source3/include/event.h
index 108026e..77a9ad1 100644
--- a/source3/include/event.h
+++ b/source3/include/event.h
@@ -40,3 +40,8 @@ struct idle_event *event_add_idle(struct tevent_context *event_ctx,
bool (*handler)(const struct timeval *now,
void *private_data),
void *private_data);
+
+#ifdef WITH_SPOTLIGHT /* #FIXME needs to be GLIB related */
+struct tevent_context *tevent_integrate_glib(TALLOC_CTX *mem_ctx,
+ struct tevent_context *ev);
+#endif
diff --git a/source3/lib/events.c b/source3/lib/events.c
index f5b783c..b5473d0 100644
--- a/source3/lib/events.c
+++ b/source3/lib/events.c
@@ -22,7 +22,201 @@
#include "lib/tevent/tevent_internal.h"
#include "../lib/util/select.h"
#include "system/select.h"
+#ifdef WITH_SPOTLIGHT /* #FIXME needs to be some glib related define */
+#include <glib.h>
+/*
+ * glib/tevent glue specific data
+ */
+struct loop_integration_data
+{
+ GPollFD *glib_fds;
+ size_t glib_fds_size;
+
+ struct foreign_pollfd *tevent_fds;
+ size_t tevent_fds_size;
+
+ struct timeval current_timeout;
+ struct timeval default_timeout;
+
+ gint priority;
+ gint num_active_fds;
+};
+
+static int glib_query(GMainContext *ctx,
+ struct loop_integration_data *glib_data,
+ int *p_timer)
+{
+ gint num_glib_fds = 0;
+ if (glib_data->glib_fds_size == 0)
+ {
+ glib_data->glib_fds =
+ talloc_zero_array(glib_data, GPollFD, 20);
+ glib_data->glib_fds_size = 20;
+ glib_data->tevent_fds =
+ talloc_zero_array(glib_data,
+ struct foreign_pollfd,
+ glib_data->glib_fds_size);
+ glib_data->tevent_fds_size = glib_data->glib_fds_size;
+ }
+
+ while (1)
+ {
+ num_glib_fds =
+ g_main_context_query(ctx,
+ glib_data->priority,
+ p_timer,
+ glib_data->glib_fds,
+ glib_data->glib_fds_size);
+ DEBUG(0,("num_glib_fds = %d timer = %d\n", num_glib_fds, *p_timer));
+ if (num_glib_fds <= glib_data->glib_fds_size) {
+ break;
+ }
+ glib_data->glib_fds = talloc_realloc(glib_data,
+ glib_data->glib_fds,
+ GPollFD, num_glib_fds);
+ glib_data->glib_fds_size = num_glib_fds;
+ /* #FIXME check for error talloc errors above */
+ }
+ if (glib_data->tevent_fds_size != glib_data->glib_fds_size) {
+ glib_data->tevent_fds = talloc_realloc(glib_data,
+ glib_data->tevent_fds,
+ struct foreign_pollfd,
+ num_glib_fds);
+ glib_data->tevent_fds_size = glib_data->glib_fds_size;
+ }
+ return num_glib_fds;
+}
+
+static bool glib_poll_prepare(GPollFD *pfds,
+ struct foreign_pollfd *ffds,
+ int count)
+{
+ int i;
+ for (i = 0; i < count; i++) {
+ ffds[i].fd = pfds[i].fd;
+ if (pfds[i].events & G_IO_IN) {
+ ffds[i].events |= FOREIGN_FD_READ;
+ }
+ if (pfds[i].events & G_IO_OUT) {
+ ffds[i].events |= FOREIGN_FD_WRITE;
+ }
+ if (pfds[i].events & G_IO_HUP) {
+ ffds[i].events |= FOREIGN_FD_HUP;
+ }
+ if (pfds[i].events & G_IO_ERR) {
+ ffds[i].events |= FOREIGN_FD_ERR;
+ }
+ pfds[i].revents = 0;
+ ffds[i].revents = 0;
+ }
+ return true;
+}
+
+static struct timeval * prepare_poll(struct foreign_pollfd **pollfds,
+ uint64_t *numfds, void *private_data)
+{
+ int glib_timeout;
+ bool source_ready = false;
+ struct loop_integration_data *glib_data = private_data;
+
+ GMainContext *ctx = g_main_context_default();
+
+ /*
+ * this looks particually dodgy, is it? just looks like there is
+ * potential for some holdup/spinning here due to waiting for the
+ * context, guess there is nothing that can be done
+ */
+ while (!g_main_context_acquire(ctx));
+
+ // PREPARE
+ source_ready = g_main_context_prepare(ctx, &glib_data->priority);
+ // QUERY
+ *numfds = glib_query(ctx, glib_data, &glib_timeout);
+ glib_data->num_active_fds = *numfds;
+ *pollfds = glib_data->tevent_fds;
+ DEBUG(0,("##### priority %d, timeout %d source_read = %s for ctx %p\n",
+ glib_data->priority, glib_timeout, source_ready ? "true" : "false", ctx ));
+ if (glib_timeout == -1) {
+ glib_data->current_timeout = glib_data->default_timeout;
+ } else {
+ glib_data->current_timeout.tv_sec = glib_timeout / 1000;
+ glib_data->current_timeout.tv_usec = (glib_timeout % 1000) * 1000;
+ }
+ glib_poll_prepare(glib_data->glib_fds, glib_data->tevent_fds, *numfds);
+ /*
+ * for some reason if don't do the entire;
+ * prepare/query/poll/check/dispatch set of phases with the
+ * the context acquired event a simple glib application seems to take
+ * a significantly longer time to complete.
+ * note: epoll/poll/select should be performed immediately by
+ * tevent when we return from here.
+ */
+ return &glib_data->current_timeout;
+}
+
+static void check_poll(uint64_t nfds, void *private_data)
+{
+ GMainContext *ctx = g_main_context_default();
+ int i;
+ struct loop_integration_data *glib_data = private_data;
+
+ DEBUG(0,("#### checkpoll for ctx %p nfds %ld\n", ctx, nfds));
+
+
+ for (i = 0; nfds && i < glib_data->num_active_fds; i++) {
+ if (glib_data->tevent_fds[i].revents & FOREIGN_FD_READ) {
+ glib_data->glib_fds[i].revents |= G_IO_IN;
+ }
+ if (glib_data->tevent_fds[i].revents & FOREIGN_FD_WRITE) {
+ glib_data->glib_fds[i].revents |= G_IO_OUT;
+ }
+ if (glib_data->tevent_fds[i].revents & FOREIGN_FD_HUP) {
+ glib_data->glib_fds[i].revents |= G_IO_HUP;
+ }
+ if (glib_data->tevent_fds[i].revents & FOREIGN_FD_ERR) {
+ glib_data->glib_fds[i].revents |= G_IO_ERR;
+ }
+ }
+ // CHECK
+ if (g_main_context_check(ctx, glib_data->priority, glib_data->glib_fds,
+ glib_data->num_active_fds)) {
+ // DISPATCH
+ DEBUG(0,("dispatching...\n"));
+ g_main_context_dispatch(ctx);
+ }
+ g_main_context_release(ctx);
+}
+
+struct tevent_context *tevent_integrate_glib(TALLOC_CTX *mem_ctx,
+ struct tevent_context *ev)
+{
+ struct tevent_context *event_ctx = ev;
+ struct loop_integration_data *glib_data =
+ talloc_zero(mem_ctx, struct loop_integration_data);
+ struct foreign_loop_handler *handler =
+ talloc_zero(mem_ctx, struct foreign_loop_handler);
+
+ if (!event_ctx) {
+ DEBUG(0,("jeez no event loop, exiting\n"));
+ exit(1);
+ }
+
+ glib_data->default_timeout = tevent_common_loop_timer_delay(ev);
+
+ handler->prepare_poll = prepare_poll;
+ handler->check_poll = check_poll;
+ if (!tevent_add_foreign_fd_handler(event_ctx, handler, glib_data)) {
+ DEBUG(0,("failed to add foreign poll handler\n"));
+ if (!ev) {
+ TALLOC_FREE(event_ctx);
+ return NULL;
+ }
+ }
+ return event_ctx;
+}
+
+#endif
struct tevent_poll_private {
/*
* Index from file descriptor into the pollfd array
diff --git a/source3/wscript_build b/source3/wscript_build
index adf210d..ee3cd4c 100755
--- a/source3/wscript_build
+++ b/source3/wscript_build
@@ -360,7 +360,7 @@ bld.SAMBA3_SUBSYSTEM('samba3core',
messages_util
messages_dgm
talloc_report
- TDB_LIB''')
+ TDB_LIB ''' + bld.env['libtracker']) #FIXME need a glib dependency not libtracker
bld.SAMBA3_LIBRARY('smbd_shim',
source='''lib/smbd_shim.c''',
--
2.1.4
More information about the samba-technical
mailing list