tevent in multithreaded programs

Simo simo at samba.org
Fri Jan 30 13:03:25 MST 2015


On Fri, 2015-01-30 at 20:49 +0100, Volker Lendecke wrote:
> On Fri, Jan 30, 2015 at 12:30:27PM -0500, Jeff Layton wrote:
> > Hi All!
> > 
> > I'm looking at using tevent in a multithreaded program, primarily to
> > handle socket descriptors. The basic idea is to have a tevent_wait loop
> > running in one thread that will handle events by dispatching them to a
> > workqueue (queue of jobs and a pile of threads to handle them).
> > 
> > So basically we'll open a listener socket, and set it up to be watched
> > by tevent and run that in one thread. When a connect request comes in,
> > I want to dispatch the accept to a workqueue, which will then add the
> > accept()ed fd to the event context.
> > 
> > Questions:
> > 
> > - is this legit? Can I add events in one thread while the event loop is
> >   running in another?
> > 
> > - if so, do I need to serialize that somehow (with pthread mutexes or
> >   something)? I assume so, since tevent is based on talloc and you need
> >   to synchronize between threads when dealing with talloc contexts...
> 
> You pretty much nailed it I guess. talloc does not play well
> with threads if you want to share contexts. If you have a
> talloc root per thread it should be okay, but the NULL
> context is always a problem.
> 
> Who will handle the tevent_loop_once? If it's only one
> thread, this might work. However, doing an tevent_add_fd
> from a thread where another thread loops in
> tevent_loop_once this will cause problems I guess. I'd
> rather use other mechanisms to make the tevent_loop_once
> thread add the fd.

FWIW, I had a chat with Jeff on IRC and suggest him to use the same
approach we use to handle signals.

Namely create a sentinel pipe and add it as fd event, also create a main
list that is accessed via mutexes (or other suitable locking or lockless
mechanism).
When the non-main threads need to add fd events to the main loop, add
the fd to the main list then write() 1 byte to the sentinel pipe.
The write() will cause the main loop to run a fd handler that will take
care of drying up both the pipe and the main list and create appropriate
fd events in it.

Simo.




More information about the samba-technical mailing list