[PATCH] restructure messaging

[Andrew Bartlett]

On Mon, 2016-10-03 at 19:53 +0200, Volker Lendecke wrote:
> Andrew Bartlett <abartlet at samba.org> writes:
> 
> > For "[PATCH 20/30] messages_dgm: Drop a segment if we can't ship it
> > for
> > 60 seconds" does this impact on the IRPC use case?  Under what
> > situation would this occur?
> 
> It happens when the receiver is so blocked that it won't pick up
> anything from its socket. We have to protect ourselves against this
> somehow. Imagine someone sends a SIGSTOP to notifyd. Right now this
> will
> create a nice memory leak without bounds.
> 
> > I ask because it is quite possible that we might be in a
> > replication-
> > induced transaction for more than 60 seconds with the current
> > code. 
> 
> If the sending process has already handed this over to the kernel, it
> will arrive. 

Great, that should be enough.  Does that include in the single-process
case?  

> You could argue that the DC needs more time, but I looked
> at it from an SMB perspective: If there is something waiting for
> anything for more than 60 seconds, the client will usually be very,
> very
> unhappy.

Indeed, a lot of things are very unhappy at 60 seconds

> Is it a legitimate use case that something takes more than 60
> seconds?

Yes and No.  Currently our code can get stuck for that long and longer
blocked with the transaction lock held in the client-side of
replication.  That is a serious issue we are working on, as you can
imagine it creates pretty serious issues. 

> The alternative would be to block the sender, but this might have
> negative impact on client interaction.
> 
> Maybe we need to have two messaging subsystems again: One for AD and
> one
> for SMB. AD has legitimate use cases where everything can block for
> minutes or hours, SMB can not live with that.

Bugs aside, the main use case we have that can wait for minutes or
hours is waiting for replication to complete, but that isn't meant to
be blocking, but a very long time before the reply, and I think that is
quite different.

> Was it a mistake to unify the messaging subsystems?

I don't think so, I am really glad not to have a second messaging
system to maintain. 

> > Can you add a variation on "Subject: [PATCH 16/30] messaging: add
> > an
> > overflow test" that confirms that all the messages are picked up?
> 
> I'll try.

Thanks.  I'll follow up on that thread.

Andrew Bartlett

-- 
Andrew Bartlett
https://samba.org/~abartlet/
Authentication Developer, Samba Team         https://samba.org
Samba Development and Support, Catalyst IT   
https://catalyst.net.nz/services/samba