Resolving problems in the generator->receiver pipes

[jw schultz]

On Thu, Jan 15, 2004 at 12:11:39PM -0800, Wayne Davison wrote:
> When I was working on the the hard-link change, I noticed that many
> of the hard-link verbose messages were getting lost.  These messages
> get output very near the end of the transfer, and it turns out that
> the reason for the loss was that there are two pipes flowing from
> the generator and the receiver, and it was possible for the "we're
> all done" message to get received down the redo pipe without all of
> the messages getting sent down the error pipe.  It's also a long-
> known bug that the redo pipe can clog (but only if a really large
> number of redo items build up).
> 
> In looking at this code, I figured out that it would be much better
> to just dump the redo pipe and use the error pipe as a message pipe.
> Since it already uses the same transfer protocol as the multiplexed
> IO between other components, it turned out to be a very simple thing
> to add a REDO and a DONE message to the existing code.  This gets
> rid of the redo pipe and all of its problems.
> 
> At the same time I though it would be good to separate the existing
> message-sending code out of the log-file code.  This is because one
> other non-logging message is already supported (the "data" stream in
> multiplexed IO) so the addition of two extra messages makes it a
> better fit for the io.c code than the log.c code (i.e. when the log
> code needs to send a message to another program, it calls the code
> in io.c to do it instead of doing it itself).
> 
> Finally, the flushing code was tweaked to allow it to signal that
> the writing out of the receiver->generator pipe should be completely
> flushed (from the in-memory list of messages).  This allows the
> final flushing to complete more efficiently than calling io_flush()
> in a loop.
> 
> In my exuberance for having finally solved the redo-hang problem in
> a much better way than my previous suggested fixes for it, I went
> ahead and checked in my changes.  However, feedback is still
> gratefully accepted.  The changes can be found here:
> 
>     http://www.blorf.net/redo.patch

What i looked at from the cvs digest looked like decent
code.  It was just lacking the overview you provided here.

This sounds pretty good.  Just needs some thorough testing.

-- 
________________________________________________________________
	J.W. Schultz            Pegasystems Technologies
	email address:		jw at pegasys.ws

		Remember Cernan and Schmitt