Truncated output from "rsync -e ssh ... 2>&1 | tee"

[Wayne Davison]

On Wed, Sep 29, 2004 at 08:44:43AM +0000, David Evers wrote:
> I'm not seeing how we know that ssh has finished messing with stderr
> by the time the patch comes to try to set it back to blocking.

The reason there's no race is that this call comes after we've begun to
talk over the socket to the remote rsync.  Since we've done a little bit
of communication already (but not enough to have generated any large
quantities of output), we know that ssh has finished its setup on its
file descriptors.

> More generally, there's presumably a reason why ssh wants to set its
> stderr non-blocking

Yes, but the big question is is any of that relevant to rsync?  Since
there is almost never any stderr output coming from the remote side (or
from ssh itself), and if it occurs it indicates that the whole shebang
is coming to a grinding halt anyway, I'm thinking that we don't care
that setting ssh's stderr to blocking might possibly irritate ssh in
some hard-to-trigger circumstance.  We also don't need to worry about
things like people using rsync's ssh for port forwarding.

I just ran a test where I caused the receiving rsync to output a debug
line on stderr for every chunk that it receives, and then ran this
modified rsync on a data transfer (using --verbose and --progress for
extra stdout output).  I compared the output of a merged stdout/stderr
stream that has a built-in delay (so that we know that the output
blocked) with a merged stream that has no delay and with an unmerged
stream captured via screen's built-in output logging feature.  All 3
worked fine and had no loss of data.

So I think that for rsync's use, this simple kluge will work fine (and
avoid a lot of extra complexity of creating another pipe to monitor the
stderr output from ssh).

If anyone disagrees or still has concerns about this, please express
them now -- rsync 2.6.3 is going to be released soon, and I'm thinking
that this is a safe enough change to include in the final release.

..wayne..