[Bug 5478] rsync: writefd_unbuffered failed to write 4092 bytes [sender]: Broken pipe (32)

[samba-bugs at samba.org]

https://bugzilla.samba.org/show_bug.cgi?id=5478

--- Comment #19 from Eric Shubert <ejs at shubes.net> 2011-10-06 15:28:32 UTC ---
(In reply to comment #14)
> So, looking at your trace, it looks like the sender is waiting to write more
> data (its select lists the write and error FDs).  The receiving side's 2 pids
> are waiting to read more data.  So, something in between has either failed to
> deliver the data, or blocked, or something (it would appear). Are there other
> processes in between the 2 sides (e.g. ssh) that could have blocked?  Or
> perhaps a failed pipe (or unix domain socket)?

After further testing, I'm tending to agree that the problem lies outside of
rsync. Somehow, the select() is getting stuck (not receiving a signal from the
kernel?). I'm not yet certain which side gets stuck first, but I would guess
that it's the target, given that in one test there was about 2MB of data "in
the pipe", that had been written by the source but not read yet by the target.

Working on the idea that the failure happens when the pipe is saturated, I
decided to try the bwlimit option to slow things down a bit. The upload speed
at the source is limited to about 100KBPS (DSL, shared with other users), so I
tried bwlimit=64. This worked noticeably better, although it also failed after
a few hours. There may have been usage spikes that maxed out the available bw
at the time, but I can't tell if that happened or not. So I reduced it to
bwlimit=32, and it's been running for 7 hours now without failure (hasn't
finished yet, but I'm optimistic).

So I'm inclined to think that this problem lies in the transport somewhere. It
also appears to come to light when the transport is saturated.

Does anyone have any ideas regarding how to go about tracking this down
further, or where to go for further help? While this bug is definitely
affecting rsync, I think the solution is going to end up being elsewhere, in
glibc or the kernel perhaps.

-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.