[Bug 5478] rsync: writefd_unbuffered failed to write 4092 bytes [sender]: Broken pipe (32)

[samba-bugs at samba.org]

https://bugzilla.samba.org/show_bug.cgi?id=5478

--- Comment #11 from Eric Shubert <ejs at shubes.net> 2011-09-22 16:54:14 UTC ---
(In reply to comment #9)
> (In reply to comment #8)
> > By any chance do you consider these files "big" files?  I am asking
> > because I saw similar timeout when sending big files (bug7757,
> > bug7195).
> 
> Not really. Most problem files appear to be 20-30MB, but a few are <10M and
> there's one which is only 25K. I'll keep this in mind though.
> 

I need to change my answer here. The files that cause it to choke are larger
than most, so I would have to say that as a general observation the files that
fail are large(r). Not huge, but large.

I also need to correct my earlier statement/thinking. While the process did
fail on the same file on consecutive runs, leading me to believe that the
content of the file was causing the problem, I've determined that this is not
always the case. Subsequent runs have transferred these files successfully, so
it's not the content of the data that's a problem.

I've been making many (10s of) runs with strace on the target, and have noticed
a pattern, which I'll attempt to describe (not having looked at the source
code).

When transmitting these large files, the maximum size read at once is 4092
bytes. In most cases of failure, there are 7 of these 4092 byte chunks,
followed by 2 smaller chunks, which vary in size. In the sample below, they are
3699, then 393. Other value pairs I've seen are 3697 and 395, 2514 and 1578,
2758 and 1334, 1039 and 3053, 1085 and 3007, 4087 and 5, etc (all pair sums are
4092). These are followed by another 7 4092 byte chunks, then another 3699 and
393, and so on. When the process hangs, it always times out waiting for the
first of the two smaller chunks (after 7 reads of 4092 bytes).

On one failure, all reads were 4092 bytes, with no smaller chunks. In this
case, the failure occurred after 7 reads of 4092 bytes which followed a write.

I hope that this gives a clue as to where the problem might be. Please let me
know if there's something else I can provide. I'll continue to investigate.

Thanks.

-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.