rsync hang on large file in a set

Eliot Moss moss at cs.umass.edu
Wed Dec 16 07:34:22 MST 2009


This is to both the rsync and cygwin lists, and I
regret that it is vaguer than I would like, but ...

With cygwin 1.7 (latest beta) and the rsync released
with it (3.0.6-1) I try to sync a couple of Gb of
Thunderbird mail files. The flag settings were
-avzuP plus a --chmod and some includes/excludes
to control which files to move.

I would repeatably get "hangs" on particular files,
which are generally modified by appending (mail
added at the end). These files (two in particular)
are approximately 100Mb or a little more in size,
and that seemed to have something to do with it.

The "hangs" I speak of are where rsync and ssh spin,
consuming cpu time (fan running fast!) but making
no progress.

Now here's the odd thing. If I kill of the rsync and
ssh processes and rsync the one file by itself, it
will go through. (Of course the --partial implied
by -P means that I may have to transfer a lot of
data :-( ).

Curiously, I got the same behavior with a recent rsync
head (20091129).

Last night I made two changes and have not seen a hang
since (though I am monitoring this, of course). One
was to substitute --progress and --partial-dir for
-P.

The other -- which is the cygwin relevant part -- was
the compile with socketpair turned off (manually in the
config file). I did this because Corinna and I had found
issues between socketpair and BLODA on my (Windows 7) box,
which she appeared to have fixed in revisions to 1.7 The
problems had to do with fork, dup, and such, not with
actual data transfer AFAIK. At her request I still have
the BLODA installed, and other cygwin things generally
work right.

I will also play with putting socketpair back in and
keeping --partial-dir. But the various reports of rsync
hangs, and a particular report a little while ago of
busy looping in buffer transfer (and I do have -z on,
might that impact?) struck me as possibly related. I
am not sure how to debug this when it happens, so if
there are steps I can take upon such a hang to get you
better info, do enlighten me!

Best wishes -- Eliot Moss
==============================================================================
J. Eliot B. Moss, Professor               http://www.cs.umass.edu/~moss    www
Director, Arch. and Lang. Impl. Lab.      +1-413-545-4206                voice
Department of Computer Science            +1-413-695-4226                 cell
140 Governor's Drive, Room 372            +1-413-545-1249                  fax
University of Massachusetts at Amherst    moss at cs.umass.edu              email
Amherst, MA  01003-9264  USA              +1-413-545-2746 Laurie Downey  sec'y
==============================================================================


More information about the rsync mailing list