Rsync connection times out on very large --files-from

Wayne Davison wayned at samba.org
Fri Mar 11 16:35:30 GMT 2005


On Tue, Mar 08, 2005 at 12:35:07PM -0500, Eli wrote:
> Anyways, the file specified in --files-from is about 500mb in size and
> contains several million files.  It takes rsync a few hours for it to
> parse through all the files in the list.  When it comes time for rsync
> to start transmitting file data to the rsync server, it gets an error
> that the connection timed out and it couldn't transmit any data.

Since rsync is keeping the socket busy during the building of the list
(it is trasmitting it during that time), I guess the sorting/cleaning of
the list is taking enough time that the socket timeouted out on you.
I'm glad that the setting of SO_KEEPALIVE did something to help fix your
problem, but I'm having a hard time seeing why:  the code should set
that for all daemon socket connections (excluding only daemon-over-ssh).
Are you using the --timeout=N option to rsync (which is an internal-to-
rsync timeout value, unrelated to the TCP timeout).

..wayne..


More information about the rsync mailing list