Intermittent rsync Issues

Thu Feb 17 03:04:01 MST 2011

Hello Mr. Gray,

What was the resulting configuration change (from default) that you had made
to fix this? I have reported the same issue today, and this configuration
change might be what I'm looking for.

My report: http://www.mail-archive.com/rsync@lists.samba.org/msg26787.html

Cheers,
Shai

On Tue, Sep 21, 2010 at 5:10 PM, J. T. Gray <jtg at intarcorp.com> wrote:

> Izidor, this appears to resolve my problem.  Thank you for your input, I
> will be sure to attribute you on serverfault.com when I add an answer to
> my question post.
>
>
>
> On Tue, Sep 21, 2010 at 2:49 AM, Izidor Jerebic <ij.list at gmail.com> wrote:
>
>> Hi,
>>
>> this might be a problem with maximal number of concurrent ssh connections
>> or connection requests. Ssh daemon has two configuration settings where you
>> can define what is the maximal number of clients which can connect
>> concurrently. This number is by default not very high, so you are probably
>> bumping against that limit.
>>
>>     MaxSessions
>>
>>             Specifies the maximum number of open sessions permitted per
>> net-
>>             work connection.  The default is 10.
>>
>>
>> MaxStartups
>>
>>             Specifies the maximum number of concurrent unauthenticated
>> con-
>>             nections to the SSH daemon.  Additional connections will be
>>             dropped until authentication succeeds or the LoginGraceTime
>>             expires for a connection.  The default is 10.
>>             Alternatively, random early drop can be enabled by specifying
>> the
>>             three colon separated values ``start:rate:full'' (e.g.
>>             "10:30:60").  sshd(8) will refuse connection attempts with a
>>             probability of ``rate/100'' (30%) if there are currently
>>             ``start'' (10) unauthenticated connections.  The probability
>>             increases linearly and all connection attempts are refused if
>> the
>>             number of unauthenticated connections reaches ``full'' (60).
>>
>>
>> izidor
>>
>> On 20.9.2010, at 19:19, J. T. Gray wrote:
>>
>> > Greetings,
>> >
>> > I posted this at serverfault, but in hopes that the people on this list
>> are different from the people there, here it is:
>> >
>> > Update: 9/20/10: Updated the EC2 AMI on both the client and the server
>> and ran a 3-box test with 2 clients downloading from 1 server over 24 hours.
>> Upon test completion, the logs had zero errors so I began replacing other
>> instances with the updated AMI instances. After a weekend of running the
>> 35-40ish clients, I have logs once again filled with:
>> >
>> > 2010/09/20 16:27:01 [18581] rsync error: error in rsync protocol data
>> stream (code 12) at io.c(601) [Receiver=3.0.7] 2010/09/20 16:30:01 [18627]
>> rsync error: unexplained error (code 255) at io.c(601)
>> > Is it unreasonable to have 35-40 clients connect to an rsync server
>> simultaneously? Is this possibly a load issue?
>> >
>> >
>> >
>> >
>> >
>> > On Thu, Sep 16, 2010 at 11:04 AM, J. T. Gray <jtg at intarcorp.com> wrote:
>> > Added --bwlimit=150 at 10:58, had a success at 10:59, failure at 11:00,
>> and success at 11:01.
>> >
>> > It takes about 1.7s to execute the rsync command, for what that's worth.
>> >
>> > That it seems to fail immediately suggests it's something
>> connection-related, but that it fails on rsh and ssh both suggests it's
>> something specific to the client app.  I'm currently updating my EC2 image
>> to see if it's specific to the build of Ubuntu I'm using.
>> >
>> >
>> >
>> >
>> > On Thu, Sep 16, 2010 at 10:54 AM, Mac User FR <macuserfr at free.fr>
>> wrote:
>> > Already tried --bwlimit. In my case it didn't change the error. It just
>> slowed down the identical file recognition. But when it came to a file
>> modified from what exist on the backup server, it dropped the connection.
>> >
>> > Thanks anyway for the help!
>> >
>> > Vitorio
>> >
>> > Le 16 sept. 10 à 15:56, Paul Slootman a écrit :
>> >
>> >
>> > On Thu 16 Sep 2010, Mac User FR wrote:
>> >
>> > I read somewhere that the ssh connection was stronger than rsync one
>> > in unstable networks. And effectively, if I run rsync via ssh ( -e
>> > option), the transfer happen without errors, on the same computer
>> > using the same networking setup.
>> >
>> > I wouldn't be surprised that the extra overhead related to ssh slows the
>> > transfer sufficiently to make a flakey network connection not give up.
>> > Perhaps try rate-limiting the rsync transfer.
>> >
>> >
>> > Paul
>> > --
>> > Please use reply-all for most replies to avoid omitting the mailing
>> list.
>> > To unsubscribe or change options:
>> https://lists.samba.org/mailman/listinfo/rsync
>> > Before posting, read:
>> http://www.catb.org/~esr/faqs/smart-questions.html
>> >
>> > --
>> > Please use reply-all for most replies to avoid omitting the mailing
>> list.
>> > To unsubscribe or change options:
>> https://lists.samba.org/mailman/listinfo/rsync
>> > Before posting, read:
>> http://www.catb.org/~esr/faqs/smart-questions.html
>> >
>> >
>> > --
>> > Please use reply-all for most replies to avoid omitting the mailing
>> list.
>> > To unsubscribe or change options:
>> https://lists.samba.org/mailman/listinfo/rsync
>> > Before posting, read:
>> http://www.catb.org/~esr/faqs/smart-questions.html
>>
>>
>
> --
> Please use reply-all for most replies to avoid omitting the mailing list.
> To unsubscribe or change options:
> https://lists.samba.org/mailman/listinfo/rsync
> Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.samba.org/pipermail/rsync/attachments/20110217/554dcf89/attachment.html>