Intermittent rsync Issues
Izidor Jerebic
ij.list at gmail.com
Tue Sep 21 00:49:17 MDT 2010
Hi,
this might be a problem with maximal number of concurrent ssh connections or connection requests. Ssh daemon has two configuration settings where you can define what is the maximal number of clients which can connect concurrently. This number is by default not very high, so you are probably bumping against that limit.
MaxSessions
Specifies the maximum number of open sessions permitted per net-
work connection. The default is 10.
MaxStartups
Specifies the maximum number of concurrent unauthenticated con-
nections to the SSH daemon. Additional connections will be
dropped until authentication succeeds or the LoginGraceTime
expires for a connection. The default is 10.
Alternatively, random early drop can be enabled by specifying the
three colon separated values ``start:rate:full'' (e.g.
"10:30:60"). sshd(8) will refuse connection attempts with a
probability of ``rate/100'' (30%) if there are currently
``start'' (10) unauthenticated connections. The probability
increases linearly and all connection attempts are refused if the
number of unauthenticated connections reaches ``full'' (60).
izidor
On 20.9.2010, at 19:19, J. T. Gray wrote:
> Greetings,
>
> I posted this at serverfault, but in hopes that the people on this list are different from the people there, here it is:
>
> Update: 9/20/10: Updated the EC2 AMI on both the client and the server and ran a 3-box test with 2 clients downloading from 1 server over 24 hours. Upon test completion, the logs had zero errors so I began replacing other instances with the updated AMI instances. After a weekend of running the 35-40ish clients, I have logs once again filled with:
>
> 2010/09/20 16:27:01 [18581] rsync error: error in rsync protocol data stream (code 12) at io.c(601) [Receiver=3.0.7] 2010/09/20 16:30:01 [18627] rsync error: unexplained error (code 255) at io.c(601)
> Is it unreasonable to have 35-40 clients connect to an rsync server simultaneously? Is this possibly a load issue?
>
>
>
>
>
> On Thu, Sep 16, 2010 at 11:04 AM, J. T. Gray <jtg at intarcorp.com> wrote:
> Added --bwlimit=150 at 10:58, had a success at 10:59, failure at 11:00, and success at 11:01.
>
> It takes about 1.7s to execute the rsync command, for what that's worth.
>
> That it seems to fail immediately suggests it's something connection-related, but that it fails on rsh and ssh both suggests it's something specific to the client app. I'm currently updating my EC2 image to see if it's specific to the build of Ubuntu I'm using.
>
>
>
>
> On Thu, Sep 16, 2010 at 10:54 AM, Mac User FR <macuserfr at free.fr> wrote:
> Already tried --bwlimit. In my case it didn't change the error. It just slowed down the identical file recognition. But when it came to a file modified from what exist on the backup server, it dropped the connection.
>
> Thanks anyway for the help!
>
> Vitorio
>
> Le 16 sept. 10 à 15:56, Paul Slootman a écrit :
>
>
> On Thu 16 Sep 2010, Mac User FR wrote:
>
> I read somewhere that the ssh connection was stronger than rsync one
> in unstable networks. And effectively, if I run rsync via ssh ( -e
> option), the transfer happen without errors, on the same computer
> using the same networking setup.
>
> I wouldn't be surprised that the extra overhead related to ssh slows the
> transfer sufficiently to make a flakey network connection not give up.
> Perhaps try rate-limiting the rsync transfer.
>
>
> Paul
> --
> Please use reply-all for most replies to avoid omitting the mailing list.
> To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
> Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
>
> --
> Please use reply-all for most replies to avoid omitting the mailing list.
> To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
> Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
>
>
> --
> Please use reply-all for most replies to avoid omitting the mailing list.
> To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
> Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
More information about the rsync
mailing list