Intermittent rsync Issues

Tue Sep 21 09:10:34 MDT 2010

Izidor, this appears to resolve my problem.  Thank you for your input, I
will be sure to attribute you on serverfault.com when I add an answer to my
question post.

On Tue, Sep 21, 2010 at 2:49 AM, Izidor Jerebic <ij.list at gmail.com> wrote:

> Hi,
>
> this might be a problem with maximal number of concurrent ssh connections
> or connection requests. Ssh daemon has two configuration settings where you
> can define what is the maximal number of clients which can connect
> concurrently. This number is by default not very high, so you are probably
> bumping against that limit.
>
>     MaxSessions
>
>             Specifies the maximum number of open sessions permitted per
> net-
>             work connection.  The default is 10.
>
>
> MaxStartups
>
>             Specifies the maximum number of concurrent unauthenticated con-
>             nections to the SSH daemon.  Additional connections will be
>             dropped until authentication succeeds or the LoginGraceTime
>             expires for a connection.  The default is 10.
>             Alternatively, random early drop can be enabled by specifying
> the
>             three colon separated values ``start:rate:full'' (e.g.
>             "10:30:60").  sshd(8) will refuse connection attempts with a
>             probability of ``rate/100'' (30%) if there are currently
>             ``start'' (10) unauthenticated connections.  The probability
>             increases linearly and all connection attempts are refused if
> the
>             number of unauthenticated connections reaches ``full'' (60).
>
>
> izidor
>
> On 20.9.2010, at 19:19, J. T. Gray wrote:
>
> > Greetings,
> >
> > I posted this at serverfault, but in hopes that the people on this list
> are different from the people there, here it is:
> >
> > Update: 9/20/10: Updated the EC2 AMI on both the client and the server
> and ran a 3-box test with 2 clients downloading from 1 server over 24 hours.
> Upon test completion, the logs had zero errors so I began replacing other
> instances with the updated AMI instances. After a weekend of running the
> 35-40ish clients, I have logs once again filled with:
> >
> > 2010/09/20 16:27:01 [18581] rsync error: error in rsync protocol data
> stream (code 12) at io.c(601) [Receiver=3.0.7] 2010/09/20 16:30:01 [18627]
> rsync error: unexplained error (code 255) at io.c(601)
> > Is it unreasonable to have 35-40 clients connect to an rsync server
> simultaneously? Is this possibly a load issue?
> >
> >
> >
> >
> >
> > On Thu, Sep 16, 2010 at 11:04 AM, J. T. Gray <jtg at intarcorp.com> wrote:
> > Added --bwlimit=150 at 10:58, had a success at 10:59, failure at 11:00,
> and success at 11:01.
> >
> > It takes about 1.7s to execute the rsync command, for what that's worth.
> >
> > That it seems to fail immediately suggests it's something
> connection-related, but that it fails on rsh and ssh both suggests it's
> something specific to the client app.  I'm currently updating my EC2 image
> to see if it's specific to the build of Ubuntu I'm using.
> >
> >
> >
> >
> > On Thu, Sep 16, 2010 at 10:54 AM, Mac User FR <macuserfr at free.fr> wrote:
> > Already tried --bwlimit. In my case it didn't change the error. It just
> slowed down the identical file recognition. But when it came to a file
> modified from what exist on the backup server, it dropped the connection.
> >
> > Thanks anyway for the help!
> >
> > Vitorio
> >
> > Le 16 sept. 10 à 15:56, Paul Slootman a écrit :
> >
> >
> > On Thu 16 Sep 2010, Mac User FR wrote:
> >
> > I read somewhere that the ssh connection was stronger than rsync one
> > in unstable networks. And effectively, if I run rsync via ssh ( -e
> > option), the transfer happen without errors, on the same computer
> > using the same networking setup.
> >
> > I wouldn't be surprised that the extra overhead related to ssh slows the
> > transfer sufficiently to make a flakey network connection not give up.
> > Perhaps try rate-limiting the rsync transfer.
> >
> >
> > Paul
> > --
> > Please use reply-all for most replies to avoid omitting the mailing list.
> > To unsubscribe or change options:
> https://lists.samba.org/mailman/listinfo/rsync
> > Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html<http://www.catb.org/%7Eesr/faqs/smart-questions.html>
> >
> > --
> > Please use reply-all for most replies to avoid omitting the mailing list.
> > To unsubscribe or change options:
> https://lists.samba.org/mailman/listinfo/rsync
> > Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html<http://www.catb.org/%7Eesr/faqs/smart-questions.html>
> >
> >
> > --
> > Please use reply-all for most replies to avoid omitting the mailing list.
> > To unsubscribe or change options:
> https://lists.samba.org/mailman/listinfo/rsync
> > Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html<http://www.catb.org/%7Eesr/faqs/smart-questions.html>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.samba.org/pipermail/rsync/attachments/20100921/3d495c26/attachment.html>