Rsync between 2 datacenters not working

[Marc Roos]

Hmmm, looks indeed like changing the mtu helps, strange thing is that 
with the bigger mtu sometimes I had no problems and often TB are copied 
without problems. 
Problem is I can't test the connection that much because the connection 
has been established by a third party and it is just some vlan without 
any hops. 

But thanks for the link to the nfs-test that looks interesting! 




-----Original Message-----
From: Dan Stromberg [mailto:drsalists at gmail.com] 
Sent: donderdag 29 maart 2018 4:36
To: Marc Roos
Cc: rsync
Subject: Re: Rsync between 2 datacenters not working

If reducing the MTU is helping, you might look into turning on Path MTU 
Discovery.

NFS can be fast for large transfers if you tune it.
http://stromberg.dnsalias.org/~strombrg/nfs-test.html

NFS is not terribly secure though - at least v2 and v3 weren't. Not sure 
about v4.

On Wed, Mar 28, 2018 at 12:59 AM, Marc Roos via rsync 
<rsync at lists.samba.org> wrote:
>
> Kevin, Dan, Thanks for the pointers to work arounds, at the moment I 
> am testing with lower mtu size that seems to be working. Otherwise I 
> need to fall back on mounting the fs maybe even as nfs.
>
>
>
> -----Original Message-----
> From: Kevin Korb via rsync [mailto:rsync at lists.samba.org]
> Sent: zondag 25 maart 2018 22:42
> To: rsync at lists.samba.org
> Subject: Re: Rsync between 2 datacenters not working
>
> Note that if you do this you are stuck with --whole-file
>
> On 03/25/2018 04:36 PM, Dan Stromberg via rsync wrote:
>> You could try using an automounter, like autofs, in combination with 
>> sshfs.  It'll be slower, possibly a lot slower, but it should be more 

>> reliable over an unreliable connection.
>>
>> I've been using:
>> remote
>> -fstype=fuse,allow_other,nodev,noatime,reconnect,ServerAliveInterval=
>> 1 5,ServerAliveCountMax=40,uid=0,gid=0,ro,nodev,noatime
>> :sshfs\#root at remote.host.com\:/
>>
>> BTW, I'm not sure it's necessary to escape the #.  I never tried it
> without.
>>
>> Also note that it flattens the remote host's mount tree into a single 

>> filesystem - so things like /proc look like they are in the same 
>> filesystem as /.  This can lead to backing up /proc's contents (many 
>> pseudofiles), if you don't exclude it, even if you use rsync's -x 
>> option.
>>
>> On Sun, Mar 25, 2018 at 6:43 AM, Marc Roos via rsync 
>> <rsync at lists.samba.org> wrote:
>>>
>>> I still stuck with these errors
>>>
>>> packet_write_wait: Connection to 192.168.10.43 port 22: Broken pipe
>>> rsync: connection unexpectedly closed (534132435 bytes received so
>>> far) [receiver] rsync error: error in rsync protocol data stream 
>>> (code 12) at io.c(605) [receiver=3.0.9]
>>> rsync: connection unexpectedly closed (27198 bytes received so far) 
>>> [generator] rsync error: unexplained error (code 255) at io.c(605) 
>>> [generator=3.0.9]
>>
>
> --
> 
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,
>         Kevin Korb                      Phone:    (407) 252-6853
>         Systems Administrator           Internet:
>         FutureQuest, Inc.               Kevin at FutureQuest.net  (work)
>         Orlando, Florida                kmk at sanitarium.net (personal)
>         Web page:                       http://www.sanitarium.net/
>         PGP public key available on web site.
> ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._
> .,
>
>
>
> --
> Please use reply-all for most replies to avoid omitting the mailing 
list.
> To unsubscribe or change options: 
> https://lists.samba.org/mailman/listinfo/rsync
> Before posting, read: 
> http://www.catb.org/~esr/faqs/smart-questions.html