<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0//EN" "http://www.w3.org/TR/REC-html40/strict.dtd">
<html><head><meta name="qrichtext" content="1" /><style type="text/css">
p, li { white-space: pre-wrap; }
</style></head><body style=" font-family:'DejaVu Sans'; font-size:9pt; font-weight:400; font-style:normal;">
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">Ok, i'l try setting timeout, but according to documentation, default is timeout=0 which means no timeout.</p>
<p style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; "> </p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">Maybe setting --contimeout to some large value should help?</p>
<p style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; "> </p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">Libor</p>
<p style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; "> </p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><br />Dne Thursday 02 August 2012 07:03:51, Tony Abernethy napsal(a):<br /></p>
<p style=" margin-top:12px; margin-bottom:12px; margin-left:40px; margin-right:40px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Calibri,sans-serif'; font-size:11pt; color:#1f497d;"> --timeout=SECONDS set I/O timeout in seconds</span></p>
<p style=" margin-top:12px; margin-bottom:12px; margin-left:40px; margin-right:40px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Calibri,sans-serif'; font-size:11pt; color:#1f497d;">I think this timeout must be set big enough so that data actually gets transferred during the window.</span></p>
<p style=" margin-top:12px; margin-bottom:12px; margin-left:40px; margin-right:40px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Calibri,sans-serif'; font-size:11pt; color:#1f497d;">Having the target verify that nothing has changed yet seems to not qualify as resetting the timeout.</span></p>
<p style=" margin-top:12px; margin-bottom:12px; margin-left:40px; margin-right:40px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Calibri,sans-serif'; font-size:11pt; color:#1f497d;">Figure on enough time so that the target can read the entire target file.</span></p>
<p style=" margin-top:12px; margin-bottom:12px; margin-left:40px; margin-right:40px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Calibri,sans-serif'; font-size:11pt; color:#1f497d;">(Most likely there is a wee bit of change at the very end of the files)</span></p>
<p style=" margin-top:12px; margin-bottom:12px; margin-left:40px; margin-right:40px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Calibri,sans-serif'; font-size:11pt; color:#1f497d;"> </span></p>
<p style=" margin-top:12px; margin-bottom:12px; margin-left:40px; margin-right:40px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'Tahoma,sans-serif'; font-size:10pt; font-weight:600;">From:</span><span style=" font-family:'Tahoma,sans-serif'; font-size:10pt;"> rsync-bounces@lists.samba.org [mailto:rsync-bounces@lists.samba.org] </span><span style=" font-family:'Tahoma,sans-serif'; font-size:10pt; font-weight:600;">On Behalf Of </span><span style=" font-family:'Tahoma,sans-serif'; font-size:10pt;">Libor Klepác<br /></span><span style=" font-family:'Tahoma,sans-serif'; font-size:10pt; font-weight:600;">Sent:</span><span style=" font-family:'Tahoma,sans-serif'; font-size:10pt;"> Thursday, August 02, 2012 5:47 AM<br /></span><span style=" font-family:'Tahoma,sans-serif'; font-size:10pt; font-weight:600;">To:</span><span style=" font-family:'Tahoma,sans-serif'; font-size:10pt;"> rsync@lists.samba.org<br /></span><span style=" font-family:'Tahoma,sans-serif'; font-size:10pt; font-weight:600;">Subject:</span><span style=" font-family:'Tahoma,sans-serif'; font-size:10pt;"> Re: Rsyncing huge file, timeout on rsyncd</span></p>
<p style=" margin-top:12px; margin-bottom:12px; margin-left:40px; margin-right:40px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"> </p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:40px; margin-right:40px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'DejaVu Sans,serif';">Just to clarify</span></p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:40px; margin-right:40px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'DejaVu Sans,serif';">When using rsyncd script, there is also no progress from beginning, it just timeouts</span></p>
<p style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; "> </p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:40px; margin-right:40px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'DejaVu Sans,serif';">Libor</span></p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:40px; margin-right:40px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'DejaVu Sans,serif';"><br />Dne Thursday 02 August 2012 12:26:13, Libor Klepáč napsal(a):</span></p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:40px; margin-right:40px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'DejaVu Sans,serif';">> Hello,</span></p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:40px; margin-right:40px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'DejaVu Sans,serif';">> let me describe my setup.</span></p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:40px; margin-right:40px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'DejaVu Sans,serif';">> Source server is debian wheezy, rsync 3.0.9</span></p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:40px; margin-right:40px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'DejaVu Sans,serif';">> Destination is qnap TS-410, with rsyncd enabled, rsync 3.0.7</span></p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:40px; margin-right:40px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'DejaVu Sans,serif';">> </span></p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:40px; margin-right:40px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'DejaVu Sans,serif';">> I'm trying to rsync two files, which are exported from source server as</span></p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:40px; margin-right:40px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'DejaVu Sans,serif';">> iscsi targets (windows iscsi backups). Files are on btrfs, so I use</span></p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:40px; margin-right:40px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'DejaVu Sans,serif';">> snapshots during rsync, so files don't change.</span></p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:40px; margin-right:40px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'DejaVu Sans,serif';">> One file is 50GB, second is 550GB in size.</span></p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:40px; margin-right:40px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'DejaVu Sans,serif';">> </span></p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:40px; margin-right:40px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'DejaVu Sans,serif';">> Initial sync over rsyncd daemon went fine.</span></p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:40px; margin-right:40px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'DejaVu Sans,serif';">> Now, i'm trying to sync data, after change.</span></p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:40px; margin-right:40px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'DejaVu Sans,serif';">> </span></p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:40px; margin-right:40px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'DejaVu Sans,serif';">> When i try to sync over rsyncd, it ends after 20 minutes with:</span></p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:40px; margin-right:40px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'DejaVu Sans,serif';">> </span></p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:40px; margin-right:40px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'DejaVu Sans,serif';">> rsync: connection unexpectedly closed (2438836 bytes received so far)</span></p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:40px; margin-right:40px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'DejaVu Sans,serif';">> [sender] rsync error: error in rsync protocol data stream (code 12) at</span></p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:40px; margin-right:40px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'DejaVu Sans,serif';">> io.c(605) [sender=3.0.9]</span></p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:40px; margin-right:40px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'DejaVu Sans,serif';">> </span></p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:40px; margin-right:40px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'DejaVu Sans,serif';">> </span></p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:40px; margin-right:40px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'DejaVu Sans,serif';">> When i try to rsync over ssh, it takes ages to show progress status (around</span></p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:40px; margin-right:40px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'DejaVu Sans,serif';">> 1 hour for smaller file, 4-5 hours for big file).</span></p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:40px; margin-right:40px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'DejaVu Sans,serif';">> Whats going out there?</span></p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:40px; margin-right:40px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'DejaVu Sans,serif';">> Source server is idle during this "no progress" period, destination Qnap has</span></p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:40px; margin-right:40px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'DejaVu Sans,serif';">> one rsync process eating cpu.</span></p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:40px; margin-right:40px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'DejaVu Sans,serif';">> </span></p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:40px; margin-right:40px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'DejaVu Sans,serif';">> My commands are (example with one file)</span></p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:40px; margin-right:40px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'DejaVu Sans,serif';">> for rsyncd</span></p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:40px; margin-right:40px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'DejaVu Sans,serif';">> rsync --inplace --progress --numeric-ids --times --password-file .passwd</span></p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:40px; margin-right:40px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'DejaVu Sans,serif';">> iscsi/iscsiDisk1 rsync://backup-rsync@192.168.5.250:/iSCSI-backup/iscsiDisk1</span></p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:40px; margin-right:40px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'DejaVu Sans,serif';">> </span></p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:40px; margin-right:40px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'DejaVu Sans,serif';">> for ssh</span></p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:40px; margin-right:40px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'DejaVu Sans,serif';">> rsync --inplace --progress --numeric-ids --times iscsi/iscsiDisk1</span></p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:40px; margin-right:40px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'DejaVu Sans,serif';">> </span><a href="mailto:admin@192.168.5.250:/share/MD0_DATA/iSCSI-backup/iscsiDisk1"><span style=" font-family:'DejaVu Sans,serif'; text-decoration: underline; color:#508ed8;">admin@192.168.5.250:/share/MD0_DATA/iSCSI-backup/iscsiDisk1</span></a></p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:40px; margin-right:40px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'DejaVu Sans,serif';">> </span></p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:40px; margin-right:40px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'DejaVu Sans,serif';">> With regards</span></p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:40px; margin-right:40px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><span style=" font-family:'DejaVu Sans,serif';">> Libor Klepac</span></p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;"><br /><br /></p></body></html>