Feature request, or HowTo? State-full resume rsync transfer

[Matthias Schniedermeyer]

On 15.07.2011 13:10, Donald Pearson wrote:
> 
> Matthias,
> 
> A vpn tunnel is an interesting idea.  Do you know how long you're able to
> keep rsync in limbo before it will give up?

I haven't really tried. But it was about 15 Minutes the one time it 
didn't reconnect in time.

> The issue I think is keeping the sockets open, thereby keeping the processes
> active.

The problem is that TCP really isn't made for transfer over an 
unreliable medium.

Besides rsync, the OSes on both sides also have to keep the connction 
running, but i really don't know which knobs have to be turned to extent 
the time after which the OS times out the connect.

> My first guess was the tcp_fin_timeout setting of the Ubuntu operating
> system, but this is set for 60 seconds.  Leaving everything alone I see
> roughly 6 minutes before the rsync client errors out.

tcp_fin is, AFAIK, after you close the socket normaly.

> Thanks again for everybody's help and by all means keep the ideas coming.  I
> am trying new angles as I come up with them, I haven't given up on this yet.

I got another idea.

If you have the storage on the source-side, to keep the state of the 
target-side. You can create batch-files (--write-batch & Co.). Of couse 
you also need the storage on the target-side for the batch-files and the 
need space for eventuell growth and temporary files.

You can then copy these batch-files with --append, make sure the 
target-rsync is dead before you retry, for an added bonus i would MD5 
the file on the source side and veryify it after copying. After you have 
transferred a complete batch-file, you can apply it. (Use 'screen' if 
you do it remotely)





Bis denn

-- 
Real Programmers consider "what you see is what you get" to be just as 
bad a concept in Text Editors as it is in women. No, the Real Programmer
wants a "you asked for it, you got it" text editor -- complicated, 
cryptic, powerful, unforgiving, dangerous.