Aw: Dealing with an unreliable remote
devzero at web.de
devzero at web.de
Tue Nov 25 11:36:39 MST 2014
you may have a look here:
if you use inplace or append, for security reason you could even run another rsync "diff" to compare if
source and destination is really identical, see:
> Gesendet: Dienstag, 25. November 2014 um 16:02 Uhr
> Von: net.rsync at io7m.com
> An: rsync at lists.samba.org
> Betreff: Dealing with an unreliable remote
> I've run into a frustrating issue when trying to synchronize a
> directory hierarchy over a reliable (but slow) connection to an
> unreliable remote. Basically, I have the following:
> This is a set of nested directories containing binaries and sources for
> projects I develop/maintain. Every time a new release is made, I deploy
> the binaries and sources to an exact copy of the above hierarchy on my
> local machine, and then rsync that (over SSH) to
> $ rsync -avz --delete --progress local/ io7m.com:/home/io7m/mvn-repository.io7m.com/
> The problem:
> The latest project produces one .jar file that's about 80mb.
> Evidently, the hosting provider I use for io7m.com is using some sort
> of process tracking system that kills processes that have been running
> for too long (I think it just measures CPU time). The result of this is
> that I get about 50% of the way through copying that
> (comparatively) large file, and then the remote rsync process is
> suddenly killed because it has been running for too long.
> This would be fine, except that it seems that rsync is utterly refusing
> all efforts to continue copying that file from wherever it left off. It
> always restarts copying of the file from nothing and tries to copy the
> full 80mb, resulting it being killed halfway through and causing much
> grinding of teeth.
> The documentation for --partial states that "Using the --partial option
> tells rsync to keep the partial file which should make a subsequent
> transfer of the rest of the file much faster.". Well, for whatever
> reason, it doesn't (or it at least fails to continue using it).
> I've tried --partial-dir, passing it an absolute path to the temporary
> directory in my home directory. It created a file in there the first time,
> but after being killed by the remote side and restarting, it ignored
> that file and instead created a new temporary file (with a random suffix)
> in the destination directory! Am I doing something wrong?
> $ rsync -avz --delete --progress --partial-dir=/home/io7m/tmp/rsync io7m.com:/home/io7m/mvn-repository.io7m.com/
> I'm at a loss. How can I reliably get this directory hierarchy up onto
> the server? I don't care if I have to retry the command multiple times
> until the copy has fully succeeded, but I obviously can't do that if
> rsync keeps restarting the failed file from scratch every time.
> Please use reply-all for most replies to avoid omitting the mailing list.
> To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
> Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
More information about the rsync