(--delay-updates and --partial) re-hashing the already downloaded files?

Kevin Korb kmk at sanitarium.net
Wed Apr 17 14:43:38 MDT 2013


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Are you also using --partial-dir ?

I discussed this problem with someone recently in #rsync and IIRC the
solution was to not use --partial-dir

On 04/17/13 16:40, John Pierman wrote:
> I am backing up a LARGE data set, over a very unstable internet
> connection.
> 
> I NEED to --delay-updates, because I do a flash cut-over once
> everything has transferred. Yes I know --copy-dest does this, but
> if the connection breaks, anything that's already made it over gets
> put into place (not what I want), I need it to go from beginning to
> end, And then I handle the files seperately.
> 
> If I use --partial and --delay-updates together (I get the desired 
> effect, and all is well)......EXCEPT for.....
> 
> Let's say I have 10k files I start my rsync and get through 2k
> files and the internet connection is lost.
> 
> Everything is fine, my partial files are safe, nothing has been
> moved into place yet, I'm happy.
> 
> So I restart rsync (well a script does once the internet comes
> back). Here is where I get unwanted behavior. This rsync run LOOKS
> like it is starting from the beginning (transferring those first 2k
> files INSANELY fast if you look at --progress), which I realize
> it's not really TRANSFERRING anything but it's CHECKING all my
> partial files that haven't been put into place yet.
> 
> The problem is, my connection is dog slow and unreliable (I can't
> change this). If the connection drops frequently I'm basically
> spending too much time (re-hashing?) those files I already have,
> and not picking up where I left off.
> 
> I'm sure there's something I can do to get the desired behavior of 
> picking up where I left off without rehashing the first 2k files??
> 
> Help?
> 
> 

- -- 
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~
	Kevin Korb			Phone:    (407) 252-6853
	Systems Administrator		Internet:
	FutureQuest, Inc.		Kevin at FutureQuest.net  (work)
	Orlando, Florida		kmk at sanitarium.net (personal)
	Web page:			http://www.sanitarium.net/
	PGP public key available on web site.
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iEYEARECAAYFAlFvCXoACgkQVKC1jlbQAQckZgCeKeYPZ74xl1VD5LzwRxbJUoki
GawAnR/n3vczCLNk2+WO/HWWvZzzgdip
=9VH8
-----END PGP SIGNATURE-----


More information about the rsync mailing list