What is it doing?

[Kevin Korb]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

I agree completely about getting rsync onto the NFS server.  Rsync
(especially over ssh) is far more secure than NFS.  Even if the NFS
server is actually a NAS appliance many of them can do rsync as well.

On 01/13/2014 11:14 PM, Linda A. Walsh wrote:
> Perry Smith wrote:
>> This is my first time to really use rsync.  I did small tests to
>> get the arguments like I wanted and then kicked off the big rsync
>> about 2 and a half hours ago.  So far, it has not copied over any
>> files.
>> 
>> -----
>> 
>> Is it really making progress?  Or will it take this long to
>> really start copying files over each day I start it?
>> 
>> I expect the total amount copied to be about 400G and about 4
>> million files
> ----- This appears to be a classic case of using a hammer to drive
> in a screw.
> 
> Um... rsync was designed to save network bandwidth by running on
> the host and doing file-stat intensive stuff ON the local hard disk
> (by running on the server and on your client).
> 
> But your usage case does very badly because rsync needs direct
> access on both end --- THEN it optimizes the stuff transferred to
> minimize the amount needing to be copied over the network....   But
> you are not getting ANY benefit because it will do all of those 
> stats over the network via NFS which is notoriously slow in many or
> most cases (especially with lots of stat calls). Your copy job
> would already be done if you did it with 'tar' and just copied over
> everything. On the receiving end tell tar not to overwrite newer
> stuff.   Yes it will waste more network bandwidth, but it would
> very likely, already be done.
> 
> As you have described the problem, there is no real reason to use
> rsync, as it is unable to optimize network bandwidth because all
> the stats are remote.
> 
> Even "cp -au src/. dst/." will likely be faster than trying to use 
> rsync....   talk about tool abuse!  ;-) For rsync to do a
> reasonable job, you really need to tell whoever owns that server to
> put rsync ON that server so it can access the files locally, then
> it could do what it does best and build up a list of differences so
> it only needs to transfer the changed stuff.
> 
> Certainly, even if you have rsync on the remote end -- for the 1st 
> transfer, if you need to transfer most of the files, it would be
> better just to create a tar on the remote end, compress it, and
> copy that locally. How is it that you have so much data on a server
> you don't have any ability to run a local 'job' on?   It really
> sounds like an impediment to you getting your work done.
> 
> 

- -- 
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~
	Kevin Korb			Phone:    (407) 252-6853
	Systems Administrator		Internet:
	FutureQuest, Inc.		Kevin at FutureQuest.net  (work)
	Orlando, Florida		kmk at sanitarium.net (personal)
	Web page:			http://www.sanitarium.net/
	PGP public key available on web site.
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iEYEARECAAYFAlLUuo0ACgkQVKC1jlbQAQc5AACfTPtDKA1m2T0K1NNkKFZs12KA
Mi8AnR+f1I/MOxOZ2lg1QoMvD+/ziwC0
=9ETo
-----END PGP SIGNATURE-----