Rsync takes long time to finish

[Kevin Korb]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

And make sure both systems are running rsync v3.  It indexes in
parallel to the copying.

On 04/12/12 16:59, Karl O. Pinc wrote:
> On 04/12/2012 03:28:18 PM, vijay patel wrote:
>> 
>> Thanks friends.  We are using Redhat Linux 5.8 on Production and 
>> Disaster Recovery side.  By drilling down we have found out it
>> is taking lot of time to check what has changed while data
>> tranfer is very fast.  As i mentioned data in these folders is
>> very less (hardly 40GB) and whenever new file is created, it is
>> of max 30KB.
>> 
>> Since we have to sync production environment to DR every 10 mins
>> as per Business requirement i have to schedule it via cron. This
>> already distributed folder structure i am using. I already have
>> another rsync job which runs every 5 mins on another folder
>> structure. It is running fine. Is there any option i can use with
>> rsync to make this folder check fast?
> 
> No.  Per the response below you need to look at your filesystems.
> 
> Use "tune2fs -l" and see if the dir_index option is on.  If not, 
> then turn it on using tune2fs.  This probably won't fix the
> existing directories.  If this is the problem you'll have to do a 
> backup/restore, or a move of all the files into a new directory 
> hierarchy and then replace the old hierarchy, or something else to
> fix all the existing directories. (I don't think e2fsck will help,
> but I've not looked.  As I say, there may also be some other
> approach.)
> 
>>> If it is mostly looking for something to transfer then you
>>> need filesystem optimizations. Such as directory indexing. You
>>> didn't specify the OS or anything but if you are on Linux this
>>> is where an ext3 > ext4 conversion would be helpful.
> 
> 
> Karl <kop at meme.com> Free Software:  "You don't pay back, you pay
> forward." -- Robert A. Heinlein
> 

- -- 
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~
	Kevin Korb			Phone:    (407) 252-6853
	Systems Administrator		Internet:
	FutureQuest, Inc.		Kevin at FutureQuest.net  (work)
	Orlando, Florida		kmk at sanitarium.net (personal)
	Web page:			http://www.sanitarium.net/
	PGP public key available on web site.
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.17 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk+HQ4IACgkQVKC1jlbQAQeUTQCgrA7MIbX73hVZO3YsLxHsaUlN
O9IAnipAWOvrU4mdXuWNHP0/Wc6hmI2H
=CIUJ
-----END PGP SIGNATURE-----