Rsync takes long time to finish
kmk at sanitarium.net
Thu Apr 12 15:05:06 MDT 2012
-----BEGIN PGP SIGNED MESSAGE-----
And make sure both systems are running rsync v3. It indexes in
parallel to the copying.
On 04/12/12 16:59, Karl O. Pinc wrote:
> On 04/12/2012 03:28:18 PM, vijay patel wrote:
>> Thanks friends. We are using Redhat Linux 5.8 on Production and
>> Disaster Recovery side. By drilling down we have found out it
>> is taking lot of time to check what has changed while data
>> tranfer is very fast. As i mentioned data in these folders is
>> very less (hardly 40GB) and whenever new file is created, it is
>> of max 30KB.
>> Since we have to sync production environment to DR every 10 mins
>> as per Business requirement i have to schedule it via cron. This
>> already distributed folder structure i am using. I already have
>> another rsync job which runs every 5 mins on another folder
>> structure. It is running fine. Is there any option i can use with
>> rsync to make this folder check fast?
> No. Per the response below you need to look at your filesystems.
> Use "tune2fs -l" and see if the dir_index option is on. If not,
> then turn it on using tune2fs. This probably won't fix the
> existing directories. If this is the problem you'll have to do a
> backup/restore, or a move of all the files into a new directory
> hierarchy and then replace the old hierarchy, or something else to
> fix all the existing directories. (I don't think e2fsck will help,
> but I've not looked. As I say, there may also be some other
>>> If it is mostly looking for something to transfer then you
>>> need filesystem optimizations. Such as directory indexing. You
>>> didn't specify the OS or anything but if you are on Linux this
>>> is where an ext3 > ext4 conversion would be helpful.
> Karl <kop at meme.com> Free Software: "You don't pay back, you pay
> forward." -- Robert A. Heinlein
Kevin Korb Phone: (407) 252-6853
Systems Administrator Internet:
FutureQuest, Inc. Kevin at FutureQuest.net (work)
Orlando, Florida kmk at sanitarium.net (personal)
Web page: http://www.sanitarium.net/
PGP public key available on web site.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.17 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
-----END PGP SIGNATURE-----
More information about the rsync