Rsync takes long time to finish
kmk at sanitarium.net
Thu Apr 12 15:19:47 MDT 2012
-----BEGIN PGP SIGNED MESSAGE-----
Yes, you have the feature in your filesystem. Good. If it is ext3
then converting it to ext4 would still help assuming your distro
You are using rsync over ssh. This is my preference as well for
security reasons. Using rsyncd would be faster because it would
remove the encryption overhead but that shouldn't be a big deal on
only 100Mbits. It would make no difference in the indexing.
Have you checked your version yet? Run rsync --version on both
systems. If it isn't 3.0.something upgrade. That will make a big
On 04/12/12 17:16, vijay patel wrote:
> I am getting following thing in 'tune2fs -l' :
> Filesystem features: has_journal resize_inode dir_index
> filetype needs_recovery sparse_super large_file
> Does this mean it is set? One more thing i am not using rsync as
> daemon (Because i am confused with its usage at the moment), will
> it make any difference?
>> Date: Thu, 12 Apr 2012 17:05:06 -0400 From: kmk at sanitarium.net
>> To: rsync at lists.samba.org Subject: Re: Rsync takes long time to
> And make sure both systems are running rsync v3. It indexes in
> parallel to the copying.
> On 04/12/12 16:59, Karl O. Pinc wrote:
>> On 04/12/2012 03:28:18 PM, vijay patel wrote:
>>> Thanks friends. We are using Redhat Linux 5.8 on Production
>>> and Disaster Recovery side. By drilling down we have found out
>>> it is taking lot of time to check what has changed while data
>>> tranfer is very fast. As i mentioned data in these folders is
>>> very less (hardly 40GB) and whenever new file is created, it
>>> is of max 30KB.
>>> Since we have to sync production environment to DR every 10
>>> mins as per Business requirement i have to schedule it via
>>> cron. This already distributed folder structure i am using. I
>>> already have another rsync job which runs every 5 mins on
>>> another folder structure. It is running fine. Is there any
>>> option i can use with rsync to make this folder check fast?
>> No. Per the response below you need to look at your filesystems.
>> Use "tune2fs -l" and see if the dir_index option is on. If not,
>> then turn it on using tune2fs. This probably won't fix the
>> existing directories. If this is the problem you'll have to do a
>> backup/restore, or a move of all the files into a new directory
>> hierarchy and then replace the old hierarchy, or something else
>> to fix all the existing directories. (I don't think e2fsck will
>> help, but I've not looked. As I say, there may also be some
>> other approach.)
>>>> If it is mostly looking for something to transfer then you
>>>> need filesystem optimizations. Such as directory indexing.
>>>> You didn't specify the OS or anything but if you are on Linux
>>>> this is where an ext3 > ext4 conversion would be helpful.
>> Karl <kop at meme.com> Free Software: "You don't pay back, you pay
>> forward." -- Robert A. Heinlein
>> -- Please use reply-all for most replies to avoid omitting the
>> mailing list. To unsubscribe or change options:
>> Before posting, read:
Kevin Korb Phone: (407) 252-6853
Systems Administrator Internet:
FutureQuest, Inc. Kevin at FutureQuest.net (work)
Orlando, Florida kmk at sanitarium.net (personal)
Web page: http://www.sanitarium.net/
PGP public key available on web site.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.17 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
-----END PGP SIGNATURE-----
More information about the rsync