Rsync takes long time to finish

[Kevin Korb]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

That is a bit old for ext4.  You need 2.6.28 as the bare minimum but
there were a few early issues.  I don't remember exactly when it
stabilized but I think it was in the low 2.6.30s.  Your 2.6.18 is from
2006.  (Yes, I know, RedHat has been patching it for years.  Doesn't
mean they have done any performance improvements.)

On 04/12/12 17:36, vijay patel wrote:
> We are running Kernel 2.6.18-308.1.1.el5 which is latest in RHEL
> 5.8 on both the server. I think i might have to explore option of
> using ext4.
> 
>> Date: Thu, 12 Apr 2012 17:31:35 -0400 From: kmk at sanitarium.net 
>> To: rsync at lists.samba.org Subject: Re: Rsync takes long time to
>> finish
>> 
> There was also a serious performance regression in 2.6.39.
> 
> 
> On 04/12/12 17:29, Dan Stromberg wrote:
> 
>> I've heard lots of good suggestions already - another thing that 
>> I've not seen mentioned is, upgrading your kernel may help. 
>> Somewhere shortly before kernel 3.0, pathname lookups got 
>> noticeably faster.
> 
>> You could also try an alternative filesystem like xfs. It's 
>> supposed to be pretty good at large directories.
> 
>> On Thu, Apr 12, 2012 at 11:29 AM, vijay patel 
>> <catchvjay at hotmail.com <mailto:catchvjay at hotmail.com>> wrote:
> 
>> Hi Friends,
> 
>> I am using rsync to copy data from Production File Server to 
>> Disaster Recovery file server. I have 100Mbps link setup between 
>> these two servers. Folder structure is very deep. It is having 
>> path like /reports/folder1/date/folder2/file.tx, where we have 
>> 1600 directories like 'folder1', daily folders since last year
>> in date folder and 2 folders for each date folder like folder2
>> which ultimately will contain the file. Files are not too big but
>> just design of folder structure is complex. Folder structure
>> design is done by application and we can't change it at the
>> moment. I am using following command in cron to run rsync.
> 
>> rsync -avh --delete --exclude-from 'ex_file.txt' /reports/ 
>> 10.10.10.100:/reports/ | tee /tmp/rsync_report.out >> 
>> /tmp/rsync_report.out.$today
> 
>> Initially we were running it every 5 mins then we increased it
>> to every 30 mins since one instance was not getting finished in
>> 5 mins. Now we have made it to run every 8 hours because of lots
>> of folders. Is there a way i can improve performance of my
>> rsync??
> 
> 
>> Regards, Vijay
> 
> 
>> -- Please use reply-all for most replies to avoid omitting the 
>> mailing list. To unsubscribe or change options: 
>> https://lists.samba.org/mailman/listinfo/rsync Before posting, 
>> read: http://www.catb.org/~esr/faqs/smart-questions.html 
>> <http://www.catb.org/%7Eesr/faqs/smart-questions.html>
> 
> 
> 
> 
> 
>> -- Please use reply-all for most replies to avoid omitting the
>> mailing list. To unsubscribe or change options:
> https://lists.samba.org/mailman/listinfo/rsync
>> Before posting, read:
>> http://www.catb.org/~esr/faqs/smart-questions.html
> 
> 

- -- 
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~
	Kevin Korb			Phone:    (407) 252-6853
	Systems Administrator		Internet:
	FutureQuest, Inc.		Kevin at FutureQuest.net  (work)
	Orlando, Florida		kmk at sanitarium.net (personal)
	Web page:			http://www.sanitarium.net/
	PGP public key available on web site.
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.17 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk+HS7AACgkQVKC1jlbQAQd76ACgoXyQ/e+BjENecGaTIayGs+gl
kagAn18vI5dcDAveoB//K6TRQKMydL3s
=uGXf
-----END PGP SIGNATURE-----