Rsync takes long time to finish

[vijay patel]

We are running Kernel 2.6.18-308.1.1.el5 which is latest in RHEL 5.8 on both the server. I think i might have to explore option of using ext4.
 

> Date: Thu, 12 Apr 2012 17:31:35 -0400
> From: kmk at sanitarium.net
> To: rsync at lists.samba.org
> Subject: Re: Rsync takes long time to finish
> 
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> There was also a serious performance regression in 2.6.39.
> 
> 
> On 04/12/12 17:29, Dan Stromberg wrote:
> > 
> > I've heard lots of good suggestions already - another thing that
> > I've not seen mentioned is, upgrading your kernel may help.
> > Somewhere shortly before kernel 3.0, pathname lookups got
> > noticeably faster.
> > 
> > You could also try an alternative filesystem like xfs. It's
> > supposed to be pretty good at large directories.
> > 
> > On Thu, Apr 12, 2012 at 11:29 AM, vijay patel
> > <catchvjay at hotmail.com <mailto:catchvjay at hotmail.com>> wrote:
> > 
> > Hi Friends,
> > 
> > I am using rsync to copy data from Production File Server to 
> > Disaster Recovery file server. I have 100Mbps link setup between 
> > these two servers. Folder structure is very deep. It is having
> > path like /reports/folder1/date/folder2/file.tx, where we have
> > 1600 directories like 'folder1', daily folders since last year in
> > date folder and 2 folders for each date folder like folder2 which 
> > ultimately will contain the file. Files are not too big but just 
> > design of folder structure is complex. Folder structure design is 
> > done by application and we can't change it at the moment. I am
> > using following command in cron to run rsync.
> > 
> > rsync -avh --delete --exclude-from 'ex_file.txt' /reports/ 
> > 10.10.10.100:/reports/ | tee /tmp/rsync_report.out >> 
> > /tmp/rsync_report.out.$today
> > 
> > Initially we were running it every 5 mins then we increased it to 
> > every 30 mins since one instance was not getting finished in 5
> > mins. Now we have made it to run every 8 hours because of lots of
> > folders. Is there a way i can improve performance of my rsync??
> > 
> > 
> > Regards, Vijay
> > 
> > 
> > -- Please use reply-all for most replies to avoid omitting the
> > mailing list. To unsubscribe or change options: 
> > https://lists.samba.org/mailman/listinfo/rsync Before posting,
> > read: http://www.catb.org/~esr/faqs/smart-questions.html 
> > <http://www.catb.org/%7Eesr/faqs/smart-questions.html>
> > 
> > 
> > 
> > 
> 
> - -- 
> ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~
> Kevin Korb Phone: (407) 252-6853
> Systems Administrator Internet:
> FutureQuest, Inc. Kevin at FutureQuest.net (work)
> Orlando, Florida kmk at sanitarium.net (personal)
> Web page: http://www.sanitarium.net/
> PGP public key available on web site.
> ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v2.0.17 (GNU/Linux)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
> 
> iEYEARECAAYFAk+HSbcACgkQVKC1jlbQAQfW+wCgn9wl1RFxLhFFaEAqQi7rbQcc
> i1MAoPqFk0qbcPvcBIlYYU5T7/HG0H6i
> =abbJ
> -----END PGP SIGNATURE-----
> -- 
> Please use reply-all for most replies to avoid omitting the mailing list.
> To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
> Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.samba.org/pipermail/rsync/attachments/20120413/3ef69dd3/attachment.html>