sync 54 million files, tuning rsync?

Jerry juanino at yahoo.com
Wed Feb 15 18:59:05 GMT 2006


Believe it or not, NDMPcopy was slower.  At best it
was the same speed.  NDMPcopy takes 4 hours to lay
down the files and directories, then it starts copying
data. (ontap ver 6.4.5).



--- Ryan Kather <RDKath at Roushind.com> wrote:

> Because of the inefficiencies of using NFS to NFS
> rsync, would you not be better using a native
> utility from filer to file, such as ndmp copy? 
> 
> >>> Jerry <juanino at yahoo.com> 02/13/06 09:31PM >>>
> I'm trying to sync up 54 million files.  I can break
> it down into different applications, but I still
> have
> to accomplish 17 million files in one "chunk" if
> possible.
> 
> I'm using a fast (v440) cpu system with 32G ram. 
> Originally I was running out of memory (seemed to
> increase like crazy with --delete on).  I've
> upgraded
> to rsync 2.6.6 and the memory seems to be more
> stable
> now, but it's still not using much CPU to perform
> it's
> task, I have CPU to spare!  I'm trying to sync from
> one NFS mount to another NFS mount (Netapp).
> 
> Is there anyway I can force rsync to use *more*
> resources?  My Netapp doesn't seem to be pounded and
> my V440 is only using 2%-8% of cpu, 27G of real
> memory
> free.  Am I doing something wrong?  It's going on 5
> hours now.  It's "considered" all the files and
> printed out a few it deleted, but that this point
> I'm
> not sure where it is at.
> 
> __________________________________________________
> Do You Yahoo!?
> Tired of spam?  Yahoo! Mail has the best spam
> protection around 
> http://mail.yahoo.com 
> -- 
> To unsubscribe or change options:
> https://lists.samba.org/mailman/listinfo/rsync 
> Before posting, read:
> http://www.catb.org/~esr/faqs/smart-questions.html
> 
> --
> To unsubscribe or change options:
> https://lists.samba.org/mailman/listinfo/rsync
> Before posting, read:
> http://www.catb.org/~esr/faqs/smart-questions.html
> 


__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 


More information about the rsync mailing list