Question about suitability of rsync utility

[Dan Pritts]

In particular, each additional file means rsync needs more memory.

If you find that you don't have enough memory (or that you start
to swap!!) the easy solution is to split the rsync job into two
or more pieces.  If you've got any sort of directory hierarchy
this should be simple.  

On Sun, Aug 22, 2004 at 08:29:35PM +0400, Ivan Manida wrote:
> To be short - yes, we transferred larger quantities of files using rsync. Just make sure you have enough memory on the machines which run it, at least 512MB free for rsync's use and it'll sync a million file partition. 
> 
> So if you're sure you only got around 100K files, you'll be fine (does not calculate for me though, 1TB and 100K files - they should be 10MB each, not exactly very small :) ).
> 
> 
> ----- Original Message -----
> From: Sai Nanduru <snanduru at yahoo.com>
> Date: Sunday, August 22, 2004 6:50 pm
> Subject: Question about suitability of rsync utility
> 
> > Hi there,
> > I would like to ask a quick question to the experts on
> > this mailing list. If it has already been answered
> > please pardon me (and point me to the location where I
> > can view the answer).
> > 
> > I have a need to setup a mirror for an NFS file system
> > exported from a NetApp Filer device (based on BSD Unix
> > 4.x) to a Sun server. About 100,000+ very small files
> > get written to this file system every day on the Sun
> > server. The file system size is roughly 1.0 TB
> > currently. Please let me know if "rsync" utility can
> > handle this kind of load (100,000 files) to setup
> > backup mirror (backup new files to an alternate
> > location every night). 
> > 
> > Thank you very much,
> > NLS. 
> > 
> > __________________________________________________
> > Do You Yahoo!?
> > Tired of spam?  Yahoo! Mail has the best spam protection around 
> > http://mail.yahoo.com 
> > -- 
> > To unsubscribe or change options: 
> > http://lists.samba.org/mailman/listinfo/rsyncBefore posting, read: 
> > http://www.catb.org/~esr/faqs/smart-questions.html
> > 
> 
> -- 
> To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync
> Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html


danno
--
dan pritts                                       danno at internet2.edu
systems administrator                            734/352-4953 office
internet2                                        734/834-7224 mobile