Professional grade rsync?
jim at jrssystems.net
Fri May 28 18:32:56 GMT 2004
Silly suggestion, perhaps, but...
Ever considered simply breaking your sync down into several separate
sets? IE instead of rsync /foo/baz user at machine:/foo, where /foo/baz
contains ten directories each then spiraling down into hundreds of
thousands more... rsync /foo/baz/1 user at machine:/foo/baz, then rsync
/foo/baz/2 user at machine:/foo/baz, and so on. Enterprising sorts can, of
course, write wrappers in Perl or the shell of your choice to
automatically sync everything in /foo/baz one directory and/or file at a
time with ease.
I haven't personally run into your upper end limits on heavy-duty
equipment, but I've used the technique described above to keep old 32MB
and 64MB machines in successful service as rsync servers; the principle
is the same - scale your single sync down to a level that your hardware
(It should also be possible to make rsync more memory-friendly when the
size of the job exceeds the resources available, by causing it to do the
exact same thing as the wrapper described above does, but I'm not going
> Hi, folks.
> We've gone where no man has gone before. On HP-UX, rsync bombs at about
> 1.75 million directories, files and links (combined sum) in a single
> Is there a professional-grade alternative on HP-UX for folks willing to pay
> for it? It wouldn't even need to be network-aware, just from single-system
> areas to the same box, but with the nifty delete and update features that
> rsync has. My searches turn up unison and some other tools (BSD mirror,
> etc.), but rsync has beaten any other open-source solution hands down on the
> scalability side of things. Now, we need more ...
> A. Daniel King, System Analyst
> HP-UX, Linux, Solaris
More information about the rsync