rsync 1tb+ each day
jw at pegasys.ws
Thu Feb 6 06:31:41 EST 2003
On Wed, Feb 05, 2003 at 10:53:25AM -0800, Kenny Gorman wrote:
> Eric Whiting wrote:
> >I've learned some good things from this discussion. THanks.
> >Kenny, I have one concern/idea -- The original post says the 'disk is
> >fairly slow'. That is one bottleneck that should probably be examined a
> >little more. How fast are your disks? HOw fast is your network? An IDE
> >disk with DMA disabled might run 5M/s and when you enable DMA you can
> >see up to 45M/s. Perhaps this is a root cause problem that has already
> >been looked at, but perhaps it would be good to look at it again. Also
> >do you have enough RAM on the destination to do some caching of the file
> >for the multiple reads of the file? That might also help.
> The disks on the destination side are a stripe on a sun 5200. not sure
> of the underlaying disk size, etc at this point. On top of that we have
> the VxFS file system with caching enabled. The box has 12gb ram. It
> does indeed cache the second read of the file, so thats good.
> We are going to speed up the disks, and try this again very soon (it's
> going to take a while for us to get the 2tb file system rebuilt ;-)
> thx for the tips!
Been a couple of years since i worked one of those.
I'd guess you have 10KRPM 100GB drives in a 22 drive FCAL cabinet.
If so you either have two cabinets or RAID-5. Software
RAID-5 would certainly explain poor write performance.
That hardware is certainly capable of reasonable
One thing i'd certainly look at is the VxFS cache mode. I
seem to recall it supporting both write-through and
write-back. If so, try switching to write-back caching that
will allow the elevators and scatter-gather to do their
J.W. Schultz Pegasystems Technologies
email address: jw at pegasys.ws
Remember Cernan and Schmitt
More information about the rsync