performance suggestion: sparse files

Jon Howell jonh at jonh.net
Wed Sep 10 08:38:44 EST 2003


> I'd want to be convinced that this was really enough cheaper than -z1 to
> justify the complexity.
Right; as I thought about it more, it's only interesting in a corner case.
If you have enough CPU cycles lying around (because you're bottlenecked on
the network, and your CPU isn't busy with other stuff), compressing the
incompressible doesn't matter. You burn some small fraction of CPU, but
buffering and rsync's pipeline means it has no effect on the throughput of
the transfer.

So I agree that my original suggestion is probably not worth the added
complexity.

> Actually you can guess by looking at the allocated-blocks measure, and
> use this to guess whether it's preallocated zeros or sparse, which might
> be useful for backups.  But there is no way around reading the blocks.
Sure. Bummer; that's a lot of memory bus bandwidth (having the kernel
zero-fill the blocks, then having rsync zero-compare them) wasted. Seems
like a fcntl() is in order.

   --Jon





More information about the rsync mailing list