rsync + tar archive performance
b at radziszewski.com
Fri Feb 24 07:16:16 MST 2012
On 02/24/12 14:46, Leen Besselink wrote:
> On Fri, Feb 24, 2012 at 01:45:13PM +0100, Bartlomiej Radziszewski wrote:
>> I have issue with taring speed on the directory created over rsync.
>> I'm putting directory on a system using rsync (rsync -rv) then tarred it
>> then puting the same directory on the server again using cp (cp -r) and
>> tarred it.
>> There is a result of my test http://pastebin.com/JsrVZps0 - looks like
>> tar working two times slower on directory copied by rsync.
>> - m4cp and m4sync contain same files/dirs (from ~100KB to ~3MB)
>> - using rsync 3.0.7-2 (debian squeeze)
>> - fs - ext4
>> I'm not sure what's wrong there.. waiting for your suggestions.
> Could it be there is nothing wrong ?
> I assume the files already existed in the directory when rsync did it's
> work. Because I think rsync was probably doing a 'quick check', where it looks
> at the files properties like filesize and last modification time. From the manual:
> -I, --ignore-times
> Normally rsync will skip any files that are already the same size and have the same modification timestamp. This option turns off this
> “quick check” behavior, causing all files to be updated.
> This modifies rsync’s “quick check” algorithm for finding files that need to be transferred, changing it from the default of transferring
> files with either a changed size or a changed last-modified time to just looking for files that have changed in size. This is useful when
> starting to use rsync after using another mirroring system which may not preserve timestamps exactly.
> So the files weren't read from disk by rsync after you cleared the cache, so they had to be read from disk instead of the filesystem cache when tar was running.
Thanks for your replay. I'm not sure if you correctly understand my problem.
I have two dirs 'm4cp' (created by cp -r) and 'm4rsync' (created by
rsync -rv) and next i'm running tar on these dirs..
And looks like tar processing on rsynced directory is two times slower
then on the directory created by cp.
Hope right now is more clear..
More information about the rsync