rsync + tar archive performance
leen at consolejunky.net
Fri Feb 24 07:28:37 MST 2012
On Fri, Feb 24, 2012 at 03:16:16PM +0100, Bartlomiej Radziszewski wrote:
> On 02/24/12 14:46, Leen Besselink wrote:
>> On Fri, Feb 24, 2012 at 01:45:13PM +0100, Bartlomiej Radziszewski wrote:
>>> I have issue with taring speed on the directory created over rsync.
>>> I'm putting directory on a system using rsync (rsync -rv) then tarred it
>>> then puting the same directory on the server again using cp (cp -r) and
>>> tarred it.
>>> There is a result of my test http://pastebin.com/JsrVZps0 - looks like
>>> tar working two times slower on directory copied by rsync.
>>> - m4cp and m4sync contain same files/dirs (from ~100KB to ~3MB)
>>> - using rsync 3.0.7-2 (debian squeeze)
>>> - fs - ext4
>>> I'm not sure what's wrong there.. waiting for your suggestions.
>> Could it be there is nothing wrong ?
>> I assume the files already existed in the directory when rsync did it's
>> work. Because I think rsync was probably doing a 'quick check', where it looks
>> at the files properties like filesize and last modification time. From the manual:
>> -I, --ignore-times
>> Normally rsync will skip any files that are already the same size and have the same modification timestamp. This option turns off this
>> “quick check” behavior, causing all files to be updated.
>> This modifies rsync’s “quick check” algorithm for finding files that need to be transferred, changing it from the default of transferring
>> files with either a changed size or a changed last-modified time to just looking for files that have changed in size. This is useful when
>> starting to use rsync after using another mirroring system which may not preserve timestamps exactly.
>> So the files weren't read from disk by rsync after you cleared the cache, so they had to be read from disk instead of the filesystem cache when tar was running.
> Hi Leen,
> Thanks for your replay. I'm not sure if you correctly understand my problem.
I didn't understand it correctly, luckily I have an excuse I'm sick at home. ;-)
> I have two dirs 'm4cp' (created by cp -r) and 'm4rsync' (created by
> rsync -rv) and next i'm running tar on these dirs..
> And looks like tar processing on rsynced directory is two times slower
> then on the directory created by cp.
> Hope right now is more clear..
But is the data the same ? Because the filesizes are smaller for the second.
Or are these something like filesystem-image files ? Maybe you stored them
as sparse files ?
Could that be the reason for the difference in speed ?
> Please use reply-all for most replies to avoid omitting the mailing list.
> To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
> Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
More information about the rsync