How to speed up rsync when haveing lots of files
Boniforti Flavio
flavio at piramide.ch
Wed Mar 4 07:26:18 GMT 2009
> You should not use -z at ALL with large numbers of small
> files. The increased latency of the
> compression/decompression will far exceed any time saved in
> transmission. You're not on a 300 baud modem, I assume?
> -z should be used when you are sending large files that are
> compressible in the first place (so not video or mp3s, for
> example), but it it only going to hurt you on small files.
I was lurking your thread, when I stumbled across your considerations
about "-z" option. I'm having issues with slow network, which I presume
is the main cause of failure in rsnapshot sync tasks.
Do you suggest to try *without* compression to see if something works
better?
> How small? Depends on your connection. My rule of thumb,
> based on nothing at all, is that the speed in megabits of my
> connection is the
> size in megabytes that I start worrying about compression.
> Compressible data over 15MB in size? OK, I will start to
> think about using -z. If I was on a T1? 1.5MB. ADSL? 700K.
> Arbitrary, perhaps, but it seems to have served me well. In
> fact, I found removing -z sped up transfers quite a bit on my LAN.
Uhm... It's quite interesting the way you somehow "arbitrarily" choose
when to worry about compression....
Is there any way to know *in advance* if using or not using "-z" could
be the better solution?
TIA,
Flavio.
More information about the rsync
mailing list