cut-off time for rsync ?
Ken Chase
rsync-list-m829 at sizone.org
Thu Jul 2 13:39:49 UTC 2015
Yes if rsync could keep a 'last state file' that'd be great, which would
require the target be unchanged by any other process/usage - this is however
the case with many of our uses here - as a backup only target.
Then it could just load the target statefile, and only scan the source
for changes vs the last-state file.
Cant think of any way around this issue with rsync alone without some external
parsing of previous logs, etc.
This is unfortunately why I never use 5400/5900 rpm disks on my backup targets,
and use raid 10 not 5, for speed. Little more $ in the end, but necessary
to scan 50-80M inodes per night in my ~6hr backup window.
/kc
On Thu, Jul 02, 2015 at 11:43:37AM +0200, Dirk van Deun said:
>> What is taking time, scanning inodes on the destination, or recopying the entire
>> backup because of either source read speed, target write speed or a slow interconnect
>> between them?
>
>It takes hours to traverse all these directories with loads of small
>files on the backup server. That is the limiting factor. Not
>even copying: just checking the timestamp and size of the old copies.
>
>The source server is the actual live system, which has fast disks,
>so I can afford to move the burden to the source side, using the find
>utility to select homes that have been touched recently and using
>rsync only on these.
>
>But it would be nice if a clever invocation of rsync could remove the
>extra burden entirely.
>
>Dirk van Deun
>--
>Ceterum censeo Redmond delendum
--
Ken Chase - ken at heavycomputing.ca skype:kenchase23 +1 416 897 6284 Toronto Canada
Heavy Computing - Clued bandwidth, colocation and managed linux VPS @151 Front St. W.
More information about the rsync
mailing list