[clug] using rsync for backups

Andrew Janke a.janke at gmail.com
Mon May 26 00:47:07 GMT 2008


> On 2008/May/23, at 4:21 AM, Michael James wrote:
>> From my experience backing up 3/4 of a terrabyte using rsync,
>> I've had some thoughts for possible improvements.
>>
On Mon, May 26, 2008 at 4:13 AM, Kim Holburn <kim.holburn at gmail.com> wrote:
> Can't you run an rsync daemon on the backup device?  Then hopefully the
> filelist would be generated when the process started up.

While this is not exactly on-topic (improvements to rsync) I have
always just done this sort of thing on large rsyncs. In this case
large == 4.1TB.

---

date=`date +%Y%m%d-%H%M%S`
src=/data/raid/raid01
dest=/export_lv00/raid01-backup

# mirror the raid(s)
for i in `ls -1 $src`
do
   rsync -av --stats --delete-after \
      $src/$i $dest > $dest/logs/export-raid01-$i-$date.log
done

---

For some reason this always seems to complete faster than just
rsyncing the entire tree in one bash. I also seem to remember a
wrapper aound rsync that does this called gigsync but it is a whole
lot cleverer about how it chops things up.  In my case I just happen
to know that there is a fairly even split of data in the first level
of directories.

   http://matthew.mceachen.us/geek/gigasync/


a

PS: Yes I know I _should_ use find instead of ls -1, call me slack.


More information about the linux mailing list