Using --link-dest works on small directory tree but not large one

Robot Robot arbor.hill.fashion.society at gmail.com
Thu Apr 6 10:19:44 GMT 2006


I'm using version 2.6.3 on OS X. I'm trying to use --link-dest to keep some
rolling backups of production websites. There is about 2GB of data in 60,000
total files. I do an initial rsync to get an initial copy of the data, and
each day thereafter I run rsync again with --link-dest to create a new daily
backup. For some reason, in the subsequent runs, rsync updates a whole lot
of files that haven't changed at all. The files in the source and in the
link-dest location have identical contents, are the same size, and have the
same modified date. I can't seem to find any patterns in the files it
decides to bring back down the subsequent times. Sometimes it will bring
down the entire directory tree again, and sometimes it will bring down
slightly less than the whole tree.

I've tried running the same commands on subsets of the directory tree, and
everything works perfectly. The problem only appears when I try to sync my
entire directory tree.

This is what I'm doing:

$ rsync -vv --progress --archive --compress --numeric-ids --delete -e ssh
blah at blah.com:/usr/home/blah/websites/ /backup/websites/1
$ rsync -vv --progress --archive --compress --numeric-ids --delete
--link-dest=../1 -e ssh blah at blah.com:/usr/home/blah/websites/
/backup/websites/2
$ du -hsc /backup/websites/*
2.1G    /backup/websites/1
2.1G    /backup/websites/2
4.2G    total

If, however, I run the following, it all seems to work perfectly:

$ rsync -vv --progress --archive --compress --numeric-ids --delete -e ssh
blah at blah.com:/usr/home/blah/websites/site1.com /backup/websites/1
$ rsync -vv --progress --archive --compress --numeric-ids --delete
--link-dest=../1 -e ssh blah at blah.com:/usr/home/blah/websites/site1.com
/backup/websites/2
$ du -hsc /backup/websites/*
35M    /backup/websites/1
0    /backup/websites/2
35M    total

This is frustrating to debug because there's a lot of data being
transferred, and it only occurs when I try to sync the full tree. Is this a
bug? Am I doing something completely wrong and stupid? I'm happy to provide
more detailed info if you need it. Thanks!!
-------------- next part --------------
HTML attachment scrubbed and removed


More information about the rsync mailing list