Macintosh rsync: Inconsistent Behaviour: Hints Appreciated

Mike Bombich mike at bombich.com
Thu Feb 12 02:40:24 GMT 2009


You're using the version of rsync that is built-in to Mac OS X.  Don't  
do that.  It's a fairly old version (though it is maintained by  
Apple), and it doesn't handle extended attribute modifications (or  
lack thereof), thus it transfers them every time.  When you have many  
files with resource forks or other extended attributes, you'll see  
files getting copied every time regardless of whether they've been  
modified.  rsync 3.0.x handles these much more efficiently.

I posted some tips for compiling and running rsync on Mac OS X here:

http://www.bombich.com/mactips/rsync.html

Note that the arguments will be slightly different, so be sure to  
review the man page and modify your script accordingly.  Also be sure  
to call rsync from /usr/local/bin/rsync (or wherever you choose to  
install it), otherwise you'll still be using the built-in version.

Mike

On Feb 11, 2009, at 6:59 PM, Bob McKay wrote:

> I have a problem with rsync on macintosh copying files that have not  
> been updated. Despite careful reading of the archives, I can't find  
> an explanation (or a fix), but I strongly suspect I'm just missing  
> something that an expert will recognise instantly... Your assistance  
> would be greatly appreciated.
>
> I'm running a nightly backup of my photo/home video directory from a  
> partition BIGPIX on a firewire drive on one machine (G3 running OSX  
> 10.3, imaginatively named g3) to another (Pix) on a firewire drive  
> on my laptop (G4 running 10.4, even more imaginatively bobsmac).  
> I'll include the script below. All are HFS+ partitions.
>
> The problem is, that every night, something like 3/4 of the whole  
> directory gets updated. This doesn't reflect reality - in fact, I  
> only add stuff to the directory once a week or so. If the whole of  
> the directory was getting updated, I'd understand that I just had  
> the options wrong, but it's completely inconsistent - some files get  
> re-synched night after night, while others are just synched once and  
> then rsync does what I expected and leaves them alone. I can't see  
> any consistency in this. There is some correlation with age (the  
> older directories are much more likely to be re-synched than newer  
> ones). But this correlation is not reliable. I'll use an example of  
> a particular directory from May 2008. Of five files in the  
> directory, two (creation dates Jan 7 2008) are synched every night.  
> The other three are omitted. One has creation date Nov. 15 2007, the  
> other two April 14 2008. Other than age, I can't see any difference  
> between the files, either from unix shell (see below) or from finder  
> info (I'd include this if I could figure a way to cut and paste, but  
> please accept that it is completely consistent with the shell  
> results). The rsync options I'm using are '-vpcrutxzE --ignore- 
> existing'; I've read the man pages carefully, and I can't understand  
> why they would produce this behaviour.
>
> If you can see any way I can further diagnose this, any hints would  
> be appreciated - I'm completely stuck. Of course, if you know why  
> it's happening - or know a fix - that would be even better!
>
>      Thanks and Best Wishes
>      Bob
>
> *********************** directory listings of example source and  
> target directories  ********************
>
> G3:/Volumes/BIGPIX/PIXREADONLY/200805-FriendsMisc080502 rim$ ls -l
> total 12472
> -rwxr-xr-x   1 root  admin  1827015 Apr 14  2008 HannahP1060453.jpg
> -rwxr-xr-x   1 root  admin  2219607 Apr 14  2008 HannahP1060578.jpg
> -rwxr-xr-x   1 root  admin   192966 Jan  7  2008 VikyP1030042.jpg
> -rwxr-xr-x   1 root  admin   206962 Jan  7  2008 VivP1030020.jpg
> -rwxr-xr-x   1 root  admin  1900455 Nov 15  2007 WOOK-DSCN3830.jpg
>
> bobsmac:/Volumes/Pix/PIXBACKUP/PIXREADONLY/200805-FriendsMisc080502  
> rim$ ls -l
> total 12472
> -rwxr-xr-x   1 rim  rim  1827015 Apr 14  2008 HannahP1060453.jpg
> -rwxr-xr-x   1 rim  rim  2219607 Apr 14  2008 HannahP1060578.jpg
> -rwxr-xr-x   1 rim  rim   192966 Jan  7  2008 VikyP1030042.jpg
> -rwxr-xr-x   1 rim  rim   206962 Jan  7  2008 VivP1030020.jpg
> -rwxr-xr-x   1 rim  rim  1900455 Nov 15  2007 WOOK-DSCN3830.jpg
>
> (Viky and Viv get synched repeatedly, the rest do not)
>
>
> *********************** abbreviated transcript showing relevant  
> stages of copying  ********************
>
> Copying g3.local:/Volumes/BIGPIX/PIXREADONLY /Volumes/Pix/PIXBACKUP
> receiving file list ... done
> PIXREADONLY/._.DS_Store
> PIXREADONLY/200202-China/Wuhan Early Pics/._PIC_6250.JPG
> .
> .
> .
> PIXREADONLY/200805-FriendsMisc080502/._.DS_Store
> PIXREADONLY/200805-FriendsMisc080502/._VikyP1030042.jpg
> PIXREADONLY/200805-FriendsMisc080502/._VivP1030020.jpg
> PIXREADONLY/200805-KaesongNK080511/._.DS_Store
> .
> .
> .
>
> sent 59156 bytes  received 115825818 bytes  48599.28 bytes/sec
> total size is 22910676123  speedup is 197.70
>
>
> *********************** shell script  ********************
>
> #!/bin/sh
>
> SRCDIR="g3.local:/Volumes/BIGPIX/PIXREADONLY"
> DESTDIR="/Volumes/Pix/PIXBACKUP"
> echo "Copying "$SRCDIR" "$DESTDIR
> rsync -vpcrutxzE --ignore-existing\
>         --rsync-path="/usr/bin/rsync" root@$SRCDIR $DESTDIR
> -- 
> Please use reply-all for most replies to avoid omitting the mailing  
> list.
> To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
> Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html



More information about the rsync mailing list