rsync Digest, Vol 62, Issue 1
Zane Brady
Zane_Brady at trimble.com
Fri Feb 1 12:20:14 GMT 2008
Yep
Zane
-----Original Message-----
From: rsync-bounces+zane_brady=trimble.com at lists.samba.org [mailto:rsync-bounces+zane_brady=trimble.com at lists.samba.org] On Behalf Of rsync-request at lists.samba.org
Sent: Friday, February 01, 2008 7:01 AM
To: rsync at lists.samba.org
Subject: rsync Digest, Vol 62, Issue 1
Send rsync mailing list submissions to
rsync at lists.samba.org
To subscribe or unsubscribe via the World Wide Web, visit
https://lists.samba.org/mailman/listinfo/rsync
or, via email, send a message with subject or body 'help' to
rsync-request at lists.samba.org
You can reach the person managing the list at
rsync-owner at lists.samba.org
When replying, please edit your Subject line so it is more specific than "Re: Contents of rsync digest..."
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
---------------------------------------
Today's Topics:
1. Re: OS X xattr troubles (was Re: --exclude patterns)
(Mike Bombich)
2. Re: rsync-ing from two locations with same filenames (at
different versions) (Mojca Miklavec)
3. Re: rsync-ing from two locations with same filenames (at
different versions) (Matt McCutchen)
4. Re: rsync-ing from two locations with same filenames (at
different versions) (Mojca Miklavec)
----------------------------------------------------------------------
Message: 1
Date: Thu, 31 Jan 2008 06:54:44 -0600
From: Mike Bombich <mike at bombich.com>
Subject: Re: OS X xattr troubles (was Re: --exclude patterns)
To: Anthony Morton <amorton at fastmail.fm>
Cc: rsync at lists.samba.org
Message-ID: <3C41FC6F-4A4A-44D9-AC1C-B06A00E0B985 at bombich.com>
Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes
On Jan 30, 2008, at 11:03 PM, Anthony Morton wrote:
>>> I have a similar problem. I'm trying to specify a custom per-
>>> directory filter using
>>>
>>> --filter='dir-merge .rsync-filter-m'
>>>
>>> but because the whole thing is double-quoted the filter rule arrives
>>> in single quotes. I can't simply leave out the quotes here because
>>> the --filter option only gets the first token as its argument.....
>>>
>
>> You can use an underscore instead of a space after the "dir-merge".
>
> Thanks, that's solved it.
>
> Now there's a new problem I'd appreciate any help with. I've just
> started out with 3.0.0pre8 on OS X Leopard 10.5.1, patched according
> to Mike Bombich's recipe on this list at
> <http://lists.samba.org/archive/rsync/2008-January/019618.html>
>
> I have also run the backup-bouncer test as suggested (using -aHAX
> and --fileflags) and obtained results identical to Rob DuToit's.
> (Devices and fifos did not copy with their xattrs but all else was OK
> - this may be related to the bug Wayne opened.)
I think the errors that rsync complains about on devices and fifos should be ignored. I applied this patch and the errors go away and the backup-bouncer test succeeds (e.g. the fifos and devices are recreated without error):
diff -Naur rsync-3.0.0pre8/xattrs.c rsync-3.0.0pre8_mod/xattrs.c
--- rsync-3.0.0pre8/xattrs.c 2008-01-12 11:14:56.000000000 -0600
+++ rsync-3.0.0pre8_mod/xattrs.c 2008-01-28 22:31:11.000000000 -0600
@@ -128,7 +128,7 @@
}
if (list_len >= 0)
return list_len;
- if (errno == ENOTSUP)
+ if (errno == ENOTSUP || errno == EPERM)
return 0;
if (errno == ERANGE) {
list_len = sys_llistxattr(fname, NULL, 0); @@ -766,6 +766,8 @@
}
if (sys_lsetxattr(fname, name, rxas[i].datum, rxas[i].datum_len) <
0) {
+ if (errno == EPERM)
+ return 0;
rsyserr(FERROR_XFER, errno,
"rsync_xal_set: lsetxattr(\"%s\",\"%s\") failed",
fname, name);
I plan to submit that, I just haven't had a spare moment.
>
>
> However, trying a local rsync on my own home directory with -X and the
> destination set to a directory on a local HFS+-formatted FireWire disk
> (actually an iPod) immediately errors out with
>
> [sender] could not find xattr #2 for .
> rsync error: error in rsync protocol data stream (code 12) at
> xattrs.c(561) [sender=3.0.0pre8]
>
> Things seem to be OK if I omit -X to leave the xattrs out.
>
> Is this a bug or is there something up with my filesystem? (My
> installation's only a month old.)
I'm getting the same error with pre8 and the nightly from the 27th. I haven't boiled it down to a reproducible case yet, though. I tried to do that last night but ran out of time...
Mike
>
>
> Tony M.
>
> --
> To unsubscribe or change options:
> https://lists.samba.org/mailman/listinfo/rsync
> Before posting, read:
> http://www.catb.org/~esr/faqs/smart-questions.html
>
------------------------------
Message: 2
Date: Fri, 1 Feb 2008 02:06:23 +0100
From: "Mojca Miklavec" <mojca.miklavec.lists at gmail.com>
Subject: Re: rsync-ing from two locations with same filenames (at
different versions)
To: "Matt McCutchen" <matt at mattmccutchen.net>
Cc: rsync at lists.samba.org
Message-ID:
<6faad9f00801311706o7d341e6an4affeb85c4f60256 at mail.gmail.com>
Content-Type: text/plain; charset=UTF-8
On Jan 30, 2008 2:38 PM, Matt McCutchen wrote:
> On Wed, 2008-01-30 at 09:48 +0100, Mojca Miklavec wrote:
> > Neither helps. Even if I have a file of differest size and with a
> > different timestamp, and even if I add --checksum or --ignore-times,
> > the old file in dest won't be modified (overwritten by a newer file).
>
> I can't reproduce the problem. I ran the script in your original
> message, except I changed "b2" to "b22" and waited a few seconds
> before running that line so the file would get a later mtime. Both
> rsync 2.6.9 and the latest development rsync correctly replaced b.txt
> with the version from new/ on the second run. Am I missing something?
I don't know. Somtimes it works and sometimes not (but mostly not as a rule of thumb). Even if I wait for a few minutes inbetween, the new file won't be chosen.
> rsync --version
rsync version 2.6.3 protocol version 28
That might be old, but that was the default that came with fink on Mac OS X (if the error has been fixed in the meantime, I will upgrade).
> ll new/dir1/
skupno 4,0K
-rw-r--r-- 1 mojca wheel 6 feb 1 02:00 b.txt
> ll full/dir1/
skupno 8,0K
-rw-r--r-- 1 mojca wheel 2 jan 30 09:36 a.txt
-rw-r--r-- 1 mojca wheel 4 feb 1 01:52 b.txt
> cat full/dir1/b.txt
b12
> cat new/dir1/b.txt
b2222
> rsync -rpztlv --delete --checksum new/dir1/ new/dir2/ full/dir1/
> full/dir2/ dest
building file list ... done
./
sent 261 bytes received 20 bytes 562.00 bytes/sec total size is 24 speedup is 0.09
> cat dest/b.txt
b12
Thanks a lot,
Mojca
------------------------------
Message: 3
Date: Thu, 31 Jan 2008 20:36:26 -0500
From: Matt McCutchen <matt at mattmccutchen.net>
Subject: Re: rsync-ing from two locations with same filenames (at
different versions)
To: Mojca Miklavec <mojca.miklavec.lists at gmail.com>
Cc: rsync at lists.samba.org
Message-ID: <1201829786.7077.72.camel at localhost>
Content-Type: text/plain; charset=UTF-8
On Fri, 2008-02-01 at 02:06 +0100, Mojca Miklavec wrote:
> I don't know. Somtimes it works and sometimes not (but mostly not as a
> rule of thumb). Even if I wait for a few minutes inbetween, the new
> file won't be chosen.
>
> > rsync --version
> rsync version 2.6.3 protocol version 28
>
> That might be old, but that was the default that came with fink on Mac
> OS X (if the error has been fixed in the meantime, I will upgrade).
Duh. I realize now that it's perfectly reasonable for you to be able to reproduce the problem while I can't. Versions 2.6.9 and earlier of rsync sort the file-list using the C library's quicksort, an unstable sort, so the results in case of duplicate files are highly sensitive to both the C library implementation and the order of directory entries in the source (which in turn is sensitive to the filesystem implementation). You probably have both a different C library and a different filesystem than I do.
In any case, since rsync 3.0.0pre1, the default file-list sorting algorithm is a mergesort, which is stable, so files from earlier source arguments take priority. If you upgrade to an rsync 3.0.0pre* version, your scenario should work consistently. If it doesn't, that's a bug we should try to fix.
Matt
------------------------------
Message: 4
Date: Fri, 1 Feb 2008 09:46:41 +0100
From: "Mojca Miklavec" <mojca.miklavec.lists at gmail.com>
Subject: Re: rsync-ing from two locations with same filenames (at
different versions)
To: "Matt McCutchen" <matt at mattmccutchen.net>
Cc: rsync at lists.samba.org
Message-ID:
<6faad9f00802010046t7629e852jfda267de42a3dd77 at mail.gmail.com>
Content-Type: text/plain; charset=UTF-8
On Feb 1, 2008 2:36 AM, Matt McCutchen wrote:
> On Fri, 2008-02-01 at 02:06 +0100, Mojca Miklavec wrote:
> > I don't know. Somtimes it works and sometimes not (but mostly not as
> > a rule of thumb). Even if I wait for a few minutes inbetween, the
> > new file won't be chosen.
> >
> > > rsync --version
> > rsync version 2.6.3 protocol version 28
> >
> > That might be old, but that was the default that came with fink on
> > Mac OS X (if the error has been fixed in the meantime, I will upgrade).
>
> Duh. I realize now that it's perfectly reasonable for you to be able
> to reproduce the problem while I can't. Versions 2.6.9 and earlier of
> rsync sort the file-list using the C library's quicksort, an unstable
> sort, so the results in case of duplicate files are highly
> sensitive to both the C library implementation and the order of
> directory entries in the source (which in turn is sensitive to the
> filesystem implementation). You probably have both a different C
> library and a different filesystem than I do.
>
> In any case, since rsync 3.0.0pre1, the default file-list sorting
> algorithm is a mergesort, which is stable, so files from earlier
> source arguments take priority. If you upgrade to an rsync 3.0.0pre*
> version, your scenario should work consistently. If it doesn't,
> that's a bug we should try to fix.
Oh, thanks a lot for the explanation :)
I will try to figure out how to build it and test then.
In this particular case it is not needed, but what about a switch:
--always-take-the-latest-file-version-no-matter-of-order-of-supplied-arguments
?
Two problems still remain then: one is that I cannot rely on the fact that users will have the latest version (at least for a while, but that argument will disapear with time), and the other one - it would simplify the work a lot if I didn't have to reproduce a bunch of empty folders in "new" (for which you have said: "I agree that this feature is useful, and I might implement it at some point as a patch.")
But thanks a lot for the explanation. You just gave me a really nice idea for a real-life task for the forthcomming informatics competition here :)
Mojca
------------------------------
_______________________________________
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
rsync mailing list
rsync at lists.samba.org
https://lists.samba.org/mailman/listinfo/rsync
End of rsync Digest, Vol 62, Issue 1
************************************
More information about the rsync
mailing list