DO NOT REPLY [Bug 6276] crtimes.patch does not preserve creation dates on Mac x86_64 only

samba-bugs at samba.org samba-bugs at samba.org
Sat Feb 6 00:14:45 MST 2010


https://bugzilla.samba.org/show_bug.cgi?id=6276





------- Comment #6 from steve at ortizaggies.com  2010-02-06 01:14 CST -------
Created an attachment (id=5288)
 --> (https://bugzilla.samba.org/attachment.cgi?id=5288&action=view)
patch to fix creation dates bug on Mac x86_64

I ran into this same bug (6276) today, using the same tool (backup bouncer) to
check how rsync would work on Mac OS X (Snow Leopard, 10.6.2).  rsync passed
every test except for the "creation-date" test... this was surprising because I
had installed the crtimes (and fileflags) patch.  I expected rsync to work
flawlessly.

I looked into it and figured out the problem was in syscall.c, which effects
both rsync and the testsuite.  Following is a summary of the problem, the fix,
and steps to verify the problem and the fix.

Problem:

The get_create_time function calls the getattrlist system call, but the
assumptions that worked on a 32-bit version don't hold for 64-bit.  Here's an
excerpt from the man page for getattrlist...

=============================

The data returned in the buffer described by attrBuf and attrBufSize is
formatted as follows.

1.   The first element of the buffer is a u_int32_t that contains the overall
length, in bytes, of the attributes returned. This size includes the length
field itself.

2.   Following the length field is a list of attributes.  Each attribute is
represented by a field of its type, where the type is given as part of the
attribute description (below).

3.   The attributes are placed into the attribute buffer in the order that they
are described below.

4.   Each attribute is aligned to a 4-byte boundary (including 64-bit data
types).

=============================

The key thing to note is that the data start 4 bytes into the buffer.  The
current implementation uses the following struct as the attrBuf...

struct create_time {
        unsigned long length;
        struct timespec crtime;
};

Assuming the unsigned long is 4 bytes, and the crtime field continues
immediately afterwards, this would be valid.  Unfortunately, what's happening
is the crtime field starts 8 bytes into the struct, so it is 4 bytes offset,
and the time returned is 0, which is also the value returned by get_create_time
to indicate an error.  Since this is not really a problem with the system call,
it's a problem with how the result is being extracted, the errno is 0 (Unknown
error).

Unfortunately, the crtimes test in the test suite passes.  crtimes.test calls
on checkit in rsync.fns, which calls on rsync_ls_lR in the same file, which
calls on the tls tool, which uses the same get_create_time function that's
causing rsync to fail.  tls produces the following stderr message with the
--crtimes (or -N) option, but it results in a zero length output, which matches
the other file listing it compares to "verify" that crtimes.test is working. 
That's why the test appears to pass.

tls: get_create_time show-bug/from/foo: Unknown error: 0

Fix:

So, the fix is fairly straightforward... the get_create_time function needs to
grab the right set of bytes.  My patch is attached... you can see I got rid of
the struct create_time, and I use a char array to load results from the system
call.  It works, but if you want to do it differently, I won't mind one bit (or
4 bytes).

Verify:

Here's what you can do to verify the problem and the fix.  Note, I think this
problem is limited to Mac OS X (64-bit), so you'll need to test it accordingly.

tar xzf rsync-3.0.7.tar.gz 
tar xzf rsync-patches-3.0.7.tar.gz 
cd rsync-3.0.7
patch -p1 <patches/fileflags.diff 
patch -p1 <patches/crtimes.diff 
./configure
make
make test
# it should pass all the tests, so here's how you can manually see it is really
broken...
mkdir -p show-bug/from
mkdir show-bug/to
touch -t 200111111111.11 show-bug/from/foo  # created date/time
touch -t 200212122222.22 show-bug/from/foo  # modified date/time
./rsync -rtg --crtimes show-bug/from/ show-bug/to
./tls -N show-bug/from/foo  # this is how the testsuite tries to verify it...
broken test
GetFileInfo -d show-bug/from/foo  # shows created date
GetFileInfo -d show-bug/to/foo    # shows the modified date, but should show
created... broken rsync

Now, you should be able to run my patch and go through the same steps... you'll
see rsync applies the correct creation date now.

patch -p1 <../fix_get_create_time.patch   # apply my patch (wherever you saved
it)
make
make test  # tests still pass, but tls works this time so it's more meaningful
rm show-bug/to/foo
./rsync -rtg --crtimes show-bug/from/ show-bug/to
./tls -N show-bug/from/foo  # shows correct output now, no errors
./tls -N show-bug/to/foo  # output matches above, good
GetFileInfo -d show-bug/from/foo  # shows created date
GetFileInfo -d show-bug/to/foo    # output matches above, good

More comments:

To be thorough, the test for crtime should probably be updated too, so it fails
if the files are empty.  There are a lot of ways to do this... You may want to
consider adding some output to stdout when tls fails... you could then grep for
that text in the checkit function to see if any tls commands failed, and then
you'd probably want to fail that test.  Otherwise, similar problems to this
could silently pass.

Philosophically, maybe it would be better not to use tls in the testing, but I
guess you've got to trust something for your tests.  This was probably a rare
case, where the same bug was affecting both rsync and the testsuite.

Thanks to everyone who maintains rsync!  It's a great tool, and I wouldn't have
bothered to write up this fix if I didn't find it useful.  Thanks to n8gray for
backup bouncers, and Mike Bombich who has some nice rsync for Mac OS X help on
his website.

Here are some odds and ends... the rsync version info and my backup bouncer
0.2.0 test results for my modified rsync (the untouched version 3.0.7 fails the
creation-date test, but this fix will produces a version of rsync that passes
everything).  If you need anything else to close out this bug, please let me
know.  I am glad to help.

$ rsync --version
rsync  version 3.0.7  protocol version 30
Copyright (C) 1996-2009 by Andrew Tridgell, Wayne Davison, and others.
Web site: http://rsync.samba.org/
Capabilities:
    64-bit files, 64-bit inums, 64-bit timestamps, 64-bit long ints,
    socketpairs, hardlinks, symlinks, IPv6, batchfiles, inplace,
    append, ACLs, xattrs, no iconv, symtimes, file-flags


$ sudo ./bbouncer copy -c 15-rsync-3.0.7 -d /Volumes/Src /Volumes/Dst
src = /Volumes/Src
dst = /Volumes/Dst
Enabling owners on src/dst disks
/dev/disk1s1 on /Volumes/Src (hfs, local, journaled)
/dev/disk2s1 on /Volumes/Dst (hfs, local, journaled)
Cleaning.
Copying with:          rsync-3.0.7 ... ok

------------------ rsync-3.0.7 ------------------
Verifying:    basic-permissions ... ok (Critical)
Verifying:           timestamps ... ok (Critical)
Verifying:             symlinks ... ok (Critical)
Verifying:    symlink-ownership ... ok 
Verifying:            hardlinks ... ok (Important)
Verifying:       resource-forks ... 
   Sub-test:             on files ... ok (Critical)
   Sub-test:  on hardlinked files ... ok (Important)
Verifying:         finder-flags ... ok (Critical)
Verifying:         finder-locks ... ok 
Verifying:        creation-date ... ok 
Verifying:            bsd-flags ... ok 
Verifying:       extended-attrs ... 
   Sub-test:             on files ... ok (Important)
   Sub-test:       on directories ... ok (Important)
   Sub-test:          on symlinks ... ok 
Verifying: access-control-lists ... 
   Sub-test:             on files ... ok (Important)
   Sub-test:              on dirs ... ok (Important)
Verifying:                 fifo ... ok 
Verifying:              devices ... ok 
Verifying:          combo-tests ... 
   Sub-test:  xattrs + rsrc forks ... ok 
   Sub-test:     lots of metadata ... ok 


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug, or are watching the QA contact.


More information about the rsync mailing list