creation date and OSX [performance]

Vitorio Machado v.machado at
Sat Feb 2 08:24:16 GMT 2008


I think it's OK to run getattrlist once assuming that there are  
creation date. My arguments:

1) First of all, most of Macs run only on HFS+, some exceptions will  
be those running under UFS (only saw one person talk about this for a  
server, but I think it's very rare) or that have volumes under FAT/ 
NTFS filesystem to be compatible with Windows.

2) From getattrlist manpage:
     The getattrlist() function is only supported by certain volume  
      implementations.  For maximum compatibility, client programs  
should use
      high-level APIs (such as the Carbon File Manager) to access  
file system
      attributes.  These high-level APIs include logic to emulate  
file system
      attributes on volumes that don't support getattrlist().

In other words, if we really care about compatibility, we should use  
Carbon system call that tests it for us and make the dirt work for  
us. May be a good idea.

3) Also from getattrlist manpage:
      Not all volumes support all attributes.  See the discussion of
      ATTR_VOL_ATTRIBUTES for a discussion of how to determine  
whether a par-
      ticular volume supports a particular attribute.

I don't really know what discussion it refers, but I suppose it  
should be on the Apple developer site . I  
didn't have the time to look, yet.

4) Always from getattrlist manpage:

      Not all volumes support getattrlist().  The best way to test  
whether a
      volume supports this function is to simply call it and check  
the error
      result.  getattrlist() will return ENOTSUP if it is not  
supported on a
      particular volume.

I suppose that a getattrlist on unsupported volumes will return this  
error, if it works like I'm expecting, we should only catch it and  
that's it.

Also note that (always from getattrlist manpage):

      The getattrlist() function has been undocumented for more than  
two years.
      In that time a number of volume format implementations have  
been created
      without a proper specification for the behaviour of this  
routine.  You
      may encounter volume format implementations with slightly  
different be-
      haviour than what is described here.  Your program is expected  
to be tol-
      erant of this variant behaviour.

So, there are some clues to be checked. I will probably check  
something if I have some time, but I already engaged myself with 10.3  
compatibility and I had unexpected personal problems. So I can't say  
when I would be able to give some time for those projects.

Best regards,


Le 2 févr. 08 à 07:18, Mike Bombich a écrit :

> Looking at this patch from a performance perspective, it appears  
> that getattrlist is called twice for every file:
> 23:57:24.341  lstat                                  00-basic- 
> permissions/owned-by- 
> root                                                                   
>                                     0.000011   rsync
> 23:57:24.341  listxattr                              00-basic- 
> permissions/owned-by- 
> root                                                                   
>                                     0.000006   rsync
> 23:57:24.341  getattrlist                            00-basic- 
> permissions/owned-by- 
> root                                                                   
>                                     0.000006   rsync
> 23:57:24.341  getattrlist                            00-basic- 
> permissions/owned-by- 
> root                                                                   
>                                     0.000005   rsync
> 23:57:24.341  lstat                                  00-basic- 
> permissions/owned-by- 
> www                                                                    
>                                     0.000008   rsync
> 23:57:24.341  listxattr                              00-basic- 
> permissions/owned-by- 
> www                                                                    
>                                     0.000005   rsync
> 23:57:24.341  getattrlist                            00-basic- 
> permissions/owned-by- 
> www                                                                    
>                                     0.000006   rsync
> 23:57:24.341  getattrlist                            00-basic- 
> permissions/owned-by- 
> www                                                                    
>                                     0.000005   rsync
> The first time it is called by
> sys_llistxattr()
> 	getCreationTime()
>  -- basically we determine if the file has a creation date.  If it  
> does, then add the CRTIME_XATTR string to the xattr list.  The  
> creation date isn't actually cached here, though.  To get the  
> actual creation date, getattrlist is called again via xattrs.c
> get_xattr_data()
> 	sys_lgetxattr()
> 		get_crtime_xattr()
> 			getCreationTime()
> The performance hit is significant, and I'm wondering how safe it  
> is to simply assume that every file has a creation date (given that  
> this section is wrapped in #if HAVE_OSX_XATTRS), therefore, drop  
> the first getCreationDate and add the CRTIME_XATTR string to the  
> xattr list by default.  For example:
> // sysxattrs.c:150
> ssize_t sys_llistxattr(const char *path, char *list, size_t size)
> {
> 	ssize_t ret = listxattr(path, list, size, XATTR_NOFOLLOW);
> 	if (ret < 0)
> 		return ret;
> //	if (getCreationTime(path) != NULL) {
> 		ret += sizeof CRTIME_XATTR;
> 		if (list) {
> 			if ((size_t)ret > size) {
> 				errno = ERANGE;
> 				return -1;
> 			}
> 			memcpy(list + ret - sizeof CRTIME_XATTR,
> 			       CRTIME_XATTR, sizeof CRTIME_XATTR);
> 		}
> //	}
> 	return ret;
> }
> Or would this bomb out running on MOSX with a non-HFS volume as the  
> source?  Or is there a better way to avoid this call (e.g.  
> determine the underlying filesystem)?
> Mike
> On Dec 1, 2007, at 10:45 PM, Robert DuToit wrote:
>> Hi,
>>  I've been using rsync (OSX Tiger now Leopard) to backup my home  
>> folder daily using -a -H -A -X link-dest=dir to make incremental  
>> backups. There was a problem though since many files especially  
>> images, movies etc would be recopied each time instead of creating  
>> hard links. I have been testing the pre5 release and found that it  
>> seems to make hard links correctly for all files. I am hoping  
>> rsync 3.0 can replace the Apple version which has been so flawed.
>> I tried the osx-create-time.diff patch too and it works but it  
>> took twice as long to copy my home folder as without and ground to  
>> a halt the last time. I know the creation date issue is somewhat  
>> "fringe" for rsync but it does matter to a lot of OSX folk so I  
>> don't know if there is any way to speed it up. I've made some  
>> small backup wrapper applications for people and they always want  
>> the creation date.....
>> I noticed the rsync version that is used now in Carbon Copy Cloner  
>> is pretty "clean" with meatdata and saves the creation date and is  
>> very fast.....  Just some thoughts-I don't have any experience  
>> with this code so can't help in that way.  Thanks, Rob--
>> To unsubscribe or change options: 
>> listinfo/rsync
>> Before posting, read: 
>> questions.html
> -- 
> To unsubscribe or change options: 
> listinfo/rsync
> Before posting, read: 
> questions.html

-------------- next part --------------
HTML attachment scrubbed and removed

More information about the rsync mailing list