DO NOT REPLY [Bug 5324] Reduce the performance penalty of --xattrs on Mac OS X

[samba-bugs at samba.org]

https://bugzilla.samba.org/show_bug.cgi?id=5324


mike at bombich.com changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |mike at bombich.com




------- Comment #4 from mike at bombich.com  2009-06-20 21:53 CST -------
Mac OS X makes extensive use of xattrs, and I've seen hundreds of thousands of
unique xattrs on several end-user systems.  In those cases rsync eventually
runs out of memory and bails.

rsync should probably stop calling find_matching_xattr when the list reaches a
specified size.  And at least for Mac OS X, xattrs shouldn't be cached in
rsync_xal_l, they should probably be compared on the fly by the generator
somewhere near generate_and_send_sums.

Also, to address Fabrice's concern, performance will ultimately be directly
linked to the lack of a quick heuristic for determining whether xattrs have
been modified.  xattrs don't have modifications dates, so you can use either
the size or a checksum to determine if they've changed.  In my analysis, rsync
spent 20% of its time in md5_process on a task involving many xattrs with lots
of data.  Unless we come up with something faster than md5 (potentially at the
cost of reliability), this is just a performance hit we'll have to live with.

I'm going to take a stab at this, but I'm curious whether you've put any more
thought into how xattr support is implemented since your last comment here.

Thanks!
Mike Bombich


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug, or are watching the QA contact.