DO NOT REPLY [Bug 5324] Reduce the performance penalty of --xattrs
on Mac OS X
samba-bugs at samba.org
samba-bugs at samba.org
Sun Jun 21 02:53:53 GMT 2009
https://bugzilla.samba.org/show_bug.cgi?id=5324
mike at bombich.com changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |mike at bombich.com
------- Comment #4 from mike at bombich.com 2009-06-20 21:53 CST -------
Mac OS X makes extensive use of xattrs, and I've seen hundreds of thousands of
unique xattrs on several end-user systems. In those cases rsync eventually
runs out of memory and bails.
rsync should probably stop calling find_matching_xattr when the list reaches a
specified size. And at least for Mac OS X, xattrs shouldn't be cached in
rsync_xal_l, they should probably be compared on the fly by the generator
somewhere near generate_and_send_sums.
Also, to address Fabrice's concern, performance will ultimately be directly
linked to the lack of a quick heuristic for determining whether xattrs have
been modified. xattrs don't have modifications dates, so you can use either
the size or a checksum to determine if they've changed. In my analysis, rsync
spent 20% of its time in md5_process on a task involving many xattrs with lots
of data. Unless we come up with something faster than md5 (potentially at the
cost of reliability), this is just a performance hit we'll have to live with.
I'm going to take a stab at this, but I'm curious whether you've put any more
thought into how xattr support is implemented since your last comment here.
Thanks!
Mike Bombich
--
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug, or are watching the QA contact.
More information about the rsync
mailing list