[Bug 3099] Please parallelize filesystem scan
Ken Chase
rsync-list-m829 at sizone.org
Fri Jul 17 14:51:27 UTC 2015
I dont understand - scanning metadata is sped up by thrashing the head
all over the disk instead of mostly-sequentially scanning through?
How does that work out?
/kc
On Fri, Jul 17, 2015 at 02:37:21PM +0000, samba-bugs at samba.org said:
>https://bugzilla.samba.org/show_bug.cgi?id=3099
>
>--- Comment #8 from Chip Schweiss <chip at innovates.com> ---
>I would argue that optionally all directory scanning should be made parallel.
>Modern file systems perform best when request queues are kept full. The
>current mode of rsync scanning directories does nothing to take advantage of
>this.
>
>I currently use scripts to split a couple dozen or so rsync jobs in to
>literally 100's of jobs. This reduces execution time from what would be days
>to a couple hours every night. There are lots of scripts like this appearing
>on the net because the current state of rsync is inadequate.
>
>This ticket could reasonably combined with 5124.
>
>--
>You are receiving this mail because:
>You are the QA Contact for the bug.
>
>--
>Please use reply-all for most replies to avoid omitting the mailing list.
>To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
>Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
--
Ken Chase - ken at heavycomputing.ca skype:kenchase23 Toronto Canada
Heavy Computing - Clued bandwidth, colocation and managed linux VPS @151 Front St. W.
More information about the rsync
mailing list