RFC: slow-down option

Marian Marinov mm at yuhu.biz
Thu Apr 3 08:47:27 MDT 2014


On 04/03/2014 03:35 PM, Christoph Biedl wrote:
> Joe wrote...
>
>> This is way beyond my level of expertise, but wouldn't something like
>> ionice help with that?
>
> Although I'm not Marian, probably not. The ionice program does a
> reasonable good job when it's about prioritizing read operation. The
> context makes me guess it's rather about writing.

We were using ionice and it did not help. The problem is that if for some reason all of your concurrent rsyncs are 
running with the same ionice level and class, even thou they are ioniced, effectively there is no change.
So at that point we wrote a simple perl daemon that monitored rsyncs and changed ionice levels based on the amount of 
time each rsync spent in each level.
We pushed all new rsyncs onto the lowest level and dynamically moved them to the top. It simply does not work as well as 
I imagined.

The best solution was to use the bklio control-group, BUT, with it the backups were still slower then with the slow-down 
option.

Marian

>
>> Also, check out:
>>
>> 2 more pipe utilities
>>
>> Viewer & throttle
>> http://www.ivarch.com/programs/pv.shtml
>>
>> Throttle - limits bandwidth of a pipe - for use with network transfers
>> http://linux.die.net/man/1/throttle
>
> The pv utility is way to little known and served me well in many
> situations, and throttle seems to do quite the same. Both however seem
> to do quite the same thing rsync's --bwlimit option does, while the
> latter is more sophisticated.
>
> It the issue is the one I have in mind (which is one I constantly
> suffer from), the actual problem is the dirty buffer writeback
> strategy, deep in the Linux kernel.
>
>      Christoph
>



More information about the rsync mailing list