[Bug 7120] Variable bandwidth limit .. bwlimit
lars.ellenberg at linbit.com
Thu Mar 23 16:00:24 UTC 2017
On Wed, Mar 01, 2017 at 05:38:15PM +0000, samba-bugs at samba.org wrote:
> --- Comment #7 from roland <devzero at web.de> ---
> please mind the note from this posting when using pv:
> It seems that pv is waiting for data from rsync, and rsync is waiting for data
> too (stuck in select()) and not closing the input to pv. So it's a deadlock.
> Same happens when you substitute pv with something else (like dd). It seems
> that those commands just don't behave like rsync expects them to.
> Haven't found a workaround short of killing everything:
> export RSYNC_RSH="sh -c 'pv -qL10k | ssh \"\$@\" | (pv -qL11k; kill \$\$)' ssh"
The trick is to avoid any remaining parent- or sub-shell,
achieved by using exec (the inner execs are not always needed,
that depends on the bash version. but they don't hurt, either).
this runs both pv on the local box:
RSYNC_RSH="bash -xc '\
exec 3< <(exec pv -L10k); \
exec 4< <(exec \"\$@\" <&3 ) 3<&-; \
exec pv -L11k <&4 4<&-' -- ssh"
feel free to add -N IN -c, -N OUT -c, or similar...
Or runs outgoing limit on local box,
incomming limit on remote box
(each box limits the stream that leaves it):
O_LIM="bash -c 'exec 3< <(exec pv -L10k); exec \"\$@\" <&3 3<&-' -- ssh"
I_LIM="bash -c 'exec 3< <(exec \"\$@\"); exec pv -L11k <&3 3<&-' -- rsync"
rsync --rsh "$O_LIM" --rsync-path "$I_LIM" ...
Or the other way around,
if you want each box limit their respective incoming stream instead.
More information about the rsync