Rsync / ssh high cpu load

[Jan Alphenaar]

Hi,

Today I did another interesting test in an attempt to resolve this issue.
With Microsoft Process Explorer (downloadable from the Microsoft website) it
is possible to 'pause' and 'resume' processes.

When I start a rsync/ssh session to my server and pause the SSH process. The
rsync process consumes 100% CPU, while the IO is 0. In the case I pause
rsync and leave SSH running, there is no CPU load and no IO.

In the first case rsync cannot send any data into the SSH tunnel. I think
rsync goes into a CPU consuming loop while it waits for access to the SSH
tunnel. This is somehow the same as when rsync tries to send data with SSH
and it has to wait because it is exceeding the bwlimit variable.

The next step for me would be to dive into the rsync code, which is
something I really try to avoid at this moment.

Any volunteers ?

Regards,

Jan

-----Oorspronkelijk bericht-----
Van: Jan Alphenaar [mailto:jan.alphenaar at dotcolour.com] 
Verzonden: woensdag 17 februari 2010 23:17
Aan: 'rsync at lists.samba.org'
Onderwerp: RE: Rsync / ssh high cpu load

Hi again,

Still working on this one. When I rsync/ssh a big file to my server over the
local network (100Mbit/s) with the --bwlimit=1000 I only have a small CPU
load. SSH takes about 10% and rsync approximately 0-5%. In the end the
transfer speed usually is around the value I set in bwlimit.

When I transfer the same file to a remote machine, where the bandwidth is
limited to roughly 140KB/s and the bandwidth limit is set to something below
this value I get around the same values which I have on the local network,
SSH 10% and rsync somewhere between 0-5%.

If I change the bwlimit value to 150KB/s or higher the SSH load goes up to
70% and rsync to 30% fully consuming my CPU.

I think the problem is pretty clear now. Does anybody have an idea, what the
next step could be to the resolution of this ?

Regards,

Jan

-----Oorspronkelijk bericht-----
Van: Jan Alphenaar [mailto:jan.alphenaar at dotcolour.com] 
Verzonden: woensdag 17 februari 2010 12:17
Aan: 'rsync at lists.samba.org'
Onderwerp: RE: Rsync / ssh high cpu load

Hi,

In an attempt to narrow this down a bit I installed a RedHat machine with
openssh-5.3p1 and rsync-3.0.7 and did the same test as below.

Transferring data, both up and downstream, are not giving me any CPU load.

Although there is a small version difference in the rsync versions, I would
say that this is caused by the cygwin layer.

I will mention this in the cygwin list.

Still, if anybody has any ideas how to solve the issue below, please let me
know.

Warm regards,

Jan

-----Oorspronkelijk bericht-----
Van: Jan Alphenaar [mailto:jan.alphenaar at dotcolour.com] 
Verzonden: dinsdag 16 februari 2010 17:31
Aan: 'rsync at lists.samba.org'
Onderwerp: Rsync / ssh high cpu load

Hi everybody,

I am using Cygwin rsync/ssh to synchronize my files from my home computer to
my server. I raised a question on the Cygwin list as well, but thought this
list would be appropriate too.

The issue is that the CPU on the sending side is fully allocated by
rsync/ssh (both taking 50%) during a file transfer. This is the command I
use:

	rsync -e ssh * <user>@<server>:/data

After some more testing on this I figured out that rsync/ssh is only taking
the full cpu if the bandwidth allocated to the session exceeds the true
bandwidth available on the link.

For example, if the uplink is 128KB/s and I pass in the --bwlimit=24
parameter the cpu is around 0%. Even --bw-limit=126 does not take 100% cpu,
but --bwlimit=132 does. Note this is the first install, the /data directory
on the receiving side is empty.

Would this be because of packet loss on the link and packets have to be
retransmitted ? I will do a quick test tonight with rsync/ssh between two
linux boxes (without cygwin), see what that gives me.

Note that I am using these versions:

Cygwin: 1.7.1-1
ssh: OpenSSH_5.3p1
rsync: 3.0.6

Has anybody seen this before ?

Thanks in advance.

Regards,

Jan