Rsync / ssh high cpu load

Thu Dec 6 20:29:41 MST 2012

Hi,

ssh: OpenSSH_5.3p1
rsync: 3.0.6
OS: CentOS release 6.3

I got the same problem here, syncing 95M to amazon ec2 without
--bwlimit parameter. It should be not caused by cygwin. From 3.0.6 to
latest version, I did not find any ssh related problem fixed.

After the syncing job is over, ssh process did not quit and consumes
100% CPU. I collected other information, wish it may help.

related process:
root     29296  0.0  0.0 107612   964 pts/1    T    Dec06   0:00 rsync
-arv -e ssh data root at xxx.com:/tmp/data/ --exclude=.svn
root     29297 99.9  0.0  59760  3476 pts/1    R    Dec06 770:20 ssh
-l root xxx.com rsync --server -vlogDtpre.iLs . /tmp/data/
root     29306  0.0  0.0  55160  2028 ?        Ss   Dec06   0:00 scp
-f -- /yyy/zzz.sql

# strace -fF -v -p 29297
Process 29297 attached - interrupt to quit
--- SIGTTOU (Stopped (tty output)) @ 0 (0) ---
rt_sigreturn(0x16)                      = -1 EINTR (Interrupted system call)
ioctl(4, SNDCTL_TMR_CONTINUE or TCSETSF, {c_iflags=0x500,
c_oflags=0x5, c_cflags=0xbf, c_lflags=0x8a3b, c_line=0,
c_cc="\x03\x1c\x7f\x15\x04\x00\x01\x00\x11\x13\x1a\x00\x12\x0f\x17\x16\x00\x00\x00"})
= ? ERESTARTSYS (To be restarted)
--- SIGTTOU (Stopped (tty output)) @ 0 (0) ---
rt_sigreturn(0x16)                      = -1 EINTR (Interrupted system call)
ioctl(4, SNDCTL_TMR_CONTINUE or TCSETSF, {c_iflags=0x500,
c_oflags=0x5, c_cflags=0xbf, c_lflags=0x8a3b, c_line=0,
c_cc="\x03\x1c\x7f\x15\x04\x00\x01\x00\x11\x13\x1a\x00\x12\x0f\x17\x16\x00\x00\x00"})
= ? ERESTARTSYS (To be restarted)
--- SIGTTOU (Stopped (tty output)) @ 0 (0) ---
rt_sigreturn(0x16)                      = -1 EINTR (Interrupted system call)
(repeat)

-----Oorspronkelijk bericht-----
Van: Jan Alphenaar [mailto:jan.alphenaar at dotcolour.com]
Verzonden: woensdag 17 februari 2010 23:17
Aan: 'rsync at lists.samba.org'
Onderwerp: RE: Rsync / ssh high cpu load

Hi again,

Still working on this one. When I rsync/ssh a big file to my server over the
local network (100Mbit/s) with the --bwlimit=1000 I only have a small CPU
load. SSH takes about 10% and rsync approximately 0-5%. In the end the
transfer speed usually is around the value I set in bwlimit.

When I transfer the same file to a remote machine, where the bandwidth is
limited to roughly 140KB/s and the bandwidth limit is set to something below
this value I get around the same values which I have on the local network,
SSH 10% and rsync somewhere between 0-5%.

If I change the bwlimit value to 150KB/s or higher the SSH load goes up to
70% and rsync to 30% fully consuming my CPU.

I think the problem is pretty clear now. Does anybody have an idea, what the
next step could be to the resolution of this ?

Regards,

Jan

-----Oorspronkelijk bericht-----
Van: Jan Alphenaar [mailto:jan.alphenaar at dotcolour.com]
Verzonden: woensdag 17 februari 2010 12:17
Aan: 'rsync at lists.samba.org'
Onderwerp: RE: Rsync / ssh high cpu load

Hi,

In an attempt to narrow this down a bit I installed a RedHat machine with
openssh-5.3p1 and rsync-3.0.7 and did the same test as below.

Transferring data, both up and downstream, are not giving me any CPU load.

Although there is a small version difference in the rsync versions, I would
say that this is caused by the cygwin layer.

I will mention this in the cygwin list.

Still, if anybody has any ideas how to solve the issue below, please let me
know.

Warm regards,

Jan

-----Oorspronkelijk bericht-----
Van: Jan Alphenaar [mailto:jan.alphenaar at dotcolour.com]
Verzonden: dinsdag 16 februari 2010 17:31
Aan: 'rsync at lists.samba.org'
Onderwerp: Rsync / ssh high cpu load

Hi everybody,

I am using Cygwin rsync/ssh to synchronize my files from my home computer to
my server. I raised a question on the Cygwin list as well, but thought this
list would be appropriate too.

The issue is that the CPU on the sending side is fully allocated by
rsync/ssh (both taking 50%) during a file transfer. This is the command I
use:

        rsync -e ssh * <user>@<server>:/data

After some more testing on this I figured out that rsync/ssh is only taking
the full cpu if the bandwidth allocated to the session exceeds the true
bandwidth available on the link.

For example, if the uplink is 128KB/s and I pass in the --bwlimit=24
parameter the cpu is around 0%. Even --bw-limit=126 does not take 100% cpu,
but --bwlimit=132 does. Note this is the first install, the /data directory
on the receiving side is empty.

Would this be because of packet loss on the link and packets have to be
retransmitted ? I will do a quick test tonight with rsync/ssh between two
linux boxes (without cygwin), see what that gives me.

Note that I am using these versions:

Cygwin: 1.7.1-1
ssh: OpenSSH_5.3p1
rsync: 3.0.6

Has anybody seen this before ?

Thanks in advance.

Regards,

Jan

--
Phone: 18601240631
Blog: http://liruqi.info
Weibo: http://weibo.com/liruqi

-- 
Phone: 18601240631
Blog: http://liruqi.info
Weibo: http://weibo.com/liruqi