[distcc] SSH encryption overhead

[Aaron Lehmann]

> However, ssh encryption is pretty expensive. Rough measurements show
> that transferring a file across ssh takes CPU time comparable to
> compiling it, which would make it fairly impractical for distcc.

You must be using 3des...

[aaronl at vitelus:~/openssl-0.9.6g]$ cat /proc/cpuinfo
processor       : 0
vendor_id       : GenuineIntel
cpu family      : 6
model           : 8
model name      : Celeron (Coppermine)
stepping        : 10
cpu MHz         : 1097.099
cache size      : 128 KB
fdiv_bug        : no
hlt_bug         : no
f00f_bug        : no
coma_bug        : no
fpu             : yes
fpu_exception   : yes
cpuid level     : 2
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 mmx fxsr sse
bogomips        : 2188.90
[aaronl at vitelus:~/openssl-0.9.6g]$ ./apps/openssl speed des-ede3
Doing des ede3 for 3s on 8 size blocks: 3257959 des ede3's in 3.00s
Doing des ede3 for 3s on 64 size blocks: 424293 des ede3's in 3.00s
Doing des ede3 for 3s on 256 size blocks: 106682 des ede3's in 3.00s
Doing des ede3 for 3s on 1024 size blocks: 26712 des ede3's in 3.00s
Doing des ede3 for 3s on 8192 size blocks: 3338 des ede3's in 3.00s
OpenSSL 0.9.6g 9 Aug 2002
built on: Fri Sep  6 15:24:07 PDT 2002
options:bn(64,32) md2(int) rc4(idx,int) des(ptr,risc1,16,long) idea(int) blowfish(idx) 
compiler: gcc -DTHREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -DL_ENDIAN -DTERMIO -O3 -fomit-frame-pointer -m486 -Wall -DSHA1_ASM -DMD5_ASM -DRMD160_ASM
The 'numbers' are in 1000s of bytes per second processed.
type              8 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes
des ede3          8687.89k     9051.58k     9103.53k     9117.70k     9114.97k
[aaronl at vitelus:~/openssl-0.9.6g]$ ./apps/openssl speed bf-cbc
Doing blowfish cbc for 3s on 8 size blocks: 12446868 blowfish cbc's in 3.00s
Doing blowfish cbc for 3s on 64 size blocks: 1787532 blowfish cbc's in 3.00s
Doing blowfish cbc for 3s on 256 size blocks: 459200 blowfish cbc's in 3.00s
Doing blowfish cbc for 3s on 1024 size blocks: 115553 blowfish cbc's in 3.00s
Doing blowfish cbc for 3s on 8192 size blocks: 14450 blowfish cbc's in 3.00s
OpenSSL 0.9.6g 9 Aug 2002
built on: Fri Sep  6 15:24:07 PDT 2002
options:bn(64,32) md2(int) rc4(idx,int) des(ptr,risc1,16,long) idea(int) blowfish(idx) 
compiler: gcc -DTHREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -DL_ENDIAN -DTERMIO -O3 -fomit-frame-pointer -m486 -Wall -DSHA1_ASM -DMD5_ASM -DRMD160_ASM
The 'numbers' are in 1000s of bytes per second processed.
type              8 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes
blowfish cbc     33191.65k    38134.02k    39185.07k    39442.09k    39458.13k

These results are directly applicable to SSH since OpenSSH uses
OpenSSL's cryptographic primitives. 40MB/s is way faster than I can send
to my network, not to mention quite a bit faster than GCC. I don't deny
that there may be further overhead in  SSH protocol, including hashing
(although OpenSSL claims a rate of 85996.82k/s for SHA-1 on my modest
machine). I don't think that very much inefficiency beyond encryption
and hashing is mandated by the protocols, so ssh's speed problems are
most likely bugs in OpenSSH that could be addressed with patches much
more easily and charitably then writing an alternative secure
authentication system just for distcc.

I was pondering recommending a compromise where distcc would use ssh to
transmit commands, but actual transfer of data would occur over
unsecured sockets. The benefit is that this system removes the "any
command can be run through a simple network connection" issue. I've come
to think that this is a bad idea, because if you're on an untrusted
network you can't trust unsecured data transfer anyway. Source code
and the resulting binary aren't usually very private, but SSH also
protects the integrity of data that travels over it. Without some
cryptographic integrity verification, a malicious user could modify
either the code on its way to compilation or the object code on the way
back to the client. This would be a great way to break into any computer
which runs the binary that distcc orchestrated the compilation of.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url : http://lists.samba.org/archive/distcc/attachments/20020919/e96ce3ff/attachment.bin