[distcc] Benchmarking the server-side cache. How bad is 4ms latency to distcc server?

[Jake McGuire]

On Feb 5, 2006, at 2:44 PM, Dan Kegel wrote:
> Examining the connection between client and server
> using ping, I see that the round-trip time is usually 400
> microseconds, but sometimes 4ms; the average is
> about 1ms.

Server misconfiguration?  Or overloaded network hardware?

Is netstat -s showing noticeable packet drops?

> I guess I'll experiment with turning on compression next,
> and look at a protocol change for large files to try sending
> just the hash of the source first, and sending the full
> source only if the server replies with 'sorry, not in the cache'.

This doesn't sound like a bad idea to me (the protocol change, not  
the compression).  I don't think that many people are going to be  
running different versions of distcc/distccd to make backwards  
compatibility a problem - the rpm builds both of them - and even if  
they are, making the hash message optional seems like an obvious choice.

> But I thought I'd ask: just how awful is it to have a latency
> that alternates betwen 0.4 and 4.0 ms, with an average of 1ms?
> My guess is that it causes a significant overall penalty
> compared to a uniform 0.4ms latency, but I haven't checked.

Given the network traffic in the current distcc protocol, especially  
if DISTCC_DIR is on networked storage and you have a bunch of servers  
(i.e. many NFS/CIFS lock attempts) this is going to be bad news.

-jake