[distcc] Benchmarking the server-side cache. How bad is 4ms latency to distcc server?

[Martin Pool]

On  5 Feb 2006, Dan Kegel <dank at kegel.com> wrote:
> On 2/5/06, Jake McGuire <jake at boom.net> wrote:
> > On Feb 5, 2006, at 2:44 PM, Dan Kegel wrote:

> > This doesn't sound like a bad idea to me (the protocol change, not
> > the compression).  I don't think that many people are going to be
> > running different versions of distcc/distccd to make backwards
> > compatibility a problem - the rpm builds both of them - and even if
> > they are, making the hash message optional seems like an obvious choice.
> 
> Yeah.  And I think I'm going to do it without adding an extra
> round trip, too.  The client can, if it wants, give the hash of the
> sources just before it sends the actual source.
> The server, when it recieves that, will reply *early* with
> the compiled files rather than waiting for the client to finish sending
> the source.   Yeah, it's evil because the client ends up sending more
> than it really had to, but if the server's quick, the client won't have
> sent more than a little bit of the file before it realizes it can stop.
> And it will unambiguously reduce network traffic from large files without
> hurting short ones.

That's an interesting idea.  

You could combine it with the (currently disabled?) code that feeds the
compiler from a fifo, so as to stream back errors or terminate
compilation in the case of errors before the client finishes sending.
However I suppose failing files are fairly rare, since they will
generally terminate the build, and so not worth optimizing.

This will probably require changing the client to a nonblocking setup
where it can check for traffic back from the server while it keeps
sending.   This is certainly possible, and will complicate it a bit but
perhaps not too much.  This may not work if you use sendfile -- you
might have to sendfile in several chunks or something.

Incidentally, Van Jacobson's slides from linux.conf.au might be
of interest to you.  It was a fascinating talk.

  http://www.lemis.com/grog/Documentation/vj/lca06vj.pdf
  http://vger.kernel.org/~davem/cgi-bin/blog.cgi/2006/01/27#vj_channels

Will sending the hash first really be a problem?  I would have thought
an extra round trip would be relatively cheap compared to the 2MB
transmission, but I'm not sure.

-- 
Martin
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
Url : http://lists.samba.org/archive/distcc/attachments/20060206/54af4988/attachment-0001.bin