[distcc] Benchmarking the server-side cache. How bad is 4ms
latency to distcc server?
Martin Pool
mbp at sourcefrog.net
Mon Feb 6 07:26:20 GMT 2006
On 5 Feb 2006, Dan Kegel <dank at kegel.com> wrote:
> On 2/5/06, Jake McGuire <jake at boom.net> wrote:
> > On Feb 5, 2006, at 2:44 PM, Dan Kegel wrote:
> > This doesn't sound like a bad idea to me (the protocol change, not
> > the compression). I don't think that many people are going to be
> > running different versions of distcc/distccd to make backwards
> > compatibility a problem - the rpm builds both of them - and even if
> > they are, making the hash message optional seems like an obvious choice.
>
> Yeah. And I think I'm going to do it without adding an extra
> round trip, too. The client can, if it wants, give the hash of the
> sources just before it sends the actual source.
> The server, when it recieves that, will reply *early* with
> the compiled files rather than waiting for the client to finish sending
> the source. Yeah, it's evil because the client ends up sending more
> than it really had to, but if the server's quick, the client won't have
> sent more than a little bit of the file before it realizes it can stop.
> And it will unambiguously reduce network traffic from large files without
> hurting short ones.
That's an interesting idea.
You could combine it with the (currently disabled?) code that feeds the
compiler from a fifo, so as to stream back errors or terminate
compilation in the case of errors before the client finishes sending.
However I suppose failing files are fairly rare, since they will
generally terminate the build, and so not worth optimizing.
This will probably require changing the client to a nonblocking setup
where it can check for traffic back from the server while it keeps
sending. This is certainly possible, and will complicate it a bit but
perhaps not too much. This may not work if you use sendfile -- you
might have to sendfile in several chunks or something.
Incidentally, Van Jacobson's slides from linux.conf.au might be
of interest to you. It was a fascinating talk.
http://www.lemis.com/grog/Documentation/vj/lca06vj.pdf
http://vger.kernel.org/~davem/cgi-bin/blog.cgi/2006/01/27#vj_channels
Will sending the hash first really be a problem? I would have thought
an extra round trip would be relatively cheap compared to the 2MB
transmission, but I'm not sure.
--
Martin
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
Url : http://lists.samba.org/archive/distcc/attachments/20060206/54af4988/attachment-0001.bin
More information about the distcc
mailing list