[distcc] Working on several distcc enhancements (take 2)

[Dobes Vandermeer]

On 11/27/05, Laurent Calburtin <laurent.calburtin at free.fr> wrote:

> Hi,
>
> I'm very interested about combining ccache and distcc. I think that would
> make a huge performance improvement in our situation where many developers
> are compiling each day roughly the same set of files.
>
> But installing ccache on each distccd host means as many separate caches.
> Since filling a cache has a price (compiling!), I would prefer filling as
> few caches as possible with the same content. I know that some are using a
> file server to share the cache but that means network communications and we
> may avoid that.
>

ccache *does* avoid network communications this way, however: on the client,
it simply sends a file open request (which is much smaller than the
pre-processed output) to the file server, and gets back a file-opened or
file-not-found result.  The only really additional network overhead
is uploading to the cache by the machines compiling, which isn't a big deal.

If your network is 100MB, and you've got a reasonable performance switch and
high performance file server, network overhead on using a network cache this
way isn't particularly noticeable compared the gains in speed by grabbing
files from the cache that were compiled by other people.

As for server selection schemes, I suspect it will be worthwhile to ensure
that your load balancing avoids race conditions - it seems likely that the
server with the highest score at any given moment might find itself swamped
by several eager clients, especially as you get more programmers
and continuous integration/build-bot processes.
-------------- next part --------------
HTML attachment scrubbed and removed