[distcc] Re: Re: re: distributed compiler cache

[MartinPool]

On 12 Sep 2002, Joerg Beyer <j.beyer at web.de> wrote:

> >  2. Also, test ccache being called by the server to provide a
> >     server-side cache. 
> 
> when you talk about the server, is this what I called "master"?

Not quite.  I think every volunteer, which does the work of
compilation, might have an on-disk cache.  Whenever it is asked to
compile a file, it will first look in its cache.

What I'm proposing is to first check a local cache, and then possibly
distribute the work to another randomly chosen machine.  That machine
can satisfy the request from cache if possible, or otherwise it will
compile it.

This is effectively the same as 

  CC='ccache distcc ccache'

but possibly there are some rough edges or inefficiencies in doing
exactly that.

You can imagine, for example, broadcasting the hash of the request to
all machines, to check if it is any cache.  This would be similar to
the ICP (?) protocol used by Squid.  But really I am not sure if it's
necessary: compiling any single file only takes a second or two, so to
be of benefit any distributed protocol must be very lightweight.

You described a scenario of a group of programmers with one machine
each, all working on similar versions of the source.  This scheme will
converge towards having all relevant source files cached on every
machine.

> This scenario still needs a central component that handles the access
> to the on disk cache (that thing, that I called master), ok?

I don't understand what your master needs to do that can't be just
done by the existing ccache program.  It can handle concurrent calls
using appropriate locking.  It won't make any difference whether those
calls are local, or coming across the network via distccd.

(Please write to the mailing list so that other people can contribute
too.) 

-- 
Martin