[ccache] Using ccache with memcached
wsnyder at wsnyder.org
Thu Nov 25 03:17:47 MST 2010
>On 25 November 2010 00:52, Henrik Goldman <hg at x-formation.com> wrote:
>> I wonder if anyone ever considered using memcached as an optional backend
>> for ccached?
>> Here is the issue we're considering. In our C++ project we have a number of
>> developers doing their own local compilations. When any updates occurs (e.g.
>> they update to the latest source code from scm) the compilations are the
>> same for all the machines. Sure ccache is great when you re-do the
>> compilation, but here the compilation being re-done on a number of machines.
>> Using memcached would allow us to have a truly distributed cache that would
>> both help developers and machines in the build farm to speed up work.
>> What do you guys think?
>I think it would be a pretty interesting idea.
>Obviously you would need to trust all the other clients writing into
>the cache, but that's probably feasible in this environment.
>A shared cache kept entirely in ram might grow fairly big, but it's
>not so expensive to get an 8GB or 16GB machine devoted to this.
Why don't you just put the cache on a NFS (/CIFS) mounted
volume? With the most recent version this should work well.
If you already are, are you really doing enough writes to
swamp a NFS cache server? It probably requires hundreds of
compiling clients; since we have over a hundred here and
don't see a bottleneck - with a single well performing NFS
If you need more performance you can get a better NFS server
appliance (up to thousands of disks if you have the $$), or
on the cheap by striding the cache across 16 (or 256) small
You can also serve NFS from a ram disk via loopback,
although if you give a machine enough memory, it's going to
keep a normal file system in RAM anyways, so using a ramdisk
primarily provides you with write bandwidth, not read
bandwidth nor latency.
Memcached would provide a nice benefit of providing
tolerance for machines going down, and somewhat better
latency, but perhaps the above ideas with the existing
version can deliver enough performance for you.
More information about the ccache