[ccache] Optimizing MD4
andrew_stubbs at mentor.com
Fri Dec 11 16:45:36 UTC 2015
On 11/12/15 16:28, Anders Björklund wrote:
>> Each invocation of ccache then connects to that socket and asks the
>> daemon to do the MD4 scan on its behalf. The daemon checks the mtime on
>> the file and serves the MD4 from its memory cache if nothing has
>> changed. The stat call could probably be optimized away if the cache is
>> very fresh (<1s?)
> So basically something similar to the "sloppiness" file_stat_matches ?
> Compare size/mtime/ctime, rather than rehashing the content of a file.
Yes, except that it *has* been checked this session. This is a
significant difference when you try to share a cache between two
machines, or two users, or just simply did some git foolery between builds.
Good point though: for a lone user doing nothing exciting,
CCACHE_SLOPPINESS=file_stat_matches does nearly the same thing with
considerably less effort.
>> You could, in principle, use this communication to limit how many
>> cache-miss compilations are permitted to run in parallel, and therefore
>> run "make -j" for maximum parallelism without fear of melting your memory.
> Something like that is what I meant with a new "prefix" wrapper for cpp.
> Similar to the current wrapper for cc, which does that with e.g. distcc.
Yes. You have to be careful with how many executables you have though:
the dynamic linker is slooooow.
>> Unfortunately, I've moved on to other projects and don't have much time
>> to work on this stuff any more.
> Thank you for your ideas. Will check the code out too if I get the time.
> It seems that there are some opportunities left for faster manifest/cpp.
If you decide to go with the persistent background task approach, then
there are a few ideas in there to pluck out, but the overall
architecture had other goals, so maybe it's not that great a starting
More information about the ccache