[ccache] ccache-like tool

[Joel Rosdahl]

On 2010-07-30 05:13, David Boyce wrote:
> I recently released a GPL tool which is, in part, an alternative to
> ccache and thought I'd drop a line to this list for anyone interested.

I've read a little about AO before, but I got the impression that it
wasn't open source so I didn't investigate it further at the time.
Thanks for releasing it! It seems very cool.

> The front page is at http://audited-objects.sourceforge.net/

I've added a link to Audited Objects under the "Related projects" header
on http://ccache.samba.org.

> and a comparison with ccache is at
> http://audited-objects.sourceforge.net/html/man/ao-ccache.html. As a
> matter of fact I haven't used ccache seriously in quite a few years,
> since I've been developing my own tool, so I'd appreciate any
> corrections to the comparison document.

Here are some corrections and other comments:

    "ccache is a compiler cache. It acts as a caching pre-processor to
    C/C++ compilers, using the -E compiler switch and a hash to detect
    when a compilation can be satisfied from cache. This often results
    in a 5 to 10 times speedup in common compilations."

This is no longer true, since ccache doesn't always run the preprocessor
anymore (see http://ccache.samba.org/manual.html#_how_ccache_works). I
suggest that you use the current top-level summary instead: "ccache is a
compiler cache. It speeds up recompilation by caching previous
compilations and detecting when the same compilation is being done
again. Supported languages are C, C++, Objective-C and Objective-C++."

    "Note that for simplicity we sometimes refer to CCACHE as a C
    compiler speedup when it really works with C or C++."

Minor thing: Objective-C and Objective-C++ are also supported now.

    "CCACHE is a personal tool; it keeps its cache in the user's home
    directory. Thus each user must build up and maintain his/her cache
    without sharing."

This is the default, but as you mention further below, it's possible to
configure ccache with a shared cache, so the text above is misleading.

    "Update: this is apparently no longer true if it ever was, since the
    ccache website says nowadays 'You can also share the cache between
    users'. I do not know how recent"

At least since version 2.4, released 2004-09-13.

    "or robust this is."

On a multi-user machine with a local ccache directory, I'd say that it's
as robust as for single-user usage. On NFS, probably less robust.

    "AO has 'selfish' and 'martyr' modes [...] Update - it appears that
    CCACHE has analogues here - CCACHE_READONLY is selfish and
    CCACHE_RECACHE is altruistic."

Yes.

    "According to the blurb above, CCACHE typically provides a 5-10
    times speedup. AO has been observed to speed up some builds by much
    more. Presumably, at least part of the difference comes from the
    fact that AO can cache and replay the linking phase (as well as any
    other tools run by the build) while CCACHE optimizes only C
    compilations."

That's still true. I think it would be quite easy for ccache to support
linking as well, but I've always thought that it rarely would be useful
since the cache hit rate for linking typically will be low. Do you have
any gut feeling or statistics about this -- won't you just fill up the
cache with libraries/executables that seldom will be reused?

    "AO imposes a certain overhead on commands which it actually runs
    (as opposed to replaying). A typical penalty for the initial build
    of a piece of software might be 10%, though it is sometimes more and
    often dramatically less. CCACHE does too, because it must exec the
    compiler twice for each logical compilation, but the CCACHE overhead
    appears to be somewhat less."

ccache (on a direct mode cache miss) first runs the preprocessor and
then (on a preprocessor mode cache miss) compiles the output from the
preprocessor, so saying that it runs the compiler twice is a bit
misleading.

    "CCACHE is only useful when building objects you already have. Why
    would someone want to rebuild something they already have? Most
    commonly because they don't fully trust their build system and only
    feel safe with a clean build from scratch."

Nowadays, I think that other scenarios are at least as common as the
"make clean; make" case. Quoting from http://ccache.samba.org: "[...]
Another reason to use ccache is that the same cache is used for builds
in different directories. If you have several versions or branches of a
software stored in different directories, many of the object files in a
build directory can probably be taken from the cache even if they were
compiled for another version or branch. A third scenario is using ccache
to speed up clean builds performed by servers or build farms that
regularly check that the code is buildable."

Regarding "ACTUAL TIMINGS", I think you should mention what ccache
version you used. Compared to earlier versions, ccache 3.0 and higher is
faster for the second pass and slightly slower for the first pass. See
http://ccache.samba.org/performance.html.

One case where I think that ccache actually is more robust than AO is
when an object file should change even if no files have changed. For
instance, the expansion of the preprocessor macros __TIME__, __DATE__
and __FILE__ change independently of the source code files. Also, when
compiling with GCC and the -g option, an absolute path to the source
file is embedded in the object file, which means that if you just cache
the object file, the wrong path is included in the object file if it's
compiled from source located in another directory.

-- Joel