[ccache] direct mode

[Andrew Stubbs]

On 20/10/13 10:18, Joel Rosdahl wrote:
>> bug?
>
> Yes, see the discussion on
> http://www.mail-archive.com/ccache@lists.samba.org/msg00920.html.
>
> By the way: I'm still torn on what to do, but I'm leaning towards keeping
> direct mode on by default (documenting the behavior, of course).

I've thought about this a little, since our last exchange.

I like Christian Lohmaier's suggestion of a "safe-direct mode". Or 
rather, something like it: I'd suggest making the existing direct mode, 
the default, safe and then have a "fast-mode" flag that short-cuts back 
to the current state. The "CCACHE_FAST_MODE" would be documented in the 
same spirit as CCACHE_HARDLINK, and CCACHE_SLOPPINESS; that is, with 
lots of caveats.

Here's my idea how to make it safe:

1. When running the preprocessor, on cache-miss, use -v the capture the 
compiler's search path. This ought to be almost zero extra cost.

2. For each header file found in the preprocessed source, record the 
directories in which the compiler did *not* find that file. There should 
be no need to stat anything; it ought to be obvious from reading the 
path name where the compiler found it.

3. Store those paths in the manifest file.

4. On cache-lookup, load the manifest file, and do 
"access(potential_include_file, R_OK)" for each include path listed in 
the manifest. If any call comes back non-zero then skip direct mode.

For example, lets say that the include search patch is this:

  /usr/lib/gcc/x86_64-linux-gnu/4.8/include
  /usr/local/include
  /usr/lib/gcc/x86_64-linux-gnu/4.8/include-fixed
  /usr/include/x86_64-linux-gnu
  /usr/include

And the source contains a reference to "/usr/include/stdio.h".

We can see that "stdio.h" was not found in any of the first four include 
paths, so we encode that data into the manifest.

On lookup, ccache would only have to do four extra syscalls to make that 
header "safe". (Possibly the CWD would also have to be checked.)


This scheme does not allow for the case where the compiler's search path 
has been changed, but the "gcc" binary has not, but it should certainly 
catch the cases where the include directories have been updated, or the 
user's own -I include paths have new content.

In terms of false negatives, it's hard to tell the difference between "" 
and <> includes (in which the search path starts in a different place), 
and it'll be suboptimal where the user's source contains explicit paths 
to header files.

Andrew