[ccache] ccache and precompiled headers

[Tor Arne Vestbø]

Hey!

On Sun, Jun 27, 2010 at 2:15 PM, Joel Rosdahl <joel at rosdahl.net> wrote:
> Hi,
>
> On 2010-06-21 13:09, Tor Arne Vestbř wrote:
>> I ran into a problem where ccache would log "Preprocessor gave exit
>> status 1" and fail to cache builds.
>>
>> The solution seems to be to add the -fpch-preprocess flag, which will
>> add a custom #pragma include to the preprocessed output,
>
> I came to mostly the same conclusion some months ago but decided to
> postpone thinking about support for preprocessed headers until after the
> 3.0 release. So, now's a good time to continue. :-)

Makes sense. Love the speed of the direct mode btw :)

> See comments below.
>
>> [...]
>> @@ -1557,6 +1562,12 @@ static void process_args(int argc, char **argv, ARGS **preprocessor_args,
>>                                         }
>>
>>                                         args_add(stripped_args, argv[i]);
>> +                                       if (strcmp(argv[i], "-include")
>> +                                                       && strstr(argv[i+1], "pch")) {
>> +                                               found_pch = 1;
>> +                                               cc_log("Found precompiled header: %s",
>> +                                                      argv[i+1]);
>> +                                       }
>
> But this means that you have to include "pch" in name of the precompiled
> header file, which is quite limiting. There's also the case when using
> '#include "preprocessed.h"' in the source code to include the
> precompiled header instead of using the -include compiler option. Also,
> if the name contains "pch" when it's not a precompiled header,
> -fpch-preprocess will be added anyway, and that will make the
> preprocessor fail if the compiler isn't GCC 4.0 or higher.

Yupp, that particular part of the patch was just to try things out for
my usecase where the files had "pch" in them. (also there's a bug in
the strcmp, which should do == 0)

> I don't see how ccache in a good way could figure out if the
> -fpch-preprocess option should be added or not, so the easy option is
> this: Just document that if you want ccache caching to work with
> precompiled headers, you have make sure to add the -fpch-preprocess
> option to the compilation command. That may be done by altering the
> build system (adding -fpch-preprocess to CPPFLAGS or similar), but it
> could also be done by adding -fpch-preprocess to CCACHE_CPPFLAGS if we
> introduce such a variable. We also need the first part of your patch, of
> course.

Right, that would be a solution. I assume GCC does not barf it passed
-fpch-preprocess in non-preprocess mode.

> Another way would be to ask the compiler if it supports the
> -fpch-preprocess option and, if so, always add it. This could be done by
> checking the status code of running something like "compiler
> -fpch-preprocess -x c -c /dev/null -o /dev/null". The result of this
> command could be cached somewhere in CCACHE_DIR (keyed with the
> compiler's mtime) so that there's no need to run it more than once.
> (Storing other types of information about the compiler may be useful for
> other things as well, like doing compiler-specific handling of flags or
> maybe enabling direct mode only for compilers known to work with it.)

If that kind of compiler-feature-inspection is on the roadmap anyways
that would indeed be an option, but I'm not sure it makes sense alone
for this particular usecase since there is another workaround.

> There's also at least one more subtle thing that needs to be addressed:
> the __TIME__ macro in direct mode. As can be read in the Performance
> section in the manual
> (http://ccache.samba.org/manual.html#_performance), the direct mode is
> turned off when "__TIME__" is present in the source code. But when
> precompiled headers are being used, ccache can't easily read the code
> that was included, so it can't tell whether __TIME__ is potentially
> being used and thus has to turn off direct mode as a safety measure
> (unless CCACHE_SLOPPY=time_macros). This also needs to be implemented.

Hmm, this one worries me a bit. How are we going to figure out if the
precompiled header contains __TIME__?

> What do you think?

I think if we can make it correct, supporting PCH would be a nice
addition. On the other hand, I did some quick build-tests with the
patch and here's what I found:

(numbers in parenthesis are for retries of the same config)


 vanlilla (with pch) ............. 10:33

- pch .................. 12:42 (12:43)
+ ccache ............... 13:18 (13:17)
 ccache 2nd clean rebuild ...  4:32 (3:53, 4:14, 4:10)

[whopper:~/dev/ccache] $ ccache -s
cache directory                     /Users/torarne/.ccache
cache hit (direct)                  7655
cache hit (preprocessed)              23
cache miss                          3816
called for link                      219
unsupported source language            6
files in cache                      7720
cache size                         974.0 Mbytes
max cache size                      30.0 Gbytes

 vanlilla (with pch) ............. 10:33
+ ccache ............... 12:25 (12:23)
 ccache 2nd clean rebuild ...  5:44 (4:56, 4:39, 5:13)


cache directory                     /Users/torarne/.ccache
cache hit (direct)                  3699
cache hit (preprocessed)            1942
cache miss                          3816
called for link                      225
preprocessor error                  2435
unsupported source language          118
files in cache                      6753
cache size                        1016.5 Mbytes
max cache size                      30.0 Gbytes

As you can see removing pch from the build increases the total build
time for the initial build (empty cache), but speeds up builds when
there's something in the cache since we're hitting direct mode in
almost all cases. When pch and ccache is used together the initial
build time decreases, _but_ builds with a filled cache now take
longer.

Seems like we're hitting a lot more preprocessed cases for some
reason. Dunno why that might be?

Tor Arne