[ccache] ccache and precompiled headers

[Tor Arne Vestbø]

On 27.08.10 13.15, Tor Arne Vestbø wrote:
> On 22.08.10 12.42, Joel Rosdahl wrote:
>> On 2010-08-21 16:30, Joel Rosdahl wrote:
>>> I've started implementing support for preprocessed headers (.gch)
>>
>> Done.
>>
>> Tor Arne: It would be great if you could try the current dev version of
>> ccache and see if it works well for you and if you get direct mode hits
>> when you use precompiled headers.
>
> I'm sorry to report that I'm seeing build issues with master when
> building Qt on Mac OS X (but the issue should be fixable):

> I'm assuming the hash is based solely on the contents of the
> preprocessed output, not the flags/defines that were passed? But it
> seems that the defines are stored somewhere in the precompiled header,
> since it's barfing out when detecting a mismatch.

It seems we do include a few flags in the hash as well, but not -D. 
Using the following patch, with CCACHE_SLOPPINESS=time_macros set 
everything seems to work fine:

diff --git i/ccache.c w/ccache.c
index d5b292d..45359ee 100644
--- i/ccache.c
+++ w/ccache.c
@@ -189,6 +189,9 @@ static int nlevels = 2;
   */
  static int compile_preprocessed_source_code;

+/* Whether the output is a precompiled header */
+static int output_is_precompiled_header = 0;
+
  /* How long (in microseconds) to wait before breaking a stale lock. */
  unsigned lock_staleness_limit = 2000000;

@@ -1019,8 +1022,8 @@ calculate_object_hash(struct args *args, struct 
mdfour *hash, int direct_mode)
  		/* When using the preprocessor, some arguments don't contribute
  		   to the hash. The theory is that these arguments will change
  		   the output of -E if they are going to have any effect at
-		   all. */
-		if (!direct_mode) {
+		   all. For precompiled headers this might not be the case. */
+		if (!direct_mode && !output_is_precompiled_header) {
  			if (i < args->argc-1) {
  				if (str_eq(args->argv[i], "-D") ||
  				    str_eq(args->argv[i], "-I") ||
@@ -1342,7 +1345,7 @@ cc_process_args(struct args *orig_args, struct 
args **preprocessor_args,
  	int found_c_opt = 0;
  	int found_S_opt = 0;
  	int found_arch_opt = 0;
-	int output_is_precompiled_header = 0;
+	int found_pch = 0;
  	const char *explicit_language = NULL; /* As specified with -x. */
  	const char *file_language;            /* As deduced from file 
extension. */
  	const char *actual_language;          /* Language to actually use. */
@@ -1582,6 +1585,7 @@ cc_process_args(struct args *orig_args, struct 
args **preprocessor_args,
  			};
  			int j;
  			char *relpath;
+			char *pchpath;
  			for (j = 0; opts[j]; j++) {
  				if (str_eq(argv[i], opts[j])) {
  					if (i == argc-1) {
@@ -1594,6 +1598,14 @@ cc_process_args(struct args *orig_args, struct 
args **preprocessor_args,
  					args_add(stripped_args, argv[i]);
  					relpath = make_relative_path(x_strdup(argv[i+1]));
  					args_add(stripped_args, relpath);
+
+					/* Try to be smart about detecting precompiled headers */
+					pchpath = format("%s.gch", argv[i+1]);
+					if (stat(pchpath, &st) == 0 && S_ISDIR(st.st_mode)) {
+						found_pch = 1;
+					}
+
+					free(pchpath);
  					free(relpath);
  					i++;
  					break;
@@ -1815,6 +1827,10 @@ cc_process_args(struct args *orig_args, struct 
args **preprocessor_args,
  	if (input_charset) {
  		args_add(*preprocessor_args, input_charset);
  	}
+	if (found_pch) {
+		cc_log("Use of precompiled header detected, adding -fpch-preprocess");
+		args_add(*preprocessor_args, "-fpch-preprocess");
+	}
  	if (explicit_language) {
  		args_add(*preprocessor_args, "-x");
  		args_add(*preprocessor_args, explicit_language);


I moved output_is_precompiled_header to be a global variable so it could 
be read in calculate_object_hash(), there's probably a nicer way to do 
that, passing it back from cc_process_args in an out-argument and
then back into calculate_object_hash() or something similar.

I also include all flags in the hash when output_is_precompiled_header 
is set, just to see if things would work. We could perhaps limit this to 
-D and a few others if we can figure out exactly what GCC puts into the 
precompiled header.

Finally, I added an improved check for use of precompiled headers, 
checking for the existence of a .gch directory in -include arguments 
(instead of "pch" in the filename as before). You're right that we can't 
detect all uses, and modifying the build system is the sure-safe way, 
but I think we can at least try to be smart here?

With this setup I do get mostly direct hits on subsequent runs. The few 
preprocessed hits I get happens when looking for the object file hash in 
the manifest and not finding the manifest. Not sure why this would happen?

Tor Arne