[ccache] ccache on generated C++ files...

[Malfettone, Kris]

So what I seem to be observing is only when I do a ninja -t clean or make clean.  This removes all of my generated files (.h and .cpp files) and of course the normal build outputs.  A subsequent run is then mostly cache misses.   If I say manually do a rm $(find -name "*.o") then I get a 100% cache hit rate.  I had assumed it was due to the fact that the generated files would have a newer mtime and ctime.  My assumption must be wrong.

I had tried using:
sloppiness = include_file_ctime,include_file_mtime

But that had the same results.  This makes sense of course because according to the documentation those are only for the "file too new" problem which makes sense.

Is there an easy way to print diagnostics on why a cache miss occurred?

-Kris

-----Original Message-----
From: Joel Rosdahl [mailto:joel at rosdahl.net]
Sent: Sunday, June 23, 2019 4:18 PM
To: Malfettone, Kris <Kris.Malfettone at msx.bala.susq.com>
Cc: ccache at lists.samba.org
Subject: Re: [ccache] ccache on generated C++ files...

Hi Kris,

On Fri, 21 Jun 2019 at 02:43, Malfettone, Kris wrote:
> My project uses a large number of generated C++ files (.h/.cpp files). On a
> clean build these files are cleaned up as well. As a result during the next
> build since they are generated again they will all have a newer mtime and
> ctime than the previous runs. Their content is the same though. Is there a
> way for those files to disable checking both the mtime and ctime checks?

I'm not sure I understand your question. Since the content of the generated
files is the same, they will get the same hash sum and will therefore be seen
as the same for the purposes of caching compilation results.

Or are you talking about the "sloppiness = file_stat_matches" option, where the
normal content hashing is sidestepped and matching in the direct mode can be
done by checking only size/mtime/ctime? If so, the answer is that it's not
possible to only check the size. Or rather: It would of course be possible to
implement such an option, but that does not seem like a good idea – it would be
way too dangerous since files of course can differ in many ways but still have
the same size.

But if you really want to try it yourself, you can simply change

  if (fi->fsize != st->size) {
    return false;
  }

to

  if (fi->fsize == st->size) {
    continue;
  }

in verify_result in src/manifest.c.

Is the problem that you think that hashing the generated files takes too much
time? If so, have you in some way been able to measure that hashing indeed is a
problem?

> I would love to be able to check based on some sort of "check file size then
> check content checksum" ignoring mtime and ctime.

Again, not sure I understand what you mean. In the normal case (no "sloppiness
= file_stat_matches"), the mtime/ctime is not checked for verification (but
they are checked to rule out "too new" files; see
include_file_mtime/include_file_ctime in the manual), so what you describe is
how it already works. In the "sloppiness = file_stat_matches" case, the
verification also falls back to check the content if mtime/ctime don't match.

-- Joel

________________________________

IMPORTANT: The information contained in this email and/or its attachments is confidential. If you are not the intended recipient, please notify the sender immediately by reply and immediately delete this message and all its attachments. Any review, use, reproduction, disclosure or dissemination of this message or any attachment by an unintended recipient is strictly prohibited. Neither this message nor any attachment is intended as or should be construed as an offer, solicitation or recommendation to buy or sell any security or other financial instrument. Neither the sender, his or her employer nor any of their respective affiliates makes any warranties as to the completeness or accuracy of any of the information contained herein or that this message or any of its attachments is free of viruses.