[ccache] PATCH: Look at include files' mtimes

[Justin Lebar]

FWIW I've been using ccache with mtime checking for the past few weeks
and I haven't noticed any problems.  That is a pretty low bar, I
admit, but it's something.  :)

On Sun, Mar 3, 2013 at 3:54 PM, Joel Rosdahl <joel at rosdahl.net> wrote:
> Hi Justin,
>
>> I've resurrected these patches to look at files' mtimes and ctimes. [...]
>
> I just found out that I forgot to have a look at your patches. Sorry
> about the delay.
>
> I seem fine, so I've applied them. I did need to fix the unit tests
> since they failed, though. Please have a look and see if it looks all
> right.
>
> Thanks,
> -- Joel
>
> On 25 December 2012 08:18, Justin Lebar <justin.lebar at gmail.com> wrote:
>> Hi, all.
>>
>> I've resurrected these patches to look at files' mtimes and ctimes.
>> Hopefully the three patches here (with their commit messages) don't
>> need further explanation.  Note that the second patch here increases
>> safety for everyone, not just those who choose to have mtime matching
>> on.
>>
>> These patches seem to be working, but I'm not seeing a significant
>> speedup on my Mac.  I think that may be a separate issue, as this
>> machine isn't particularly good at I/O.  I don't have access to my
>> Linux box for a while, so I'd certainly appreciate if someone could
>> verify whether there's a speedup here.
>>
>> I'd also appreciate if some of you could test this patch by turning on
>> CCACHE_SLOPPINESS=file_stat_matches and letting me know if you have
>> any problems.
>>
>> Happy holidays.
>>
>> -Justin
>>
>> On Sun, May 20, 2012 at 4:49 PM, Justin Lebar <justin.lebar at gmail.com> wrote:
>>> This patch lets ccache examine an include file's mtime and size in
>>> lieu of hashing it, during direct mode.  If the mtime and size don't
>>> match, we fall back to hashing.
>>>
>>> The net result is roughly a factor-of-two speedup in ccache hits (*),
>>> on my machine.
>>>
>>> I'm not sure if this is a desirable feature, because obviously mtimes
>>> can be tampered with.
>>>
>>> I didn't provide a way to disable the feature in this patch because,
>>> presuming we wanted to take this patch, I'm not sure if we'd want
>>> mtime-snooping enabled by default.  Since most projects already rely
>>> on accurate mtimes in their build systems, turning this on by default
>>> doesn't seem particularly outrageous to me.
>>>
>>> Please let me know what you think about this.
>>>
>>> Regards,
>>> -Justin
>>>
>>> (*) Experimental procedure: In a Firefox debug objdir
>>> (CCACHE_HARDLINK, Linux-64, Ubuntu 12.04, 4 CPU cores), let
>>>
>>> * Let |nop| be the average real time from a few runs of
>>>
>>>     $ time make -C dom -sj16
>>>
>>>   when there's nothing to do.
>>>
>>> * Let |orig| be the average real time from a few runs of
>>>
>>>     $ find dom -name '*.o' && time make -C dom -sj16
>>>
>>>   with ccache master (701f13192ee) (discarding the first run, of course).
>>>
>>> * Let |mtime| be the real time from the same command as |orig|, but
>>> with patched ccache.
>>>
>>> Speedup is (orig - nop) / (mtime - nop).  On my machine, nop = 3.71,
>>> orig = 4.88, mtime = 4.31.  Yes, our nop build times are atrocious.
>>
>> _______________________________________________
>> ccache mailing list
>> ccache at lists.samba.org
>> https://lists.samba.org/mailman/listinfo/ccache
>>