[ccache] PATCH: Look at include files' mtimes

Joel Rosdahl joel at rosdahl.net
Sun Mar 3 13:54:28 MST 2013


Hi Justin,

> I've resurrected these patches to look at files' mtimes and ctimes. [...]

I just found out that I forgot to have a look at your patches. Sorry
about the delay.

I seem fine, so I've applied them. I did need to fix the unit tests
since they failed, though. Please have a look and see if it looks all
right.

Thanks,
-- Joel

On 25 December 2012 08:18, Justin Lebar <justin.lebar at gmail.com> wrote:
> Hi, all.
>
> I've resurrected these patches to look at files' mtimes and ctimes.
> Hopefully the three patches here (with their commit messages) don't
> need further explanation.  Note that the second patch here increases
> safety for everyone, not just those who choose to have mtime matching
> on.
>
> These patches seem to be working, but I'm not seeing a significant
> speedup on my Mac.  I think that may be a separate issue, as this
> machine isn't particularly good at I/O.  I don't have access to my
> Linux box for a while, so I'd certainly appreciate if someone could
> verify whether there's a speedup here.
>
> I'd also appreciate if some of you could test this patch by turning on
> CCACHE_SLOPPINESS=file_stat_matches and letting me know if you have
> any problems.
>
> Happy holidays.
>
> -Justin
>
> On Sun, May 20, 2012 at 4:49 PM, Justin Lebar <justin.lebar at gmail.com> wrote:
>> This patch lets ccache examine an include file's mtime and size in
>> lieu of hashing it, during direct mode.  If the mtime and size don't
>> match, we fall back to hashing.
>>
>> The net result is roughly a factor-of-two speedup in ccache hits (*),
>> on my machine.
>>
>> I'm not sure if this is a desirable feature, because obviously mtimes
>> can be tampered with.
>>
>> I didn't provide a way to disable the feature in this patch because,
>> presuming we wanted to take this patch, I'm not sure if we'd want
>> mtime-snooping enabled by default.  Since most projects already rely
>> on accurate mtimes in their build systems, turning this on by default
>> doesn't seem particularly outrageous to me.
>>
>> Please let me know what you think about this.
>>
>> Regards,
>> -Justin
>>
>> (*) Experimental procedure: In a Firefox debug objdir
>> (CCACHE_HARDLINK, Linux-64, Ubuntu 12.04, 4 CPU cores), let
>>
>> * Let |nop| be the average real time from a few runs of
>>
>>     $ time make -C dom -sj16
>>
>>   when there's nothing to do.
>>
>> * Let |orig| be the average real time from a few runs of
>>
>>     $ find dom -name '*.o' && time make -C dom -sj16
>>
>>   with ccache master (701f13192ee) (discarding the first run, of course).
>>
>> * Let |mtime| be the real time from the same command as |orig|, but
>> with patched ccache.
>>
>> Speedup is (orig - nop) / (mtime - nop).  On my machine, nop = 3.71,
>> orig = 4.88, mtime = 4.31.  Yes, our nop build times are atrocious.
>
> _______________________________________________
> ccache mailing list
> ccache at lists.samba.org
> https://lists.samba.org/mailman/listinfo/ccache
>


More information about the ccache mailing list