[ccache] Why not cache compile failures?

Shentino shentino at gmail.com
Fri Oct 5 07:12:20 MDT 2012


On Fri, Oct 5, 2012 at 2:17 AM, Andrew Stubbs <ams at codesourcery.com> wrote:
> On 04/10/12 01:50, Shentino wrote:
>>
>> Would snooping on the compiler's own error messages help weed out
>> transient failures like out of memory, disk full, and so on?
>>
>> I recently compiled webkit-gtk and it managed to exhaust all of my
>> memory AND swap.
>
>
> This does raise an interesting choice!
>
> On the one hand, the scheme Martin has suggested for not caching failures
> due to signals should mean that this result would not be cached (an
> out-of-memory condition *could* be reported by the compiler as a malloc
> failure, but overcommit makes it more likely to show up as a
> SIGTERM/SIGKILL).

It's indeed unpredictable how ccache could be notified of it.  And
overcommit is configurable at the whim of the end user anyhow, so I'd
consider it unwise to make any assumptions about it.  IMHO better to
be robust and handle both cases as well as possible.

> But on the other hand, if you're not sharing a cache with others, and it's
> physically impossible to compile a given file on your machine, it might be
> better to fail instantly rather than kill your machine everytime you run it
> by mistake!

A good special case, but it's IMHO none of ccache's business what the
user is doing with his memory, and most likely a prudent user will be
upping his resource allocations before trying again.

Or at the very least, failures of this type ought to be cached for
shorter periods of time if at all, since the underlying cause is
something the user should hopefully be strongly motivated to repair
before attempting again.

As an example, my system was able to digest the compilation
successfully after I fiddled with lvm and upped my swap space to a
whopping 64G.  After that the compilation cleared successfully.  I did
have to wait awhile, and I did fall victim to a nasty swap handling
bug in the 3.3.8 kernel and have to start over again after upgrading
to 3.5.3, but I was eventually able to deal with what made it choke to
begin with.

A happy medium might be to detect "transient" failures of this sort
and allow an easy way for the user to flush those out of the cache on
demand.

> On balance, I prefer the first answer though: not caching memory or disk
> exhaustion. (Disk full is harder to detect though, as it's not a signal.)

This was indeed my motivation for suggesting that ccache eavesdrop on
gcc's stderr output.

If the compiler wants to use stderr to bitch about something that puts
cacheability in doubt, ccache may as well pay attention.

> Andrew


More information about the ccache mailing list