[ccache] why is limit_multiple ignored?

[Joel Rosdahl]

Hi Scott,

On 19 December 2017 at 02:16, Scott Bennett via ccache <
ccache at lists.samba.org> wrote:

>      I set "limit_multiple = 0.95" in ccache.conf and "max_size = 30.0G"
> in ccache.conf, but cleanups are triggered when space usage reaches 24 GB,
> which is the default of 0.8.  Why is this happening with ccache 3.3.4?
>

The ccache manual is not very good at describing what actually happens at
cleanup. I'll try to improve it.

Here's how cleanup works: After a cache miss, ccache stores the object file
in (a subdirectory of) one of the 16 top level directories in the cache
(0-9, a-f). It then checks if that top level directory holds more than
max_cache_size/16 bytes (and similar for max_files). If yes, ccache removes
files from that top level directory until it contains at most
limit_multiple*max_cache_size/16 bytes. This means that if limit_multiple
is 0.8, the total cache size is expected to hover around 0.9*max_cache_size
when it has filled up. But due to the pseudo-randomness of the hash
algorithm, the cache size can be closer to 0.8*max_cache_size or
1.0*max_cache_size.

The above should be true for any serial usage of ccache. However, ccache is
of course very often called in parallel, and then there is a race condition
since several ccache processes that have stored an object to the same top
level directory may start the cleanup process simultaneously. Since
performing cleanup in a large cache with a low limit_multiple can take a
lot of time, more ccache processes may start to perform cleanup of the same
directory. The race can lead to the final cache size being below
limit_multiple*max_cache_size, perhaps very much so. This is a known
problem. We have had some ideas to improve the admittedly naive cleanup
logic, but nothing has been done yet.

Maybe the above described problem is why you get a 24 GB cache size?

Or maybe you ran "ccache -c"? Unlike what the manual indicates, "ccache -c"
will delete files until each top level directory holds at most
limit_multiple*max_size/16...

why is limit_multiple ignored?


It isn't. Or don't you see a difference if you e.g. set it to 0.5?

-- Joel