[ccache] Long pauses due to automatic cache clean-up
joel at rosdahl.net
Tue Apr 27 17:59:33 UTC 2021
On Sun, 18 Apr 2021 at 10:04, R. Diez via ccache <ccache at lists.samba.org> wrote:
> First of all, I have been using ccache for years. Many thanks for this great tool.
> I have come to the conclusion that these particular pauses are due to ccache
> automatically pruning its cache.
I haven't really observed pauses due to cleanup myself since I have
used SSDs for
a long time, but yes, they will occur with a slow enough disk (or loaded system)
and many parallel ccache invocations.
> I was surprised that completely clearing the cache with option '--clear' can
> take a very long time, 12 minutes the last time around, because ccache
> probably does not need to calculate anything beforehand, just delete all
> files. But I haven't benchmarked the filesystem itself, so perhaps it's just
> Linux being slow deleting so many small files.
Yes, what you're seeing is a slow disk (or filesystem) in combination with many
files in the cache directory. You will get similar performance from "rm -r".
Upgrading to ccache 4.x will actually help since then only one file will be
stored per cache entry instead of up to seven files (but more likely two) with
ccache 3.x, so there will be fewer files to consider and remove when cleaning or
> What happens if you are building in parallel, with "make -j32", and all 32
> concurrent instances decide to clean the cache at the same time? Will all of
> them attempt to delete the same files at the same time?
Yes, that could happen. A ccache invocation that writes the result of a
compilation to one of the 16 cache buckets will start cleaning up that bucket if
the cache size threshold has been reached. If another parallel compilation
finishes and writes to the same bucket before the first has finished cleaning
then it too will start cleaning and so on. There is no coordination between such
This is how the automatic cleanup has worked since ccache was initially created
in 2002. I have ideas on how this can be improved but I don't have much time and
interest to work on ccache these days so things are unfortunately slow. (I no
longer use ccache personally except for building ccache...)
> I wonder if there is a way to increase the number of buckets, from 16 to say
> to 1024. There is not reason to have so few of them.
There is a reason, and it's spelled "backward compatibility". Other minor
reasons also exist.
> I have also being thinking of triggering a manual cache clean-up on start-up
> Is that a good strategy? Or are there better ways?
I think it's a good strategy given that you have a problem with automatic
cleanups. You could consider disabling automatic cleanup completely (set
max_size to 0) and run "CCACHE_MAXSIZE=20G ccache -c" or so each night or more
often. The limit will then only affect that ccache invocation; it won't be
stored in the config file. Also, you don't have to parse the ccache
configuration since the cron job has the definition of what the max cache size
> Is there some script able to parse the ccache config file that I could use as
> an example?
Not that I know of. You can query the size with "ccache -k max_size", but it
will include a size suffix that you need to parse if you want to make it
More information about the ccache