[ccache] CCACHE_NLEVELS , locking and cache size

[Pierre Tardy]

On Thu, Aug 22, 2013 at 2:18 AM, tal zilcer <tal.zilcer at gmail.com> wrote:

> hi,
>
> i recently started using ccache and i have a couple of questions.
>
> 1)i did a little experiment with different values of CCACHE_NLEVELS  [..]

 according to the build time results it seems it is better to use low
> NLEVELS value. when should this value be high?
> i dont understand the true meaning of this variable and what is he good
> for.
>

It is rather simple: it spreads the cached object in different folders. So
the overead you have is probably the time needed to create the folder.
The number of level is depending on the overall number of cache files you
want to store.

Each level will divide by 16 the maximum number of file found in the cache
directories.
Depending on the filesystem where you store your cache the time to access a
file can explode when the number of file in a directory is high.

Reiserfs is notoriously good at storing a lot of files in the same
directory. not sure of what happens if you put nfs in the whole story.


> 2)when using the cache for multiple users(about 30 people) i noticed that i
> get much slower compilation time because cache/NUMBER/stats.lock is locked
> a lot.
> is there an option to lock a more "deep" file(when using high NLEVELS
> value)? for instance cache/NUMBER/NUMBER/NUMBER/stats.lock.
> if not is there a solution to this problem?
>

Yeah NFS does not scale too much for such a high load a big ccache database
can generate.

You can try my memcached backend for ccache:
http://permalink.gmane.org/gmane.comp.compilers.ccache/1116

memcached is a real database, so it will scale much better than the
traditional ccache adhoc db + NFS

Regards,
Pierre