[ccache] [patch] Massive performance degradation with direct mode due to bypassing filesystem cache with mmap (Mac/PPC)

Joel Rosdahl joel at rosdahl.net
Wed Aug 25 14:24:16 MDT 2010


On 2010-08-23 13:35, Christian Lohmaier wrote:
> [...] Using mmap [on Mac/PPC] to load all the include files bypasses
> the filesystem cache [...]

Ouch. That surprises me. Thanks for telling!

The reason I chose mmap() instead of read() was mainly that the C
"parser" (that basically knows about C/C++ comments and string literals)
was easier to write under the assumption that it has access to the whole
file as a string and I figured that it would be better to mmap() than to
read the whole file into memory (or at least not worse). I didn't
perform any benchmarks to back up that choice, though. Now I've done
some tests, and it seems that on my 64-bit Linux system, mmap() is
slightly faster for (very) large files, but for smaller files, I can't
measure any real difference consistently. I assume that reading the
whole file into memory instead of using mmap() isn't considerably slower
on other systems too, so I'll make ccache do that.

> * is there any reason why you don't use calloc but use malloc + memset?

I guess you mean in manifest.c? No, no special reason. Changed. Thanks!

> * when using ccache 2.4, the patch to fix --ccache-skip was needed, at
> first glance that code did not change, so my guess is that it is still
> needed?
> http://www.mail-archive.com/ccache@lists.samba.org/msg00197.html
> (talking about this one)

That's funny, I just noticed and fixed that bug three days ago! I didn't
know of the patch. I haven't scanned old mailing list posts for patches;
maybe I should.

> Also I noticed that ccache now reject everything remotely looking like
> a filelist, this is a little unfortunate since OOo uses -install_name
> @_(50 underscores)identifier/libraryname to later rewrite it to the
> target location and thus those compiles won't be cached - hashing
> install_name and its value should be OK....

I don't quite follow you. What do you mean by "reject"?

-- Joel


More information about the ccache mailing list