[ccache] Speeding up builds with clang-tidy, clang-analyzer and iwyu
christian.ledergerber at ecorobotix.com
Mon Aug 17 13:33:55 UTC 2020
> 1. The solution will require a specific ccache version to work at all.
True, a good implementation should warn about a too old ccache version
and fail or fall back to running clang-tidy always.
> 2. The method of using ccache to probe whether clang-tidy should be
> run won't work for users who don't use warnings-as-errors. Also, how
> would CMake know that warnings-as-errors is used? It could be part of
> WarningsAsErrors in the config file or it could be on the command line
> but only for some warnings, etc. Sounds brittle and complex.
I was thinking that the user should be able to choose whether clang-tidy
should be run on a ccache hit or not. This he could specify by setting a
cmake variable. E.g. CMAKE_CLANG_TIDY_ON_CCACHE_HIT. This variable would
by default be false to guarantee backwards compatibility of cmake.
I personally feel like using warning-as-errors is the only reasonable
way to use any linting tool. For the same reasons why you should use a
compiler using -Wall -Werror. This is beside the discussion here. The
point is: I think this is not a coner-case, but people use this
> 3. The method is also brittle since the clang-tidy step will be
> skipped if the source is built (and therefore cached) by a normal
> non-clang-tidy build.
This I do not understand. Would this not be caught by the extra files to
cache? So: if the ccache caches also the configuration of clang-tidy
then there cannot be a cache hit due to binaries built without using
> (But if you apply your changes to a local CMake version you can make
> any assumptions you want, of course.)
> I spontaneously think that it would be a better idea to either (a)
> stop running clang-tidy for a normal build and instead run it as a
> special build once in a while, or (b) improve
> https://github.com/ejfitzgerald/clang-tidy-cache if you find it
> deficient in some way (I know nothing about it), or (c) implement
> clang-tidy support in ccache or another compiler cache program.
We do this already for clang-analyzer: it runs only on jenkins. But its
quite easy to mess up something with clang-tidy. This means that you
will most likely need to fix things in your PR. This then means you need
to run clang-tidy on almost every commit anyway and it generates quite
some work to clean up the git history.
As pointed in the title of this thread I believe that this approach
scales to more than just clang-tidy. I wish to also add iwyu to our
build system and will face the same problems: it will take almost the
same amount of time as compiling the code because the code needs to go
through the first few stages of clang (parsing, building an ast and the
like). Hence, fixing clang-tidy-cache does not appeal to me as it
doesn't scale. I feel like the algorithm discussed so far applies to any
kind of static analysis tool / linter:
- use the tool with warnings-as-errors
- add the configuration of the tool (and the tool itself??) to the
ccache extra files
- run it only if ccache has no hit
- run the compiler only if all static analysis passed
> Just a comment on your CMake GitLab issue:
>> 2. Add Class.clang-tidy to configuration of extra_files_to_hash of ccache using the command line syntax: --set-config=KEY=VALUE see https://ccache.dev/manual/latest.html
> That's not a good idea. --set-config is just a convenience alternative
> to manually editing the ccache configuration file. If you run it once
> for each compilation you will have race conditions between invocations
> and you will mess up the user's ccache configuration. To enable some
> ccache setting for a specific compilation you should set the
> corresponding environment variable, i.e. CCACHE_EXTRAFILES in this
Thanks for this comment! I feel like for easy integration into cmake
(and other tools) one would need another command line flag which allows
adding extra files to cache. E.g. --add-extra-files=file1;file2;file3.
> -- Joel
More information about the ccache