[ccache] Build Safety

[Dalton, Tom]

Thanks!
 
I also realised that each developer builds their own compiler from source, so the modification date will also be different. However, I can think of ways that this can be solved, like using relative paths for the includes. I think that initially though, we will get a nice improvement from using a local cache for each machine :-)
 
Cheers for your help!
 
Tom


________________________________

	From: Vincent Dupuis [mailto:vincedupuis at hotmail.com] 
	Sent: 25 September 2009 14:50
	To: Dalton, Tom
	Subject: RE: [ccache] Build Safety
	
	
	Tom, 
	
	you are right. ccache is independent of your project/component setup.
	it is source file (compile) based.
	
	for each file compiled
	build a key (see below 1-4)
	if the key is found return the .o
	else add the new key and the new .o in the cache
	
	Remember that if the file is exactly the same but the compiler option
	are not (#2) like the -I option (include path). It will not retrieve the cached file.
	So your makefile needs to use relative path or all user need to use the same include paths.
	
	V
	
	
	> Date: Fri, 25 Sep 2009 14:56:48 +0200
	> From: tom.dalton at eds.com
	> To: ccache at lists.samba.org
	> Subject: Re: [ccache] Build Safety
	> 
	> Thanks Vincent, that is very useful. I have one more clarification on
	> why I think using a shared cache will be useful (if it works!)
	> 
	> We have several sub-projects, A, B, C, D.
	> Our developers (and their development/build machines) can move around
	> between the sub-projects.
	> 
	> 
	> Subproject A and B 1 use gcc3, subproject C and D use gcc4.
	> Subproject A uses libraryX v1, subprojects B,C,D use libraryX v2.
	> 
	> So effectively there are 3 configurations, (gcc3 libX1), (gcc3 libX2),
	> (gcc4 libX2).
	> 
	> Now if each developer has a local cache, then each developer needs to do
	> each compile at least once, before the cache is populated. But if each
	> subproject has 10 developers, and the cache is shared, then only 1
	> developer needs to build the subproject, and then all the other
	> developers will get lots of cache hits when they compile.
	> 
	> From my point of view, it is easier to configure a single, large shared
	> cache across all developers than to set up a single cache for each
	> subproject team. Also, in the above example, subprojects C and D have
	> the same configuration and so could actually use the same cache.
	> 
	> My issue is that the real scenario is much more complex, with each
	> subproject being comprised many components, which in turn are components
	> etc. So identifying the configurations is not trivial. I suspect that in
	> reality, each project has some compilation in common with some of the
	> other projects, but it's hard to tell which. So if ccache can cope with
	> using a network-shared cache, then having one single (large) cache would
	> give maximum efficiency across all the projects, as any
	> cache-partitioning I impose (per-project or per-developer or
	> per-machine).
	> 
	> I hope that makes sense, but I don't know how ccache actually reads the
	> cache. In effect, is it safe to use the same shared cache on multiple
	> machines? If not, then is it safe to run a parallel build (e.g. make -j
	> 2) on a local machine with a local cache, as presumably that has the
	> same implications wrt. multiple concurrent cache updates.
	> 
	> Thanks for your help!
	> 
	> Tom
	> 
	> 
	> ________________________________
	> 
	> From: Vincent Dupuis [mailto:vincedupuis at hotmail.com] 
	> Sent: 25 September 2009 13:18
	> To: Dalton, Tom
	> Cc: ccache at lists.samba.org
	> Subject: RE: [ccache] Build Safety
	> 
	> 
	> Hi, and welcome to ccache!
	> 
	> > Date: Fri, 25 Sep 2009 10:22:14 +0200
	> > From: tom.dalton at eds.com
	> > To: ccache at lists.samba.org
	> > Subject: [ccache] Build Safety
	> > 
	> > Hi,
	> > 
	> > I think I know the answer to this question, but I want a
	> second opinion!
	> > 
	> > In the past, I have used the Clearcase version control system,
	> which has
	> > it's own make system, clearmake. Clearmake provides some build
	> caching
	> > functionality (which it terms wink-ins), that allows it to
	> 'wink-in'
	> > build objects from other users views. I have experienced a lot
	> of
	> > problems with this system, because clearcase wouldn't detect
	> that two
	> > users were building the same file in a different environment
	> (e.g.
	> > external libraries at different versions. This caused me all
	> sorts of
	> > issues.
	> 
	> That's why it is not very used.
	> 
	> > Is it possible for inconsistent environments between builds to
	> cause
	> > ccache to incorrectly use a cached object instead of
	> recompiling? For
	> > example, different builds of gcc, different external libraries
	> etc? What
	> > circumstances could cause this?
	> 
	> for each compile ccache makes a key with the following
	> information: 
	> 
	> 1. the pre-processor output from running the compiler
	> with -E 
	> 2. the command line options 
	> 3. the real compilers size and modification time 
	> 4. any stderr output generated by the compiler
	> 
	> if something changes then it recompiles and recache the new
	> key/.o
	> so if you use a different external librairies, #1 and maybe #2
	> can change.
	> 
	> > Secondly, if there is not much that can break ccache, is it
	> possible to
	> > use a shared cache (e.g. on an nfs-shared disk) to allow many
	> users on
	> > many hosts to cache build results across several versions of a
	> single
	> > product? (which will mean different gcc versions, libraries,
	> even
	> > kernels.)
	> 
	> if each users use different gcc versions, libraries, even
	> kernels, then
	> sharing the cache is not useful. It's better to use it locally.
	> Try to compile
	> your product from scratch witch ccache and recompile (clean
	> before) and 
	> compare the time. (ie: time make all). Here is my current stats
	> at this moment:
	> 
	> cache directory
	> /home/vince/projects/raaf/ccache
	> cache hit 16462
	> cache miss 11307
	> called for link 70
	> compile failed 63
	> preprocessor error 4
	> not a C/C++ file 1198
	> files in cache 8779
	> cache size 793.5 Mbytes
	> max cache size 976.6 Mbytes
	> 
	> like you see, I skip compilation of 16462 files locally.
	> 
	> 
	> > Thanks for your help - ccache sounds really useful, but I want
	> to make
	> > sure there's no 'traps'!
	> > 
	> > Thanks,
	> > 
	> > Tom
	> > _______________________________________________
	> > ccache mailing list
	> > ccache at lists.samba.org
	> > https://lists.samba.org/mailman/listinfo/ccache
	> 
	> 
	> ________________________________
	> 
	> Naviguez plus vite avec Internet Explorer 8 Ici.
	> <http://go.microsoft.com/?linkid=9655263> 
	> 
	> _______________________________________________
	> ccache mailing list
	> ccache at lists.samba.org
	> https://lists.samba.org/mailman/listinfo/ccache
	
	
________________________________

	Avec Windows Live, vous gardez le contact avec tous vos amis au même endroit. <http://go.microsoft.com/?linkid=9660828>