[distcc] ccache & distcc
Marcus Gruendler
Marcus.Gruendler at aixigo.de
Thu Sep 16 08:23:32 GMT 2004
Hi Sebastien,
A long while ago (~2 years) there was a patch for ccache posted on the distcc
mailing list. This patch enabled ccache to use distcc without changing the hash.
I don't know whether it is incorporated into the current ccache version. But
we are using this patch in our ccache version here and it works perfectly.
I have attached the patch again to my mail. But please remember that it is a
patch against a very old ccache version so it could not be applied to the
current ccache version, i suppose. But maybe you can see how it could be done.
Bye, Marcus
Martin Pool wrote:
> On Mon, 6 Sep 2004 11:10:27 +0200, Perochon Sebastien
> <sebastien.perochon at mmarelli-se.com> wrote:
>
>
>>I have compile my library with ccache (without distcc).
>>Then, to test it, I have clean my directory then re-compile the library. It
>>is working very well.
>>So, I decide to add distcc in the chain.
>>I set "CCACHE_PREFIX=distcc", then I clean again my directory and re-compile
>>the library...
>>But ccache doesn't use the already cached files and recompile them again.
>>Setting "CCACHE_PREFIX" with distcc seems to make differents files for
>>ccache...
>
>
> Adding a compiler prefix command changes the hash; things compiled
> without distcc are not considered the same as things compiled with it.
> See the ccache source.
>
-------- Original Message --------
Subject: [distcc] [patch] support for distcc in ccache
Date: Thu, 15 Aug 2002 13:47:08 +0200
From: Chris Halls <chris.halls at gmx.de>
To: distcc at lists.samba.org
I've had a go at integrating support for distcc into ccache directly. It
solves these issues:
- ccache's compiler timestamp check was using the distcc binary instead of
the compiler, so a compiler upgrade would trigger a cache hit (ie no
recompilation), and a distcc upgrade would trigger a cache miss.
- ccache treated 'distcc gcc ...' and 'gcc ...' for the same source file as
different, meaning a complete recompile was necessary when switching
between using distcc and not using distcc with ccache.
- If DISTCC_VERBOSE was set, distcc's messages would be cached, too, giving
misleeding output when a file was retreived form the cache.
- distcc was called needlessly during the preprocessing step.
To use, set the enviroment variable CCACHE_DISTCC. You do not need to
specify distcc directly. In other words:
CC="ccache distcc gcc" make
becomes:
CCACHE_DISTCC=1 CC="ccache gcc" make
or, if ccache is installed as gcc on your path:
CCACHE_DISTCC=1 make
The patch is against current ccache CVS, but should apply against the latest
released version, too. I'd be grateful for feedback, and whether this would
be suitable for me to send to the ccache author.
Thanks,
Chris
--
Dipl.-Inform. Marcus Gründler
aixigo AG - financial solutions & technology
Schloß-Rahe-Straße 15, 52072 Aachen, Germany
fon: +49 (0)241 936737-43, fax: +49 (0)241 936737-99
eMail: Marcus.Gruendler at aixigo.de, web: http://www.aixigo.de
-------------- next part --------------
Index: args.c
===================================================================
RCS file: /cvsroot/ccache/args.c,v
retrieving revision 1.4
diff -u -3 -p -r1.4 args.c
--- args.c 1 Apr 2002 03:58:57 -0000 1.4
+++ args.c 15 Aug 2002 06:34:16 -0000
@@ -47,3 +47,20 @@ void args_pop(ARGS *args, int n)
args->argv[args->argc] = NULL;
}
}
+
+ARGS *args_copy_and_insert(ARGS *fromargs, int pos, int n)
+{
+ ARGS *args;
+ int i;
+ args = malloc(sizeof(ARGS));
+ args->argc = fromargs->argc + n;
+ args->argv = malloc( (args->argc + 1) * sizeof(char *));
+ for (i=0;i<pos;i++)
+ args->argv[i] = fromargs->argv[i];
+ for (;i<pos+n;i++)
+ args->argv[i] = NULL;
+ for (;i<args->argc;i++)
+ args->argv[i] = fromargs->argv[i-n];
+ return args;
+}
+
Index: ccache.1
===================================================================
RCS file: /cvsroot/ccache/ccache.1,v
retrieving revision 1.14
diff -u -3 -p -r1.14 ccache.1
--- ccache.1 3 Jun 2002 03:51:04 -0000 1.14
+++ ccache.1 15 Aug 2002 06:34:16 -0000
@@ -190,6 +190,13 @@ doesn\&'t work, for example when using t
systems like this you can use the CCACHE_EXTENSION option to override
the default\&. On HP-UX set this environment variable to "i" if you use
the aCC compiler\&.
+.IP
+.IP "\fBCCACHE_DISTCC\fP"
+If you set the environment cariable CCACHE_DISTCC, ccache will call distcc to
+perform the compilation step. You need to set DISTCC_HOSTS - see the distcc
+manpage\&. No checking is performed to test if the compiler version on the
+remote host is the same as the local version, so be careful to keep the
+compiler versions in sync\&.
.IP
.PP
.SH "CACHE SIZE MANAGEMENT"
Index: ccache.c
===================================================================
RCS file: /cvsroot/ccache/ccache.c,v
retrieving revision 1.63
diff -u -3 -p -r1.63 ccache.c
--- ccache.c 3 Jun 2002 04:34:56 -0000 1.63
+++ ccache.c 15 Aug 2002 06:34:16 -0000
@@ -124,12 +124,30 @@ static const char *tmp_string(void)
/* run the real compiler and put the result in cache */
-static void to_cache(ARGS *args)
+static void to_cache(ARGS *origargs)
{
char *path_stderr;
char *tmp_stdout, *tmp_stderr, *tmp_hashname;
+ char *distcc_log, *tmp_distcc;
struct stat st1, st2;
int status;
+ ARGS *args;
+
+ if (getenv("CCACHE_DISTCC")) {
+ cc_log("using distcc for compilation\n");
+ args = args_copy_and_insert(origargs, 0, 1);
+ args->argv[0] = "distcc";
+ distcc_log = getenv("DISTCC_LOG");
+ if(!distcc_log) {
+ /* distcc will not create a log file - log to temporary file */
+ x_asprintf(&tmp_distcc, "%s/tmp.distcc.%s", cache_dir, tmp_string());
+ setenv("DISTCC_LOG", tmp_distcc, 0);
+ }
+ } else {
+ args = origargs;
+ tmp_distcc = NULL;
+ distcc_log = NULL;
+ }
x_asprintf(&tmp_stdout, "%s/tmp.stdout.%s", cache_dir, tmp_string());
x_asprintf(&tmp_stderr, "%s/tmp.stderr.%s", cache_dir, tmp_string());
@@ -152,9 +170,24 @@ static void to_cache(ARGS *args)
unlink(tmp_stdout);
unlink(tmp_stderr);
unlink(tmp_hashname);
+ if(tmp_distcc) {
+ unlink(tmp_distcc);
+ }
failed();
}
unlink(tmp_stdout);
+
+ if (tmp_distcc) {
+ /* Send distcc output to terminal */
+ int fd;
+ fd = open(tmp_distcc, O_RDONLY);
+ if (fd != -1) {
+ copy_fd(fd, 2);
+ close(fd);
+ }
+ unlink(tmp_distcc);
+ free(tmp_distcc);
+ }
if (status != 0) {
int fd;
Index: ccache.h
===================================================================
RCS file: /cvsroot/ccache/ccache.h,v
retrieving revision 1.38
diff -u -3 -p -r1.38 ccache.h
--- ccache.h 3 Jun 2002 03:22:12 -0000 1.38
+++ ccache.h 15 Aug 2002 06:34:16 -0000
@@ -121,6 +121,7 @@ typedef struct {
ARGS *args_init(void);
void args_add(ARGS *args, const char *s);
void args_pop(ARGS *args, int n);
+ARGS *args_copy_and_insert(ARGS *fromargs, int pos, int n);
#if HAVE_COMPAR_FN_T
#define COMPAR_FN_T __compar_fn_t
Index: execute.c
===================================================================
RCS file: /cvsroot/ccache/execute.c,v
retrieving revision 1.4
diff -u -3 -p -r1.4 execute.c
--- execute.c 27 Mar 2002 00:39:06 -0000 1.4
+++ execute.c 15 Aug 2002 06:34:16 -0000
@@ -52,7 +52,7 @@ int execute(char **argv,
dup2(fd, 2);
close(fd);
- exit(execv(argv[0], argv));
+ exit(execvp(argv[0], argv));
}
if (waitpid(pid, &status, 0) != pid) {
More information about the distcc
mailing list