[distcc] [PATCH] zeroconf support for distcc

[Lennart Poettering]

Hi!

As I promised I prepared some zeroconf patches for distcc

The patch:

http://0pointer.de/public/distcc.patch

A straightforward debian package with this patch applied: 

http://0pointer.de/public/distcc_2.18.1-4_i386.deb

Some notes:

- The patch is against distcc 2.18.1, and tested with howl 0.9.6 from
  the debian packages. howl's build system is seriously broken (or at
  least "was", since current upstream is already at 0.9.8 and I didn't
  look into that), because it installs its autoconf generated config.h
  to /usr/include/howl/howl_config.h. That config file defines its own
  PACKAGE_NAME macros (and friends) which conflicts with every
  autoconf based application including distcc. My patch works around
  this by undefining these macros before including howls header
  files. I already filed a bug regarding this to the Debian BTS.

- If HOWL is not found, distcc is built without support for
  zeroconf. I hope I made no mistake with the autoconf scripts.

- the daemon got a new command line option "--zeroconf". If specified
  it registers itself in mDNS/DNS-SD using howl. This works only if a
  local mDNS responder is running.

- The client now understands a new magic host name in DISTCC_HOSTS:
  "+zeroconf". If this is found it is replaced by the adresses of all
  zeroconf enabled hosts on the net. 

- The client now understands a new option "--show-hosts" for listing
  all hosts detected by zeroconf and all otherwise specified. This is
  useful for checking if the  local zeroconf setup works. It works
  even when used without zeroconf. The output of --show-hosts is
  compatible with the format used by ~/.distcc/hosts.

- The client now understands a new option "-j" which returns a numeric
  concurrency level for usage with "make -j". It is calculated from
  the entries in the host list. This is very useful when used with
  zeroconf since most of the time you don't know how many hosts are
  available on your network, on which you can distribute your
  work. This works even without zeroconf.

- Since gathering zeroconf browse info takes some undeterministic time
  the distcc client forks off a background process for that. This
  process is reused by subsequent distcc calls. That means: the first
  call to distcc waits 100ms for letting the forked process enough
  time to gather host info data, all subsequent
  calls of distcc use the data from the background process
  immediately. When the background process is unused for 20s it
  terminates itself. Communication between the background process and
  distcc is file based using ~/.distcc/zeroconf/hosts. That file is
  properly locked and compatible with the format of ~/.distcc/hosts. I know
  that forking off a new process is a bit ugly, but there's simply no
  better way to do it. Most other implementations of things like this usually
  wait 100ms unconditionally, something I think is very ugly since
  distcc is started many times in a single build.

- distccd exports the number of local CPUs in DNS TXT RRs. The client
  makes use of that info and fills it into n_slots.

- The patch fixes an obvious typo in io.c: in dcc_readx() you check
  for EAGAIN twice. I corrected this into a check for EINTR. I also
  removed a non-sense EOF check in dcc_writex(). There's no such thing
  as an EOF when using Unix write(). This exists for read()s only.

- A added my own version of a lock function called "generic_lock"
  which is based on your sys_lock, but has the advantage of allowing
  r/o locks and unlocking of files. It makes sense to fold this back
  into sys_lock().

- what is the reason for not having the usual #ifndef/#define/#endif
  protection lines in your header files? It really makes things
  easier.

- I exported dcc_get_subdir because I needed it to get the path for
  ~/.distcc/zeroconf.

- I changed the signature of dcc_get_hostlist() a bit, since I needed
  to run it recursively on the same host list from my zeroconf code:
  you may specify a pointer to the list's tail now.

A quick step-by-step guide how to use this:

1) install mdnsresponder and howl and compile distcc with this patch
   applied

2) run distccd with --zeroconf AFTER mDNSResponder

3) set $DISTCC_HOSTS to "+zeroconf"

4) Now compile as usual with "make -j `distcc -j`"

That's it.

If you plan to install the debian package I prepared, make sure to add
--zeroconf to the init script /etc/init.d/distcc. At least on Debian
this init script is executed after that one of mDNSResponder. You've
to swap this when using --zeroconf, since it requires a running
mDNSResponder.

Please apply or tell me what to improve!

Thanks,
        Lennart
-- 
name { Lennart Poettering } loc { Hamburg - Germany }
mail { mzft (at) 0pointer (dot) de } gpg { 1A015CC4 }  
www { http://0pointer.de/lennart/ } icq# { 11060553 }