[distcc] Re: Simple autodiscovery of distcc servers

[Tobias Polzin]

> I have had problems with too high of a -j# setting, but many of my problems
> are specific to the type of farm I have setup. There are situations when
> many people are compiling at once, and with too few machines on the farm too
> many parallel makes has caused some strange problems with our cross
> compilers...

I start a little script to filter out the offline and too busy hosts. It also
does some stupid load-balancing (compiling a hello word and look, how long it
took). I found it somewhat usefull that fast hosts are placed in the beginning
of the DISTCC_HOSTS list. The script corrects the MAKEFLAGS -j# entry, too.

I would be interested to read some more details how you implemented your
host-file-mechanism. Is there anything you could post?

I have also tried icecream 
http://wiki.kde.org/tiki-index.php?page=icecream
ftp://ftp.suse.com/pub/projects/icecream
it looks nice, but I haven´t tried to use it in greater scale (and did not try
to compile it under cygwin). Any other experiences on that?

-----------------------------------------------
#
# Sorts DISTCC_HOSTS according to speed
#
# Usage "source distcc-hostsort"
#
# First Version: 17.01.05 (polzin_spamprotect_ at gmx dot de)
#
# $Id: distcc-hostsort,v 1.7 2005/08/23 09:31:12 tobias Exp $

tmplog=/tmp/dummy.log

#subshell to preserve as much enviroment as possible
(

  tmpfile=/tmp/dummy.cpp

  cat > $tmpfile <<!
#include <iostream>
int main()
{
  std::cout << "hello distcc\n";
}
!

  oldhosts="$DISTCC_HOSTS"
  export TIMEFORMAT=%R
  rm -f $tmplog
  echo $oldhosts

  for host in $oldhosts
  do
     echo "$host" " start"
     ( DISTCC_HOSTS="$host";
       # using localhost is not wanted
       export DISTCC_FALLBACK=0
       t=` ( time   distcc gcc $tmpfile -c -o /dev/null) 2>&1 `
       echo -n $host " --- " $t
       # no floating point in scrips
       if [[ ${t%.*} -lt 6 ]];
       then
          echo $t $host >> $tmplog
          echo
       else
          echo " [too slow -> skipped]"
       fi
       )&
  done
  echo waiting for subtasks
  # jobs
  wait
)
echo ready
DISTCC_HOSTS=` sort -n $tmplog | awk '{ print $2; }' `
echo DISTCC_HOSTS=$DISTCC_HOSTS

count=` wc -l $tmplog | sed -e "s@$tmplog@@"`
MAKEFLAGS=`echo "$MAKEFLAGS" | sed -e "s/-j[0-9]*/-j$count/"`
echo MAKEFLAGS=$MAKEFLAGS

#clean up, as this file has to be sourced
# rm $tmpfile
unset count
unset tmplog
---------------------------------------