[distcc] Re: Simple autodiscovery of distcc servers
Tobias Polzin
polzin_spamprotect_ at gmx.de
Tue Aug 23 09:53:57 GMT 2005
> I have had problems with too high of a -j# setting, but many of my problems
> are specific to the type of farm I have setup. There are situations when
> many people are compiling at once, and with too few machines on the farm too
> many parallel makes has caused some strange problems with our cross
> compilers...
I start a little script to filter out the offline and too busy hosts. It also
does some stupid load-balancing (compiling a hello word and look, how long it
took). I found it somewhat usefull that fast hosts are placed in the beginning
of the DISTCC_HOSTS list. The script corrects the MAKEFLAGS -j# entry, too.
I would be interested to read some more details how you implemented your
host-file-mechanism. Is there anything you could post?
I have also tried icecream
http://wiki.kde.org/tiki-index.php?page=icecream
ftp://ftp.suse.com/pub/projects/icecream
it looks nice, but I haven´t tried to use it in greater scale (and did not try
to compile it under cygwin). Any other experiences on that?
-----------------------------------------------
#
# Sorts DISTCC_HOSTS according to speed
#
# Usage "source distcc-hostsort"
#
# First Version: 17.01.05 (polzin_spamprotect_ at gmx dot de)
#
# $Id: distcc-hostsort,v 1.7 2005/08/23 09:31:12 tobias Exp $
tmplog=/tmp/dummy.log
#subshell to preserve as much enviroment as possible
(
tmpfile=/tmp/dummy.cpp
cat > $tmpfile <<!
#include <iostream>
int main()
{
std::cout << "hello distcc\n";
}
!
oldhosts="$DISTCC_HOSTS"
export TIMEFORMAT=%R
rm -f $tmplog
echo $oldhosts
for host in $oldhosts
do
echo "$host" " start"
( DISTCC_HOSTS="$host";
# using localhost is not wanted
export DISTCC_FALLBACK=0
t=` ( time distcc gcc $tmpfile -c -o /dev/null) 2>&1 `
echo -n $host " --- " $t
# no floating point in scrips
if [[ ${t%.*} -lt 6 ]];
then
echo $t $host >> $tmplog
echo
else
echo " [too slow -> skipped]"
fi
)&
done
echo waiting for subtasks
# jobs
wait
)
echo ready
DISTCC_HOSTS=` sort -n $tmplog | awk '{ print $2; }' `
echo DISTCC_HOSTS=$DISTCC_HOSTS
count=` wc -l $tmplog | sed -e "s@$tmplog@@"`
MAKEFLAGS=`echo "$MAKEFLAGS" | sed -e "s/-j[0-9]*/-j$count/"`
echo MAKEFLAGS=$MAKEFLAGS
#clean up, as this file has to be sourced
# rm $tmpfile
unset count
unset tmplog
---------------------------------------
More information about the distcc
mailing list