[distcc] Re: Announcing distccKNOPPIX

[Marcelo Matus]

Alexandre Oliva wrote:

>On Jul  8, 2003, Martin Pool <mbp at samba.org> wrote:
>
>  
>
>>Yes, but it remembers that the host is down, and won't try it for
>>another minute.
>>    
>>
>
>Yeah, this was a great improvement, thanks for that!
>
>  
>
>>I'd like to see somebody right a standalone program that does mDNS or
>>an nmap scan and creates a ~/.distcc/hosts file based on the results.
>>    
>>
>
>~/.distcc/hosts doesn't quite cut it for me.  Because of the way I use
>cross, native or cross-native (*) toolchains, the same set of hosts
>may or may not work, depending on what CC I'm using, for example, the
>system's GCC, that varies depending on the exact version of Red Hat
>Linux, sometimes betas or even rawhide I'm running on each of my
>hosts; or GNU Pro natives or crosses that work identically on all of
>them.  It would be nice if I could express this kind of dependency on
>in DISTCC_HOSTS or ~/.distcc/hosts somehow.
>
>That said, if instead of rebuilding ~/.distcc/hosts, it would just
>mark in a separate file certain hosts as unusable for the time being,
>or unusable for some particular CC, or something like that, I could
>probably cobble up a shell script or something to that effect.
>
>The need for per-host validation of CC compatibility reminded me of an
>issue I had when trying to play with distcc over (fsh/)ssh (fsh would
>cut down connection set up time) was that, since it wouldn't set up
>the PATH like a login session would, it would end up not finding
>soft-links to compilers that I had in my PATH on the build machine,
>but that would be correctly found in the distccd environment when I
>started distccd in a login session.  I couldn't find a clean solution
>for this, so I ended up giving up on distcc over ssh :-(  Anyway, if
>distcc (or distccd?) could source a script to set up an environment
>before starting the compiler on the remote host, this would help.
>~/.distcc/env?  Alternatively, it could support mapping compiler
>pathnames to host-dependent full pathnames, but this doesn't sound as
>useful or generic as a server-side env setting.
>
>Thoughts?
>
>  
>
Hmmm, I have more problems with my path using the distccd server than 
over ssh...

However, here are two scripts that can be useful:

1.- Install 'denv' in all the clients (probably as /usr/local/bin/denv)

2.- Install 'dssh' in the master machine and set the new ssh command:

              export DISTCC_SSH=`which dssh`

      and the remote location of 'denv' (if it is different from 
/usr/local/bin/denv)

            export DENVCMD=/usr/local/bin/denv
            
      If you want to use a different remote nice value, set

            export RMTNICE=4  # if you want 


3.- run as usual.



Marcelo



-------------- next part --------------
#!/bin/bash

case $1 in
  PWD=*)
     dir=${1:4}
     shift
     ;;
  *)
     dir=${TMPDIR}
     ;;
esac 

if [ -d $dir ]; then
  cd $dir
else
 if [ -d $TMPDIR ]; then
   cd $TMPDIR
 fi
fi

lenv=$1
shift

args=`echo $lenv |          \
      tr '\21' ' ' |        \
      sed  -e "s/@q/'/g"    \
           -e 's/@t/	/g' \
           -e 's/@s/ /g'    \
           -e 's/@c/;/g'    \
           -e 's/@p/|/g' |  \
      tr '\20' '@' `

exec env "$args" $*




-------------- next part --------------
#!/bin/bash
#
# run a command in the remote node at the current directory and preserving
# the local environment, ie, the command
#    
#    dssh wsxi01 ls
#
# will list the directory from where the local 'runon' command is
# called, but the remote command 'ls' will be executed in the remote
# machine, and it will be resolved by using the same local PATH
# variable.
#
# Of course, this suppose that both machines (local and remote) have
# the same directory structure.
# 
#


DEFNICE=0
RMTENVVARS='PATH'
IGNENVVARS='XAUTHORITY|DISPLAY|USER|HOME|LOGNAME|HOSTNAME|WINDOWID|SESSION_MANAGER|^_='


sshcmd=${SSHCMD:=/usr/bin/ssh}
denvcmd=${DENVCMD:=/usr/local/bin/denv}
dir=${RMTDIR:=$""}
nice=${RMTNICE:=${DEFNICE}}

cmd=
host=

while [ "$1" ]; do
  case $1 in
    -[nAaXxtTvVPqfCNg1246s])
         if [ -z "$cmd" ]; then
            ssho="$ssho $1"
         else
            cmd="$cmd $1"
         fi         
         shift 
         ;;
    -[liecmpLRo])
         if [ -z "$cmd" ]; then
            ssho="$ssho $1 $2"
            shift
         else
            cmd="$cmd $1"
         fi         
         shift 
         ;;
    -nice)
         if [ -z "$cmd" ]; then
            nice=$2
            shift
         else
            cmd="$cmd $1"
         fi         
         shift 
         ;;
    -dir)
         if [ -z "$cmd" ]; then
            dir=$2
            shift
         else
            cmd="$cmd $1"
         fi         
         shift 
         ;;
     -local)
         if [ -z "$cmd" ]; then
            dir=`pwd`
         else
            cmd="$cmd $1"
         fi         
         shift 
         ;;
     *)
         if [ -z "$host" ]; then
            host=$1
         else
            cmd="$cmd $1"
         fi
         shift
         ;;
   esac
done


if [ -z "$host" -o -z "$cmd" ]; then
   echo "Usage: "`basename $0`" <host name> [-local|-dir <dir name>] [-nice <level>] [<ssh options>] cmd"
   echo "example: "
   echo "  " `basename $0` "rmthost pwd"
   exit 1
fi



lenv=`env | egrep -v $IGNENVVARS | grep $RMTENVVARS|\
      tr '@' '\20' | \
      sed -e "s/=\(.*\)$/='\1'/g" \
          -e "s/'/@q/g" \
          -e 's/ /@s/g' \
          -e 's/	/@t/g' \
          -e 's/;/@c/g' \
          -e 's/|/@p/g' | \
      tr '\n' '\21' `

exec $sshcmd -C $ssho $host nice -$nice $denvcmd PWD=${dir} $lenv $cmd