[distcc] Contributing to distcc / Massively parallel compilations

[ozlabs]

Victor Norman wrote:

>Assaf, et al.,
>
>Sometime back I posted a message about "My big plans for distcc".  I have
>implemented most of those plans and they are in use here at Marconi.  I have a
>compilation farm of linux and solaris boxes of varying CPUs speeds and with
>varying numbers of cpus each.  I have also designated one solaris box as my
>"host server" and have a program running there that gives out hosts to
>compilations, keeps track of which machines ("hosts") in the compilation farm
>are available/accessible, gives out status information to other programs, etc.
>
>If you look at that previous posting, you'll see my goals for the project,
>which include (as I recall):
>
>o having a heterogenous system (solaris, linux, etc.)
>o having the system be load-balancing.
>o being able to add and remove hosts from the compilation according to the
>machine's load average, whether or not it is in use as a desktop, etc.
>o giving out the fastest machines first, so that compilations are the fastest
>possible.
>o supporting many compilations simultaneously from multiple machines.
>  
>
The other big thing I'd like is to automatically check versions, making 
sure that jobs are only sent to machines which have a matching gcc and 
binutils (as gregkh just discovered). 

>Now, here is the crucial part that may interest you, Assaf and friends: the
>code is all written in python, which is a wonderful language, IMO.  But, as Dan
>Kegel has suggested, the system may be more widely used, and easier to install
>if it were written in C.  I agree with him.
>  
>
I love Python too; I would be happy to ship supporting programs written 
in it.  But ideally this should be built right into distcc, and 
therefore probably in C.

I wonder if it would be feasible to automatically discover the machines 
speeds.  It would be OK if there were an OS-dependent bit that looked in 
/proc or similar.

I don't think this would be mere mechanical porting; students could do 
new development using Victor's code as a prototype.

-- 
Martin