[distcc] Web based compiling farm

[Antenore Gatta]

Thanks a lot Martin.

We are working on the first stage, so we have a lot of time to think
about LSF problems.
Probably the best way could be to choose a distrib as starting point.
Like Gentoo stage 3...

I'm totaly agree with you, security is the most important matter of
this project and we have some ideas about that, I'll let you know
about our progress.

Regarding the compile report I need to know in advance which is the
network troughput. Imagine that a new user join to us for the first
time, where can I put this host? Which is the nearest compiling farm
from it?

I know, it is quite impossible to answer to the question "how far is
it", sending an image is quite simple and 80 to 100 times works fine,
we have tried to send (several times) an image 100 times in 60 sec.
loop.

By the way, we have to do so much work. If you are interested I'll
inform you with important update/news.

Regards,
Antenore.

2005/5/4, Martin Pool <mbp at sourcefrog.net>:
> On Wed, 2005-05-04 at 10:23 +0200, Antenore Gatta wrote:
> 
> > My goal is to create a distributed system for compiling software on
> > demand for not experienced users.
> > In the first stage I'd like to have a web based form where the user
> > can choose from a software list and between some different platforms
> > (amd, intel, sparc, arm, ..., etc.) after N minutes user should be
> > able to download a tar archive with the platform optimized compiled
> > version.
> 
> I think inexperienced users are probably better off downloading rpm or
> deb packages, not tarballs.
> 
> > In my secret dreams (a madness!), besides previously described goals,
> > I'd like to have a system to compile a complete Linux From Scratch
> > system with an iso image as result.
> 
> I think it would be technically possible.  Whether there is any value is
> another question...  (I think the value in LFS is in what you learn from
> doing it.)
> 
> > Every users in exchange for compiled software should lend their CPU
> > time, so, here finally the questions:
> 
> I hope you understand the security problems with that?
> 
> > How can I know how far are every hosts between them?
> > How can I dinamically group some hosts considering the "network distance"?
> >
> > I tried with ICMP and anycast but both doesn't pass trough every routers.
> > We have obtained some result sending a little image (png) via ssh
> > counting the seconds, b ut sometimes the routing change unexpectedly.
> > Perhaps using the same distcc socket the routing should be the same
> > (except for router/backbone problems).
> 
> You need to consider several variables: latency, bandwidth and compile
> speed.
> 
> > Please do you have an idea about this?
> > Should be possible and usefull to implement a network distance
> > calculator inside distcc?
> 
> Perhaps the most useful thing is to have it report on the speed of
> compiling on different hosts?
> 
> If you're building a whole distribution it may be easier to distribute
> whole packages to different hosts.  (But if you're building along the
> lines of LFS perhaps that's not possible.)
> 
> --
> Martin
> 
> 
>