[distcc] load management

Dan Kegel dank at kegel.com
Sun May 2 06:09:25 GMT 2004

mbp at sourcefrog.net wrote:
>>I've been toying with the following idea, which combines load balancing
>>with fsh-like connection caching.  Let's call it lbfsh.
> I think that would be pretty handy, even if we end up improving
> the built in balancer.  In particular it would let people shift
> any old task onto a remote machine, which is a frequently
> requested feature for distcc.  
> So it means just "run this somewhere, I don't care where", kind
> of like some cluster tools.


> Hardcode the socket to ~/.lbfsh/socket.  (Well, maybe have an
> option to override.)  If the client finds there's no daemon
> listening, it forks one.

I suppose.  Makes me a bit nervous somehow to have daemons
started behind my back, though.  If autoforking the daemon
was a good idea, the ssh guys would have had ssh-agent
autofork off of ssh, I bet.  Hey, come to think of it,
maybe ssh-agent should be the local daemon!  And, if you're
serious about using ssh for remote command execution from
scripts, you're probably running ssh-agent anyway.  (See e.g.
http://mah.everybody.org/docs/ssh )

> Rather than ignoring the host argument I think you should pass it
> through to the daemon, which uses it to select a group of hosts.
> If there is exactly one host, then this works like a faster
> version of fsh.  You might look up the hosts names either from a
> local configuration or a multi-A DNS record or a Rendezvous
> group. 

Yup.  It had occurred to me.

I like what you say in http://distcc.samba.org/faq.html#use-fsh
about "So to make this work well, somebody either needs to rewrite
the short-lived part of fsh in C, or write a replacement in C.
I think the SSH2 protocol in fact allows for this to be done natively,
so you could also hack OpenSSH to allow adding new channels within an existing connection."

certainly does make it sound like new sessions can be added on the fly.

The idea of taking openssh and adding fsh-like abilities
and load-management abilities right into it is appealing.

- Dan

My technical stuff: http://kegel.com
My politics: see http://www.misleader.org for examples of why I'm for regime change

More information about the distcc mailing list