[distcc] load management

[Dan Kegel]

mbp at sourcefrog.net wrote:
>>I've been toying with the following idea, which combines load balancing
>>with fsh-like connection caching.  Let's call it lbfsh.
> 
> I think that would be pretty handy, even if we end up improving
> the built in balancer.  In particular it would let people shift
> any old task onto a remote machine, which is a frequently
> requested feature for distcc.  
> 
> So it means just "run this somewhere, I don't care where", kind
> of like some cluster tools.

Right.

> Hardcode the socket to ~/.lbfsh/socket.  (Well, maybe have an
> option to override.)  If the client finds there's no daemon
> listening, it forks one.

I suppose.  Makes me a bit nervous somehow to have daemons
started behind my back, though.  If autoforking the daemon
was a good idea, the ssh guys would have had ssh-agent
autofork off of ssh, I bet.  Hey, come to think of it,
maybe ssh-agent should be the local daemon!  And, if you're
serious about using ssh for remote command execution from
scripts, you're probably running ssh-agent anyway.  (See e.g.
http://mah.everybody.org/docs/ssh )

> Rather than ignoring the host argument I think you should pass it
> through to the daemon, which uses it to select a group of hosts.
> If there is exactly one host, then this works like a faster
> version of fsh.  You might look up the hosts names either from a
> local configuration or a multi-A DNS record or a Rendezvous
> group. 

Yup.  It had occurred to me.

I like what you say in http://distcc.samba.org/faq.html#use-fsh
about "So to make this work well, somebody either needs to rewrite
the short-lived part of fsh in C, or write a replacement in C.
I think the SSH2 protocol in fact allows for this to be done natively,
so you could also hack OpenSSH to allow adding new channels within an existing connection."

http://www.ietf.org/internet-drafts/draft-ietf-secsh-connect-18.txt
certainly does make it sound like new sessions can be added on the fly.

The idea of taking openssh and adding fsh-like abilities
and load-management abilities right into it is appealing.

- Dan

-- 
My technical stuff: http://kegel.com
My politics: see http://www.misleader.org for examples of why I'm for regime change