[distcc] DISTCC_POTENTIAL_HOSTS, localhost, and slots

[Dave Abrahams]

Wow, Jeff, that looks pretty awesome.  It hasn't been touched since 2006,
though; does it work with pump mode?


On Wed, May 14, 2014 at 8:00 AM, Jeff Domogala <jeff.domogala at oracle.com>wrote:

> Another suggestion if you don’t want to dig into the code…  I ran across
> DMUCS (http://dmucs.sourceforge.net) about 8 years ago when I was
> building up a farm of machines to be used by a couple hundred users.  DMUCS
> will manage the compute resources and is able to handle unavailability of
> resources.  I did make a few of my own tweaks to the source code for
> monitoring and other glitches I have run into along the way.  I’m still
> using DMUCS as my frontend for distcc and am very happy.  If there are any
> questions about my setup fire away.
>
>
>
> --Jeff
>
>
>
> *From:* Fergus Henderson [mailto:fergus at google.com]
> *Sent:* Wednesday, May 14, 2014 8:47 AM
> *To:* Dave Abrahams
> *Cc:* distcc at lists.samba.org
> *Subject:* Re: [distcc] DISTCC_POTENTIAL_HOSTS, localhost, and slots
>
>
>
> On Wed, May 14, 2014 at 6:16 AM, Dave Abrahams <dave at boostpro.com> wrote:
>
> DISTCC_POTENTIAL_HOSTS seems like a great idea; as I scale up the build
> farm I’d like to be able to deal with unavailability, but I’m having
> trouble getting it to work right.
>
>
>
> First, after much fooling around I surmise two undocumented facts(?)
>
>
>
> 1. it adds ,lzo,cpp to all the hosts implicitly
>
>
>
> Yes, that's done in ExportDISTCC_HOSTS function in the pump script (source
> file pump.in).
>
>
>
> 2. it only accepts bare host names, not the usual
> @?hostname/<slots>(,<extension>)* syntax
>
> Is that correct?
>
>
>
> Pretty much, yes.  It just passes the variable's value to lsdistcc, and
> lsdistcc accepts either bare host names, or a printf format string
> containing %d, e.g. "mydistcc%d" or "distcc%d.my.own.org.com", not *host
> specifications* as described the distcc man page.
>
>
>
> If so, it would be good if these things were documented explicitly
> somewhere.
>
>
>
> Agreed.  I would be happy to accept a patch to add such documentation.
>
>
>
> Another possibility would be to change lsdistcc to handle host
> specifications.
>
>
>
> Also, if I want localhost to be one of the hosts, it looks like I need to
> run distccd there.  That doesn’t seem like the most efficient thing to do,
> since normal local compilation will presumably have fewer layers to work
> through.  Am I mistaken?
>
>
>
> Well, you only need to run distccd for lsdistcc's initial query (lsdistcc
> doesn't have any special casing for localhost).
>
> But any actual compilations done will not go via the distccd on localhost;
> distcc itself does have special casing for localhost.
>
>
>
> I'd be happy to accept a patch to src/lsdistcc.c to make it special-case
> "localhost".
>
>
>
> Lastly, I’ve only been able to get it to run the default 6 (?) slots per
> host using DISTCC_POTENTIAL_HOSTS.  My .distcc/hosts file looks like this,
> and has been working pretty well:
>
>
>
> --randomize
>
> localhost/10
>
> bruford/10,lzo,cpp
>
> fripp/10,lzo,cpp
>
>
>
> Is there any way to get an effect like that using DISTCC_POTENTIAL_HOSTS,
> i.e. 10 slots per host, no local distccd?
>
>
>
> Not currently.  I imagine that it would require only very minor changes to
> the ExportDISTCC_HOSTS function in the pump script (source file pump.in),
> though.
>
>
>
> Cheers,
>
>    Fergus.
>
>
>
> --
>
> Fergus Henderson <fergus at google.com>
>
>
>
> "Defend the user, exclude no one, and create magic." -- Eric Schmidt.
>
>
>



-- 
Dave Abrahams
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.samba.org/pipermail/distcc/attachments/20140515/38753864/attachment.html>