<div dir="ltr">Wow, Jeff, that looks pretty awesome. It hasn't been touched since 2006, though; does it work with pump mode?</div><div class="gmail_extra"><br><br><div class="gmail_quote">On Wed, May 14, 2014 at 8:00 AM, Jeff Domogala <span dir="ltr"><<a href="mailto:jeff.domogala@oracle.com" target="_blank">jeff.domogala@oracle.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div lang="EN-US" link="blue" vlink="purple"><div><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Another suggestion if you don’t want to dig into the code… I ran across DMUCS (<a href="http://dmucs.sourceforge.net" target="_blank">http://dmucs.sourceforge.net</a>) about 8 years ago when I was building up a farm of machines to be used by a couple hundred users. DMUCS will manage the compute resources and is able to handle unavailability of resources. I did make a few of my own tweaks to the source code for monitoring and other glitches I have run into along the way. I’m still using DMUCS as my frontend for distcc and am very happy. If there are any questions about my setup fire away.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">--Jeff<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p><p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> Fergus Henderson [mailto:<a href="mailto:fergus@google.com" target="_blank">fergus@google.com</a>] <br>
<b>Sent:</b> Wednesday, May 14, 2014 8:47 AM<br><b>To:</b> Dave Abrahams<br><b>Cc:</b> <a href="mailto:distcc@lists.samba.org" target="_blank">distcc@lists.samba.org</a><br><b>Subject:</b> Re: [distcc] DISTCC_POTENTIAL_HOSTS, localhost, and slots<u></u><u></u></span></p>
<div><div class="h5"><p class="MsoNormal"><u></u> <u></u></p><div><div><div><p class="MsoNormal">On Wed, May 14, 2014 at 6:16 AM, Dave Abrahams <<a href="mailto:dave@boostpro.com" target="_blank">dave@boostpro.com</a>> wrote:<u></u><u></u></p>
<div><p class="MsoNormal">DISTCC_POTENTIAL_HOSTS seems like a great idea; as I scale up the build farm I’d like to be able to deal with unavailability, but I’m having trouble getting it to work right.<u></u><u></u></p><div>
<p class="MsoNormal"><u></u> <u></u></p></div><div><p class="MsoNormal">First, after much fooling around I surmise two undocumented facts(?)<u></u><u></u></p></div><div><p class="MsoNormal"><u></u> <u></u></p></div><div>
<p class="MsoNormal">
1. it adds ,lzo,cpp to all the hosts implicitly<br clear="all"><u></u><u></u></p></div></div><div><p class="MsoNormal"><u></u> <u></u></p></div><div><p class="MsoNormal">Yes, that's done in ExportDISTCC_HOSTS function in the pump script (source file <a href="http://pump.in" target="_blank">pump.in</a>).<u></u><u></u></p>
</div><div><p class="MsoNormal"> <u></u><u></u></p></div><blockquote style="border:none;border-left:solid #cccccc 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in"><div><div><div><p class="MsoNormal">2. it only accepts bare host names, not the usual @?hostname/<slots>(,<extension>)* syntax <u></u><u></u></p>
</div></div></div></blockquote><div><blockquote style="border:none;border-left:solid #cccccc 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in"><div><p class="MsoNormal">Is that correct?<u></u><u></u></p>
</div></blockquote><div><p class="MsoNormal"> <u></u><u></u></p></div></div><div><p class="MsoNormal">Pretty much, yes. It just passes the variable's value to lsdistcc, and lsdistcc accepts either bare host names, or a printf format string containing %d, e.g. "mydistcc%d" or "distcc%<a href="http://d.my.own.org.com" target="_blank">d.my.own.org.com</a>", not <i>host specifications</i> as described the distcc man page.<u></u><u></u></p>
</div><div><p class="MsoNormal"><u></u> <u></u></p></div><blockquote style="border:none;border-left:solid #cccccc 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in"><div><div><div><p class="MsoNormal">If so, it would be good if these things were documented explicitly somewhere.<u></u><u></u></p>
</div></div></div></blockquote><div><p class="MsoNormal"><u></u> <u></u></p></div><div><p class="MsoNormal">Agreed. I would be happy to accept a patch to add such documentation.<u></u><u></u></p></div><div><p class="MsoNormal">
<u></u> <u></u></p></div><div><p class="MsoNormal">Another possibility would be to change lsdistcc to handle host specifications.<u></u><u></u></p></div><div><p class="MsoNormal"> <u></u><u></u></p></div><blockquote style="border:none;border-left:solid #cccccc 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<div><div><div><p class="MsoNormal">Also, if I want localhost to be one of the hosts, it looks like I need to run distccd there. That doesn’t seem like the most efficient thing to do, since normal local compilation will presumably have fewer layers to work through. Am I mistaken?<u></u><u></u></p>
</div></div></div></blockquote><div><p class="MsoNormal"><u></u> <u></u></p></div><div><p class="MsoNormal">Well, you only need to run distccd for lsdistcc's initial query (lsdistcc doesn't have any special casing for localhost).<u></u><u></u></p>
</div><div><p class="MsoNormal">But any actual compilations done will not go via the distccd on localhost; distcc itself does have special casing for localhost. <u></u><u></u></p></div><div><p class="MsoNormal"><u></u> <u></u></p>
</div><div><p class="MsoNormal">I'd be happy to accept a patch to src/lsdistcc.c to make it special-case "localhost".<u></u><u></u></p></div><div><p class="MsoNormal"> <u></u><u></u></p></div><blockquote style="border:none;border-left:solid #cccccc 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<div><div><div><p class="MsoNormal">Lastly, I’ve only been able to get it to run the default 6 (?) slots per host using DISTCC_POTENTIAL_HOSTS. My .distcc/hosts file looks like this, and has been working pretty well:<u></u><u></u></p>
</div><div><p class="MsoNormal"><u></u> <u></u></p></div></div><blockquote style="margin-left:30.0pt;margin-right:0in"><div><div><p class="MsoNormal">--randomize<u></u><u></u></p></div></div><div><div><p class="MsoNormal">
localhost/10<u></u><u></u></p></div></div><div><div><p class="MsoNormal">bruford/10,lzo,cpp<u></u><u></u></p></div></div><div><div><p class="MsoNormal">fripp/10,lzo,cpp<u></u><u></u></p></div></div></blockquote><div><div>
<p class="MsoNormal"> <u></u><u></u></p></div><div><p class="MsoNormal">Is there any way to get an effect like that using DISTCC_POTENTIAL_HOSTS, i.e. 10 slots per host, no local distccd? <u></u><u></u></p></div></div></div>
</blockquote><div><p class="MsoNormal"><u></u> <u></u></p></div><div><p class="MsoNormal">Not currently. I imagine that it would require only very minor changes to the ExportDISTCC_HOSTS function in the pump script (source file <a href="http://pump.in" target="_blank">pump.in</a>), though.<u></u><u></u></p>
</div><div><p class="MsoNormal"><u></u> <u></u></p></div></div><p class="MsoNormal">Cheers,<u></u><u></u></p></div><div><p class="MsoNormal"> Fergus.<br clear="all"><u></u><u></u></p><div><p class="MsoNormal"><u></u> <u></u></p>
</div><p class="MsoNormal">-- <u></u><u></u></p><div><p class="MsoNormal">Fergus Henderson <<a href="mailto:fergus@google.com" target="_blank">fergus@google.com</a>><u></u><u></u></p></div><div><p class="MsoNormal">
<u></u> <u></u></p></div><div><div><p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif"">"Defend the user, exclude no one, and create magic." -- Eric Schmidt.</span><u></u><u></u></p>
</div></div><div><p class="MsoNormal"><u></u> <u></u></p></div></div></div></div></div></div></div></blockquote></div><br><br clear="all"><div><br></div>-- <br><div dir="ltr">Dave Abrahams<br><br></div>
</div>