[distcc] A Few Ideas

[ibaker at mail.cern.ch]

On Fri, 24 Apr 2009, Fergus Henderson wrote:

> On Fri, Apr 24, 2009 at 8:52 AM, <ibaker at mail.cern.ch> wrote:
>
>> Good evening, I would appreciate any feedback on the following.
>>
>> Our initial patch submission for implementing GSS-API mutual authentication
>> used an environment variable ($DISTCC_AUTH) and as such was a global option
>> for all hosts.
>>
>> We have another version where this is now a per host option along the lines
>> of lzo or cpp (,gssapi).  This enables a client to use mixed
>> authenticating/non-authenticating build hosts, and allows for easier
>> deployment for users as there are fewer environment variables to set up.
>
>
> That sounds reasonable.
>
> Is there a simple way for the client to detect when a server requires
> authentication,
> so that this can be done automatically be "lsdistcc"?

The simplest way to determine if a server requires authentication is 
to transmit the handshake character and see if one gets transmitted back.

>
> We also currently use DNS aliases to identify a build cluster or sub-cluster
>> and replace its host definition with a host definition for each of its IP
>> addresses if the host option of ,exp is specified (this is to avoid a
>> performance cap imposed by the lockfile names and to utilise the
>> randomisation done by DNS).
>
>
> Could you explain that in more detail?
>

Sure.  Currently we have a hostname of the form:

distcc-slc4-amd64.cern.ch/8

The set of IP addresses for this hostname represents the build cluster for
this particular platform.  Our patch replaces the dcc_hostdef list item
for distcc-slc4-amd64.cern.ch with a dcc_hostdef for each one of the IP
addresses in h_addr_list, replacing the hostname in the new list entry
with an IP address.

Randomisation as per --randomize takes place prior to this expansion so
that any randomisation done by DNS is preserved.

The rationale behind this patch is primarily to overcome the performance
ceiling imposed by the lockfile names.  Specifically, using the above
hostname, when we have 8 lockfiles created all other processes become
blocked.

Overcoming this performance ceiling and the additional randomisation
performed by DNS seemed like a good alternative to changing the lockfile
naming mechanism.


Ian