[distcc] about distcc's scheduling

Francois Berenger berenger at riken.jp
Wed Nov 25 03:38:10 MST 2009


Francois Berenger wrote:
> Francois Berenger wrote:
>> Fergus Henderson wrote:
>>> Have you tried using pump mode?  That helps a lot.
>>>
>>> Also, if the issue is that distcc isn't running enough jobs on each 
>>> host, try using the "host/JOBS" format in DISTCC_HOSTS,
>>> e.g. DISTCC_HOSTS="foo.riken.jp/20,cpp,lzo 
>>> <http://foo.riken.jp/20,cpp,lzo> bar.riken.jp/40,cpp,lzo 
>>> <http://bar.riken.jp/40,cpp,lzo>".
>>> This would allow up to 20 concurrent compiles on host foo.riken.jp 
>>> <http://foo.riken.jp> and 40 concurrent compiles on host bar.riken.jp 
>>> <http://bar.riken.jp>.
>>> (The ",cpp,lzo" part is required for pump mode.)
>>
>> Thanks a lot! I am trying this right away.
>> I'll post my speedup results when I'm done.
> 
> Well, I managed to make our soft compile with:
> distcc-server-3.1-1
> distcc-3.1-1
> 
> However, pump seems to be for 32bit machines only:
> titi at toto:x# pump scons -j24 bin mode=release
> __________Using distcc-pump from /usr/bin
> Traceback (most recent call last):
>   File 
> "/usr/lib/python2.4/site-packages/include_server/include_server.py", 
> line 42, in ?
>     import distcc_pump_c_extensions
> ImportError: 
> /usr/lib/python2.4/site-packages/include_server/distcc_pump_c_extensions.so: 
> wrong ELF class: ELFCLASS32
> __________Expected a socket at '/tmp/distcc-pump.E15442/socket'
> __________Could not start distcc-pump include server
> __________Warning: distcc-pump include server failed; running unpumped
> __________Found 0 available distcc servers
> 
> Can someone publish x86_64 binary packages in the download section of 
> the distcc website?
> Or at least the source RPM so I can to build for my systems.

I was unable to build the rpm myself from the tar ball.
I give up, I fought enough.
Here is the make rpm output:

Checking for unpackaged file(s): /usr/lib/rpm/check-files 
/var/tmp/distcc-buildroot
error: Installed (but unpackaged) file(s) found:
    /usr/lib64/python2.4/site-packages/include_server/__init__.pyo
    /usr/lib64/python2.4/site-packages/include_server/basics.pyo
    /usr/lib64/python2.4/site-packages/include_server/basics_test.pyo
    /usr/lib64/python2.4/site-packages/include_server/c_extensions_test.pyo
    /usr/lib64/python2.4/site-packages/include_server/cache_basics.pyo
    /usr/lib64/python2.4/site-packages/include_server/compiler_defaults.pyo
    /usr/lib64/python2.4/site-packages/include_server/compress_files.pyo
    /usr/lib64/python2.4/site-packages/include_server/include_analyzer.pyo
 
/usr/lib64/python2.4/site-packages/include_server/include_analyzer_memoizing_node.pyo
 
/usr/lib64/python2.4/site-packages/include_server/include_analyzer_memoizing_node_test.pyo
 
/usr/lib64/python2.4/site-packages/include_server/include_analyzer_test.pyo
    /usr/lib64/python2.4/site-packages/include_server/include_server.pyo
 
/usr/lib64/python2.4/site-packages/include_server/include_server_test.pyo
    /usr/lib64/python2.4/site-packages/include_server/macro_eval.pyo
    /usr/lib64/python2.4/site-packages/include_server/macro_eval_test.pyo
    /usr/lib64/python2.4/site-packages/include_server/mirror_path.pyo
    /usr/lib64/python2.4/site-packages/include_server/mirror_path_test.pyo
    /usr/lib64/python2.4/site-packages/include_server/parse_command.pyo
    /usr/lib64/python2.4/site-packages/include_server/parse_command_test.pyo
    /usr/lib64/python2.4/site-packages/include_server/parse_file.pyo
    /usr/lib64/python2.4/site-packages/include_server/parse_file_test.pyo
    /usr/lib64/python2.4/site-packages/include_server/run.pyo
    /usr/lib64/python2.4/site-packages/include_server/setup.pyo
    /usr/lib64/python2.4/site-packages/include_server/statistics.pyo


RPM build errors:
     Installed (but unpackaged) file(s) found:
    /usr/lib64/python2.4/site-packages/include_server/__init__.pyo
    /usr/lib64/python2.4/site-packages/include_server/basics.pyo
    /usr/lib64/python2.4/site-packages/include_server/basics_test.pyo
    /usr/lib64/python2.4/site-packages/include_server/c_extensions_test.pyo
    /usr/lib64/python2.4/site-packages/include_server/cache_basics.pyo
    /usr/lib64/python2.4/site-packages/include_server/compiler_defaults.pyo
    /usr/lib64/python2.4/site-packages/include_server/compress_files.pyo
    /usr/lib64/python2.4/site-packages/include_server/include_analyzer.pyo
 
/usr/lib64/python2.4/site-packages/include_server/include_analyzer_memoizing_node.pyo
 
/usr/lib64/python2.4/site-packages/include_server/include_analyzer_memoizing_node_test.pyo
 
/usr/lib64/python2.4/site-packages/include_server/include_analyzer_test.pyo
    /usr/lib64/python2.4/site-packages/include_server/include_server.pyo
 
/usr/lib64/python2.4/site-packages/include_server/include_server_test.pyo
    /usr/lib64/python2.4/site-packages/include_server/macro_eval.pyo
    /usr/lib64/python2.4/site-packages/include_server/macro_eval_test.pyo
    /usr/lib64/python2.4/site-packages/include_server/mirror_path.pyo
    /usr/lib64/python2.4/site-packages/include_server/mirror_path_test.pyo
    /usr/lib64/python2.4/site-packages/include_server/parse_command.pyo
    /usr/lib64/python2.4/site-packages/include_server/parse_command_test.pyo
    /usr/lib64/python2.4/site-packages/include_server/parse_file.pyo
    /usr/lib64/python2.4/site-packages/include_server/parse_file_test.pyo
    /usr/lib64/python2.4/site-packages/include_server/run.pyo
    /usr/lib64/python2.4/site-packages/include_server/setup.pyo
    /usr/lib64/python2.4/site-packages/include_server/statistics.pyo

> Thanks a lot,
> Francois.
> 
>>> At some point, depending on the size and structure of the software 
>>> that you're trying to compile,
>>> and the way your Makefiles are arranged, you may run into limitations 
>>> in the parallelizability of your build graph.
>>> For example, if your software has only three source files, then 
>>> distcc won't be able to speed up compilation by more than a factor of 
>>> three at very best.
>>> Similarly, if your software has 30 files, but they are split into 10 
>>> directories of three files each, and your Makefile(s) sequentialize 
>>> the compilation of each directory, then you still won't get more than 
>>> a factor of three speed up at very best.
>>
>> We use scons. I am not so happy with it (like when it spends 30s 
>> before giving me back control of the command line when theres is 
>> _nothing_ to compile because everything is already done).
>> However, it manages the dependencies far better than any Makefile I 
>> could have written by hand.
>>
>> Regards,
>> Francois.
>>
>>> On Mon, Nov 23, 2009 at 8:19 PM, Francois Berenger <berenger at riken.jp 
>>> <mailto:berenger at riken.jp>> wrote:
>>>
>>>     Hello,
>>>
>>>     Is there a way to use a pull scheduler instead of a push scheduler
>>>     inside distcc?
>>>
>>>     I have many compiling hosts and when they join a compiling job, 
>>> they are
>>>     not enough loaded, at least to my taste. Meaning: they could compile
>>>     more things and I could wait less for them to finish.
>>>
>>>     I think the problem is that the scheduler is pushing: the distcc
>>>     server sends jobs to distcc compiling hosts.
>>>
>>>     Instead, I would like this behavior: distcc compiling hosts ask and
>>>     take jobs as long as they are not completely loaded with compilation
>>>     jobs.
>>>     The distcc server should distribute jobs to whom asks for.
>>>
>>>     This should scale better.
>>>
>>>     If there is no way now to do so but you can give me some hint on 
>>> how to
>>>     do so (where to look, what to modify, etc.), it might be helpful.
>>>
>>>     Best regards,
>>>     Francois.
>>>
>>>     __
>>>     distcc mailing list            http://distcc.samba.org/
>>>     To unsubscribe or change options:
>>>     https://lists.samba.org/mailman/listinfo/distcc
>>>
>>>
>>>
>>>
>>> -- 
>>> Fergus Henderson <fergus at google.com <mailto:fergus at google.com>>
>>
>> __
>> distcc mailing list            http://distcc.samba.org/
>> To unsubscribe or change options:
>> https://lists.samba.org/mailman/listinfo/distcc
> 
> 



More information about the distcc mailing list