[distcc] Distcc is not scalling linearly , Pls help

[Martin Pool]

You're going to need to look at what processes are running during the
build. Are the servers busy? Maybe there are lots of non cc jobs?

On Friday, March 7, 2014 12:44:21 AM, yudhistir reddy <
yudhistir.reddy at gmail.com> wrote:

Hi Martin,

Thank you . It is helpful . I can see now, more no of CPUs are used during
build . But i am sad that its not reducing the build time .  Am i missing
something ? Any suggestion to check with to reduce build time with distcc .


My standalone build on "client" machine with -j32 takes around 66 mins
without DISTCC .

Data from Distcc Normal Mode:

I captured data with different -j value and used your suggestion to
increase the CPUs during build .

Client machinesServer machines
(distccd running)j valueCPUs used during buildtarget nameBuild time
(Secs)MinsClientbuild01 build02
client24build01/build02/Client:8final-build375062.5clientbuild01 build02
client32not captured exactlyfinal-build360960.15clientbuild01 build02
client64build01/build02: 21 , Client:20final-build380863.46667clientbuild01
build02 client144build01/build02: 50 , Client:32final-build510685.1











Configuration:



Client machine ( 16 CPU , dual socket , Linux machine)

build01/build02(64 CPU , 4 socket, Linux machine)















DISTCC_HOSTS=build01/128 build02/128 client/32








Regards

Yudhistir

On Thu, Mar 6, 2014 at 4:51 AM, Martin Pool <martinpool@<martinpool at gmail.com>
gmail.com <martinpool at gmail.com>> wrote:

distcc defaults to sending only 4 jobs to each server. (Hosts have many
more cores than they did in ~2002 when I started this.) You can change it
by setting eg

DISTCC_HOSTS='bigmachine/16 biggermachine/100'

http:// <http://distcc.googlecode.com/svn/trunk/doc/web/man/distcc_1.html>
distcc.googlecode.com<http://distcc.googlecode.com/svn/trunk/doc/web/man/distcc_1.html>
/ <http://distcc.googlecode.com/svn/trunk/doc/web/man/distcc_1.html>svn<http://distcc.googlecode.com/svn/trunk/doc/web/man/distcc_1.html>
/trunk/doc/web/man/<http://distcc.googlecode.com/svn/trunk/doc/web/man/distcc_1.html>
distcc_1 <http://distcc.googlecode.com/svn/trunk/doc/web/man/distcc_1.html>.<http://distcc.googlecode.com/svn/trunk/doc/web/man/distcc_1.html>
html <http://distcc.googlecode.com/svn/trunk/doc/web/man/distcc_1.html>

On Wed Mar 05 2014 at 10:26:25 PM, yudhistir reddy
<yudhistir.reddy<yudhistir.reddy at gmail.com>
@ <yudhistir.reddy at gmail.com>gmail.com <yudhistir.reddy at gmail.com>> wrote:

 Hi ,

I am compiling code with distcc . Below is my set up

Client machine : Linux (ubuntu) , 16 core dual socket ( logical 32 core )

Time takes  to build code on this machine : 1 hour  with -j32 .

with Distcc :

Client machine is same as above one

2 servers with distccd daemon running ( 64 cores each )

Same target takes almost same time or just 5 mins less .  I am running the
make with -j32 on client to distribute the jobs .

I am using DISTCC normal mode , writting distcclog file to local directory
only .

I set DISTCC_DIR to local directory . i can see it uses only 4 CPUs of each
machine . How i can scale it with distcc . why distcc is not using more
CPUs from other two servers as well as local host .

Is something missing in my part  ? Pls help .

Regards

Yudhistir

__
distcc mailing list            http:// <http://distcc.samba.org/>
distcc.samba.org/ <http://distcc.samba.org/>
To unsubscribe or change options:
https <https://lists.samba.org/mailman/listinfo/distcc>://<https://lists.samba.org/mailman/listinfo/distcc>
lists.samba.org
<https://lists.samba.org/mailman/listinfo/distcc>/mailman/<https://lists.samba.org/mailman/listinfo/distcc>
listinfo <https://lists.samba.org/mailman/listinfo/distcc>/<https://lists.samba.org/mailman/listinfo/distcc>
distcc <https://lists.samba.org/mailman/listinfo/distcc>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.samba.org/pipermail/distcc/attachments/20140306/e17a3fe4/attachment.html>