[PATCH] Avoid inheriting *.stdout and *.stderr into every child process

[Andrew Bartlett]

On Tue, 2018-08-28 at 14:54 +1200, Gary Lockyer wrote:
> RB+

Thanks Gary.  

Metze:  I'm unclear if you still have concerns on this.  

If you do, can you get back to me with specific issues?

Otherwise I would like to push this in a couple of days, so as to allow
the python3 work to progress.  

As agreed, we need to split up the py2 and py3 builds.  This means more
tasks and more file descriptors.  While also just the right thing to
do, this primarily works around FD limits only found on sn-devel.

Thanks,

Andrew Bartlett

> On 27/08/18 22:43, Andrew Bartlett via samba-technical wrote:
> > On Mon, 2018-08-27 at 12:29 +0200, Stefan Metzmacher wrote:
> > > Hi Andrew,
> > > 
> > > > > I hope this clarifies,
> > > > 
> > > > You can see that --tail still works on any job in the CI link:
> > > > https://gitlab.com/catalyst-samba/samba/pipelines/28743555
> > > > 
> > > > eg specifically:
> > > > https://gitlab.com/catalyst-samba/samba/-/jobs/92279525
> > > > 
> > > > A clarified patch attached.
> > > 
> > > I don't have access there, can you use the public samba-team pipelines?
> > 
> > Sorry, I've re-submitted the job there:
> > 
> > https://gitlab.com/samba-team/devel/samba/pipelines/28752233
> > 
> > > It's still unclear for the cases, where we don't explicitly
> > > pass in all three values for stdout, stderr and stdin.
> > 
> > The documentation states:
> > 
> > https://docs.python.org/2.6/library/subprocess.html
> > 
> > "If close_fds is true, all file descriptors except 0, 1 and
> > 2 will be closed before the child process is executed. (Unix only)."
> > 
> > And regarding the passed in parameters:
> > 
> > "stdin, stdout and stderr specify the executed programs’ standard
> > input,
> > standard output and standard error file handles, respectively.  "
> > ...
> > 
> > "With None, no redirection will occur;
> > the child’s file handles will be inherited from the parent. "
> > 
> > We can also see that from the fact that the tail operates in the output
> > above.
> > 
> > I trust this addresses your concerns.  These are otherwise just random
> > high FDs.
> > 
> > If you are still not convinced, we can with much less fuss ignore this
> > and just have the buildnice script changed to allow 1024 FDs.  I have
> > found these represented in the child Samba processes (via lsof and
> > /proc/$pid/fd), and up to 480 FDs consumed at the time of my
> > inspection. 
> > 
> > Thanks,
> > 
> > Andrew Bartlett
> > 
> 
> 
-- 
Andrew Bartlett
https://samba.org/~abartlet/
Authentication Developer, Samba Team         https://samba.org
Samba Development and Support, Catalyst IT   
https://catalyst.net.nz/services/samba