[PATCHES] selftest related

Michael Adam obnox at samba.org
Fri Oct 17 03:26:55 MDT 2014


Hi,

I just applied and tested these patches, and there is an issue:

If e.g. do

$ buildnice make test TESTS=messaging

then I get:

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[0/8 in 0s] samba.tests.messaging
[1/8 in 0s] samba3.smbtorture_s3.LOCAL-MESSAGING-READ1
[2/8 in 0s] samba3.smbtorture_s3.LOCAL-MESSAGING-READ2
[3/8 in 0s] samba3.smbtorture_s3.LOCAL-MESSAGING-READ3
[4/8 in 0s] samba3.smbtorture_s3.LOCAL-MESSAGING-READ4
[5/8 in 0s] samba3.smbtorture_s3.LOCAL-MESSAGING-FDPASS1
[6/8 in 0s] samba3.smbtorture_s3.LOCAL-MESSAGING-FDPASS2
[7/8 in 0s] samba4.local.messaging

ALL OK (7 tests in 8 testsuites)

A summary with detailed information can be found in:
  ./st/summary
TOP 10 slowest tests
Traceback (most recent call last):
  File "./buildtools/bin/waf", line 76, in <module>
    Scripting.prepare(t, cwd, VERSION, wafdir)
  File "/data/obnox/devel/samba/master-push.git/buildtools/wafadmin/Scripting.py", line 145, in prepare
    prepare_impl(t, cwd, ver, wafdir)
  File "/data/obnox/devel/samba/master-push.git/buildtools/wafadmin/Scripting.py", line 135, in prepare_impl
    main()
  File "/data/obnox/devel/samba/master-push.git/wscript", line 347, in main
    wildcard_main(wildcard_cmd)
  File "./buildtools/wafsamba/samba_wildcard.py", line 110, in wildcard_main
    fun(ctx)
  File "/data/obnox/devel/samba/master-push.git/selftest/wscript", line 241, in cmd_testonly
    cmd = "./script/show_testsuite_time %s/subunit %d" % (Options.options.selftest_prefix, nb)
AttributeError: Values instance has no attribute 'selftest_prefix'
make: *** [test] Error 1

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Therefore I fixed the patch "selftest: report the 10 slowest tests..."
to read  Options.options.SELFTEST_PREFIX instead of
Options.options.selftest_prefix.

One other point is this:
I am not convinced of the patch that changed the test number reported.
I always understood the output [X/Y in Zs] as
"now starting test X out of Y after Z seconds of test run".
because it is prepended to the test started just now.
When tests are executed sequentially, your patch to report
"I have run X tests out of Y in Z seconds" makes some sense,
but I find it confusing still.

I would propose to remove the "in" from "in Zs" and e.g. instead
say "at Zs" (or "after Zs"m but that seems to long).

so then the output of the above run reads:

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[1/8 at 0s] samba.tests.messaging
[2/8 at 0s] samba3.smbtorture_s3.LOCAL-MESSAGING-READ1
[3/8 at 0s] samba3.smbtorture_s3.LOCAL-MESSAGING-READ2
[4/8 at 0s] samba3.smbtorture_s3.LOCAL-MESSAGING-READ3
[5/8 at 0s] samba3.smbtorture_s3.LOCAL-MESSAGING-READ4
[6/8 at 0s] samba3.smbtorture_s3.LOCAL-MESSAGING-FDPASS1
[7/8 at 0s] samba3.smbtorture_s3.LOCAL-MESSAGING-FDPASS2
[8/8 at 0s] samba4.local.messaging

ALL OK (7 tests in 8 testsuites)

A summary with detailed information can be found in:
  ./st/summary
TOP 10 slowest tests
samba4.local.messaging -> 1
samba3.smbtorture_s3.LOCAL-MESSAGING-FDPASS1 -> 0
samba3.smbtorture_s3.LOCAL-MESSAGING-READ2 -> 0
samba3.smbtorture_s3.LOCAL-MESSAGING-READ4 -> 0
samba3.smbtorture_s3.LOCAL-MESSAGING-FDPASS2 -> 0
samba3.smbtorture_s3.LOCAL-MESSAGING-READ3 -> 0
samba3.smbtorture_s3.LOCAL-MESSAGING-READ1 -> 0
samba.tests.messaging -> -1413544791

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Proposed alternative patch attached.

I am going to push all patches (including my patch) except for this
reporting patch for a start and leave that up for discussion.

Cheers - Michael


On 2014-10-16 at 22:51 -0700, Jelmer Vernooij wrote:
> On Thu, Oct 16, 2014 at 12:02:51AM -0700, Matthieu Patou wrote:
> > On Mon, Oct 13, 2014 at 06:22:25AM +0200, Jelmer Vernooij wrote:
> > > On Sun, Oct 12, 2014 at 09:16:19PM -0700, Matthieu Patou wrote:
> > > > From 9f21d3f48b492f38297cecacceca4beb6d540e3f Mon Sep 17 00:00:00 2001
> > > > From: Matthieu Patou <mat at matws.net>
> > > > Date: Sun, 12 Oct 2014 20:59:02 -0700
> > > > Subject: [PATCH 4/4] selftest: add documentation for options --exclude and
> > > >  --include that weren't documented before
> > > > 
> > > > Change-Id: Ibf0fa1c548d376dc8d68ab8136e2532f67dd1d8a
> > > > Signed-off-by: Matthieu Patou <mat at matws.net>
> > > > ---
> > > >  selftest/selftest.pl | 2 ++
> > > >  1 file changed, 2 insertions(+)
> > > > 
> > > > diff --git a/selftest/selftest.pl b/selftest/selftest.pl
> > > > index f35b063..fdba4d5 100755
> > > > --- a/selftest/selftest.pl
> > > > +++ b/selftest/selftest.pl
> > > > @@ -190,6 +190,8 @@ Generic options:
> > > >   --help                     this help page
> > > >   --target=samba[3]|win      Samba version to target
> > > >   --testlist=FILE            file to read available tests from
> > > > + --exclude=FILE             Exclude tests listed in the file
> > > > + --inlcude=FILE             Include tests listed in the file
> > > 
> > > 
> > > ^^ s/inlcude/include/
> > > 
> > > >  
> > > >  Paths:
> > > >   --prefix=DIR               prefix to run tests in [st]
> > > > -- 
> > > > 1.9.1
> > > > 
> > > 
> > > > From 190ed0d375b022dcefe8c322be3a4d54f3759268 Mon Sep 17 00:00:00 2001
> > > > From: Matthieu Patou <mat at matws.net>
> > > > Date: Sun, 12 Oct 2014 17:57:20 -0700
> > > > Subject: [PATCH 3/4] selftest: report the 10 slowest tests (by default) make
> > > >  the number configurable
> > > > 
> > > > Change-Id: Ib1cf50199d110827a25cf198b40738f3c72cbe17
> > > > Signed-off-by: Matthieu Patou <mat at matws.net>
> > > > ---
> > > >  selftest/wscript | 10 ++++++++++
> > > >  1 file changed, 10 insertions(+)
> > > > 
> > > > diff --git a/selftest/wscript b/selftest/wscript
> > > > index a57f7c1..ff680fb 100644
> > > > --- a/selftest/wscript
> > > > +++ b/selftest/wscript
> > > > @@ -43,6 +43,9 @@ def set_options(opt):
> > > >      gr.add_option('--slow',
> > > >                    help=("enable the really slow tests"),
> > > >                    action="store_true", dest='SLOWTEST', default=False)
> > > > +    gr.add_option('--nb-slowest',
> > > > +                  help=("Show the n slowest tests (10 by defaults)"),
> > > ^^ s/defaults/by default/. For consistency, please use "(default=10)" as suffix.
> > > 
> > > You can specify "type=int, default=10" here and prevent the cludge further down.
> > > 
> > > > +                  dest='NB_SLOWEST')
> > > >      gr.add_option('--testenv',
> > > >                    help=("start a terminal with the test environment setup"),
> > > >                    action="store_true", dest='TESTENV', default=False)
> > > > @@ -226,6 +229,13 @@ def cmd_testonly(opt):
> > > >          runcmd = EXPAND_VARIABLES(opt, testrcmd)
> > > >          RUN_COMMAND(runcmd, env=env)
> > > >  
> > > > +    if os.path.exists("st/subunit"):
> > > > +        nb = int(getattr(Options.options, 'NB_SLOWEST', 10))
> > > ^^ If you set a default in the add_option call, this can just be Options.options.NB_SLOWEST.
> > > 
> > > > +        print "TOP %d slowest tests" % nb
> > > > +        cmd = "./script/show_testsuite_time ./st/subunit %d" % nb
> > > ^^ Please use Options.options.selftest_prefix here rather than ./st.
> > > 
> > >  
> > > > diff --git a/selftest/selftest.pl b/selftest/selftest.pl
> > > > index f35b063..fdba4d5 100755
> > > > --- a/selftest/selftest.pl
> > > > +++ b/selftest/selftest.pl
> > > > @@ -190,6 +190,8 @@ Generic options:
> > > >   --help                     this help page
> > > >   --target=samba[3]|win      Samba version to target
> > > >   --testlist=FILE            file to read available tests from
> > > > + --exclude=FILE             Exclude tests listed in the file
> > > > + --inlcude=FILE             Include tests listed in the file
> > > ^^ s/inlcude/include/
> > > 
> > > Jelmer
> > 
> > Updated version of the patches
> 
> Reviewed-By: Jelmer Vernooij <jelmer at samba.org> for all 4.
> 
> Jelmer
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0001-subunit-report-X-Y-at-Zs-instead-of-X-Y-in-Zs.patch
Type: text/x-diff
Size: 1059 bytes
Desc: not available
URL: <http://lists.samba.org/pipermail/samba-technical/attachments/20141017/a17b4228/attachment.patch>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature
URL: <http://lists.samba.org/pipermail/samba-technical/attachments/20141017/a17b4228/attachment.pgp>


More information about the samba-technical mailing list