[PATCH] Add substitutions %t, %j, and %J as path-safe variants of %T, %i, and %I.

[Andrew Bartlett]

On Mon, 2017-11-27 at 15:34 +0100, Dr. Thomas Orgis via samba-technical 
wrote:
> Am Fri, 24 Nov 2017 09:43:15 +1300
> schrieb Andrew Bartlett via samba-technical <samba-technical at lists.samba.org>:
> 
> > This is an interesting addition.  Could you add some tests for it?
> 
> I'm trying. I started by adding
> 
>  	ok &= subst_test("%j %J", "", "", -1, -1, "0_0_0_0 0_0_0_0");
> 
> to run_local_substitute. A test for the %t substitution is more tricky
> as the test for the existing %T is missing. Need to insert a fake time,
> too. I guess fixing that would be the task of a patch before my patch …
> 
> Actually, I also was unable to find a reference to %T, %i or %I in the
> selftest/target/Samba3.pm definitions. I would happily have added
> entries for my path-safe variants, but my cargo cult approach does not
> work when I do not have the fruits prepared for picking. I need to
> guess a lot to get the tests right, although it should be fairly simple
> for someone used to the test suite.
> 
> I am adding tests now for my variants of the substitutions that also
> had tests before, and am omitting tests for the variants of
> substitutions that also had no test before … this seems about fair;-)

I know it seems fair.  However we really do need tests, otherwise we
never move forward from our current state where not enough is tested.

> Another patch should introduce tests for %T and %t, IMHO. An updated
> version of the patch is attached. The little test I added seems to pass:

Some kind of sub test for %T and %t would make this much easier to
merge. 

> shell$ make test TESTS=local-substitute
> […]
> skipping subunit (testscenarios not available)
> OPTIONS --configfile=$SMB_CONF_PATH --option='fss:sequence timeout=1' --maximum-runtime=$SELFTEST_MAXTIME --basedir=$SELFTEST_TMPDIR --format=subunit --option=torture:progress=no
> [1(0)/1 at 0s] samba3.smbtorture_s3.LOCAL-SUBSTITUTE
> 
> ALL OK (1 tests in 1 testsuites)
> 
> A summary with detailed information can be found in:
>   ./st/summary
> TOP 10 slowest tests
> samba3.smbtorture_s3.LOCAL-SUBSTITUTE -> 0
> 'testonly' finished successfully (0.848s)
> shell$
> 
> 
> Regards,
> 
> Thomas
> 
> 
> PS: My development box cannot run the tests successfully because
> I do not have PAM (yes, still possible;-). I can configure samba
> --without-pam, but the test suite is not prepared for that:
> 
> test: running (/usr/bin/perl /stuff/src/samba/selftest/selftest.pl --target=samba --prefix=./st --srcdir=/stuff/src/samba --exclude=/stuff/src/samba/selftest/skip --testlist="/usr/bin/python /stuff/src/samba/selftest/tests.py|" --testlist="/usr/bin/python /stuff/src/samba/source3/selftest/tests.py|" --testlist="/usr/bin/python /stuff/src/samba/source4/selftest/tests.py|"  --exclude=/stuff/src/samba/selftest/slow --nss_wrapper_so_path=/stuff/src/samba/bin/default/lib/nss_wrapper/libnss-wrapper.so --resolv_wrapper_so_path=/stuff/src/samba/bin/default/lib/resolv_wrapper/libresolv-wrapper.so --socket_wrapper_so_path=/stuff/src/samba/bin/default/lib/socket_wrapper/libsocket-wrapper.so --uid_wrapper_so_path=/stuff/src/samba/bin/default/lib/uid_wrapper/libuid-wrapper.so --use-dns-faking --socket-wrapper  && touch ./st/st_done) | /usr/bin/python -u /stuff/src/samba/selftest/filter-subunit --expected-failures=/stuff/src/samba/selftest/knownfail --expected-failures=/stuff/src/samba/selftest/knownfail.d --flapping=/stuff/src/samba/selftest/flapping --flapping=/stuff/src/samba/selftest/flapping.d | tee ./st/subunit | /usr/bin/python -u /stuff/src/samba/selftest/format-subunit --prefix=./st --immediate
> LD_PRELOAD=/stuff/src/samba/bin/default/lib/nss_wrapper/libnss-wrapper.so:/stuff/src/samba/bin/default/lib/resolv_wrapper/libresolv-wrapper.so:/stuff/src/samba/bin/default/lib/socket_wrapper/libsocket-wrapper.so:/stuff/src/samba/bin/default/lib/uid_wrapper/libuid-wrapper.so
> SOCKET_WRAPPER_DIR=/stuff/src/samba/st/w
> DNS: Faking nameserver
> Traceback (most recent call last):
>   File "/stuff/src/samba/selftest/tests.py", line 43, in <module>
>     pam_wrapper_so_path=config_hash["LIBPAM_WRAPPER_SO_PATH"]
> KeyError: 'LIBPAM_WRAPPER_SO_PATH'
> Error creating recipe from /usr/bin/python /stuff/src/samba/selftest/tests.py| at /stuff/src/samba/selftest/selftest.pl line 674.
> 
> ALL OK (0 tests in 0 testsuites)
> 
> A summary with detailed information can be found in:
>   ./st/summary
> TOP 10 slowest tests
> ERROR: test command failed to complete
> make: *** [Makefile:17: test] Fehler 1
> 16:46|host:samba$ cat st/summary 
> = Skipped tests =
> 
> One might catch this at the configure stage instead.

I agree this isn't ideal.  The fix is harder, if we make the test more
flexible we risk accidentally cutting out the testsuite when a
configure test breaks (fails to detect PAM for example), but if we make
the configure time checks stricter then we won't run at all except in
the 'blessed' environment, making deployments beyond that less tested. 

> I finally resorted to the deployment machine where there is a more
> standard Linux system. The full test suite has this result:
> 
> FAILED (139 failures, 33 errors and 2 unexpected successes in 48 testsuites)

That doesn't look like enough testsuites. 

> I am not sure how bad or good that is. It doesn't say how many
> individual tests passed to put the 139 failures into perspective. This
> is a stock Ubuntu 16.04 x86-64 system. Just the samba packages on it
> are patched, obviously. The server works, so it doesn't seem to be too
> bad.

We are working to get more cloud-based testing available to make this
easier to contribute to, which will be in known-good environments.

Andrew Bartlett

-- 
Andrew Bartlett
https://samba.org/~abartlet/
Authentication Developer, Samba Team         https://samba.org
Samba Development and Support, Catalyst IT   
https://catalyst.net.nz/services/samba