smbclient_tar flapping

Aurélien Aptel aaptel at suse.com
Tue Jun 21 16:09:58 UTC 2016 

On Mon, 20 Jun 2016 16:47:31 -0700 Jeremy Allison <jra at samba.org> wrote:
> Needs to be repeatable before I'll believe it I'm afraid.
> More likely to be corruption in the testing EA store - remember
> the 'D' bit is stored there in the dos attributes EA.

I also think it might me a tdb problem as selftest uses "store dos attributes = yes"
and the tdb xattr VFS. I can also see this in logs when smbd looks for DOS attrs:

[10 2016/06/17 14:32:21.501261 ../source3/lib/xattr_tdb.c:183 xattr_tdb_getattr]
   xattr_tdb_getattr called for file 801:10051158:0, name system.fake_uid
[10 2016/06/17 14:32:21.501276 ../source3/lib/xattr_tdb.c:189 xattr_tdb_getattr]
   xattr_tdb_fetch_attrs failed: NT_STATUS_INTERNAL_DB_CORRUPTION

Attached test script is repeatable as far as b1548fd (commit date in
2012) on my system.

- If the script is already in the repo at the commit you are testing, replace it.
- If it's not add it at the same path and patch source3/selftest/tests.py
  with attached patch.

In older revisions you might need to backport stuff to compile/configure:

- for PIDL compilation problems:

    git apply <(git show b07ade6e6)
    git apply <(git show 92254d0)

- for make-proto.pl problems:

    git apply <(git show bb4995b)

In newer revisions you might need to configure with --with-libarchive to enable the tests.

In the 2012 commit I mentionned I get:

---
[1/1 in 0s] samba3.blackbox.smbclient_tar (s3dc)
UNEXPECTED(failure): samba3.blackbox.smbclient_tar (s3dc).create, files newer than a file(s3dc)
REASON: _StringException: _StringException: 
## create_file head -c 886 /dev/urandom | tee \/home\/aaptel\/prog\/samba\-git\/st\/s3dc\/share\/\.\/st\/oldest | md5sum
## -rw-r--r-- 1 aaptel suse 886 juin  21 16:25 /home/aaptel/prog/samba-git/st/s3dc/share/./st/oldest

## new file has strange attribute state (expected just N for normal)
## attr: [A] perms: -rw-r--r-- 1 aaptel suse 886 juin  21 16:25 /home/aaptel/prog/samba-git/st/s3dc/share/./st/oldest

FAILED (1 failures, 0 errors and 0 unexpected successes in 0 testsuites)
---

New file is reported with archive flag (A) instead of nothing (N) in "smbclient ls" output.
I'm still trying to find a good commit to bisect. I'm currently stuck at e39df67 (2011!)
where selftest itself fails, I cannot find the right thing to backport to make it work.

---
aaptel at aaptelpc ~/prog/samba-git % make test TESTS='smbclient_tar[^m]' SMBD_OPTIONS=-d10 TEST_OPTIONS=--fail-immediately
WAF_MAKE=1 ./buildtools/bin/waf test --fail-immediately
'test' finished successfully (0.010s)
Waf: Entering directory `/home/aaptel/prog/samba-git/bin'
[ 128/3938] Generating VERSION
[ 163/3938] Generating smbd/build_options.c
Waf: Leaving directory `/home/aaptel/prog/samba-git/bin'
'build' finished successfully (3.470s)
test: running (/usr/bin/perl /home/aaptel/prog/samba-git/selftest/selftest.pl --target=samba --prefix=./st --srcdir=/home/aaptel/prog/samba-git --exclude=/home/aaptel/prog/samba-git/selftest/skip --testlist="/usr/bin/python /home/aaptel/prog/samba-git/source3/selftest/tests.py|" --testlist="/usr/bin/python /home/aaptel/prog/samba-git/source4/selftest/tests.py|" --binary-mapping=nmblookup3:nmblookup3,smbclient3:smbclient3,smbtorture4:smbtorture,ntlm_auth3:ntlm_auth3 --exclude=/home/aaptel/prog/samba-git/selftest/slow --socket-wrapper smbclient_tar[^m] && touch ./st/st_done) | /usr/bin/python -u /home/aaptel/prog/samba-git/selftest/filter-subunit --expected-failures=/home/aaptel/prog/samba-git/selftest/knownfail --flapping=/home/aaptel/prog/samba-git/selftest/flapping --fail-immediately | tee ./st/subunit | /usr/bin/python -u /home/aaptel/prog/samba-git/selftest/format-subunit --prefix=./st --immediate
Traceback (most recent call last):
  File "/home/aaptel/prog/samba-git/selftest/filter-subunit", line 29, in <module>
Traceback (most recent call last):
  File "/home/aaptel/prog/samba-git/selftest/format-subunit", line 15, in <module>
    import subunithelper
  File "/home/aaptel/prog/samba-git/selftest/subunithelper.py", line 22, in <module>
    import subunithelper
  File "/home/aaptel/prog/samba-git/selftest/subunithelper.py", line 22, in <module>
    import subunit
    import subunit
  File "/home/aaptel/prog/samba-git/selftest/../lib/subunit/python/subunit/__init__.py", line 125, in <module>
  File "/home/aaptel/prog/samba-git/selftest/../lib/subunit/python/subunit/__init__.py", line 125, in <module>
    from testtools import content, content_type, ExtendedToOriginalDecorator
    from testtools import content, content_type, ExtendedToOriginalDecorator
  File "/home/aaptel/prog/samba-git/selftest/../lib/testtools/testtools/__init__.py", line 34, in <module>
  File "/home/aaptel/prog/samba-git/selftest/../lib/testtools/testtools/__init__.py", line 34, in <module>
    from testtools.matchers import (
    from testtools.matchers import (
  File "/home/aaptel/prog/samba-git/selftest/../lib/testtools/testtools/matchers/__init__.py", line 55, in <module>
  File "/home/aaptel/prog/samba-git/selftest/../lib/testtools/testtools/matchers/__init__.py", line 55, in <module>
    'WmfImagePlugin',
      File "/home/aaptel/prog/samba-git/selftest/../lib/testtools/testtools/matchers/_basic.py", line 26, in <module>
'WmfImagePlugin',
  File "/home/aaptel/prog/samba-git/selftest/../lib/testtools/testtools/matchers/_basic.py", line 26, in <module>
ImportError: ImportErrorcannot import name list_subtract
: cannot import name list_subtract
Exiting early because of SIGPIPE.
ERROR: test failed with exit code 1
Makefile:16: recipe for target 'test' failed
make: *** [test] Error 1
---

But having this bug for so long undetected makes me wonder if it's not
coming from something else in the setup and not from samba/selftest...

-- 
Aurélien Aptel / SUSE Labs Samba Team
GPG: 1839 CB5F 9F5B FB9B AA97  8C99 03C8 A49B 521B D5D3
SUSE Linux GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG
Nürnberg)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: test_smbclient_tarmode.pl
Type: application/x-perl
Size: 41194 bytes
Desc: not available
URL: <http://lists.samba.org/pipermail/samba-technical/attachments/20160621/4f7a8494/test_smbclient_tarmode.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: tests.py.patch
Type: text/x-patch
Size: 1112 bytes
Desc: not available
URL: <http://lists.samba.org/pipermail/samba-technical/attachments/20160621/4f7a8494/tests.py.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: <http://lists.samba.org/pipermail/samba-technical/attachments/20160621/4f7a8494/attachment.sig>

More information about the samba-technical mailing list