smbclient_tar flapping

Wed Jun 15 17:54:59 UTC 2016

On Wed, 15 Jun 2016 07:31:51 +1200 Andrew Bartlett <abartlet at samba.org>
wrote:
> That is, perhaps this, and the fact that it started happening only a
> few months ago is pointing out an underlying bug in smbd, or a change
> in the host kernel?

While experimenting with the smbclient tar test script I've noticed
something really strange. I've added an attribute check after each file
creation.

Files are created in create_file() like this:

<snip>
    unlink $fn if -e $fn;
    $size ||= main::random(512, 1024);
    $size = int($size);
    my $md5;

    # try /dev/urandom, faster
    if (-e '/dev/urandom') {
        my $cmd = sprintf('head -c %d /dev/urandom | tee %s | md5sum',
                          $size, quotemeta($fn));
        say "create_file size=$size, $cmd";
        $md5 = (split / /, `$cmd`)[0];
<snip>

Which is used by individual tests via File::new_remote(). This is where
I added the check.

<snip>
    my $self = bless {
        'attr' => {qw/r 0 s 0 h 0 a 0 d 0 n 0/},
        'dir'  => $dir,
        'name' => $file,
        'md5'  => create_file($loc.'/'.$file, $size),
        'remote' => 1,
    }, $class;

    my %remote_attrs = $self->get_attr();

    # XXX: SANITY ATTR CHECK
    if (!$remote_attrs{n}||$remote_attrs{d}||$remote_attrs{r}
        ||$remote_attrs{s}||$remote_attrs{h}||$remote_attrs{a})
    {
        die "new file has strange attribute state (expected just N for normal)\n";
    }
<snip>

get_attr() simply parses smbclient "dir" output and returns the
attributes (it doesnt mutate $self).

Well the check fails at random times pretty often. I expected new
files created from /dev/urandom to have no attributes (N flag) but it
turns out they are created with strange states very often:

> UNEXPECTED(failure): samba3.blackbox.smbclient_tar (fileserver).create, normal nested files (no attributes)(fileserver)
> REASON: Exception: Exception: create_file size=732, head -c 732 /dev/urandom | tee \/home\/aaptel\/prog\/samba\-git\/st\/fileserver\/share\/\.\/st\/foo\/bar\/bar\/file\-1 | md5sum
> /home/aaptel/prog/samba-git/bin/smbclient //FILESERVER/tmp -Uaaptel\%fileserver -I 127.0.0.35 --configfile\=/home/aaptel/prog/samba-git/st/client/client.conf -D ./st/foo/bar/bar -c ls\ \"file-1\"
> Domain=[SAMBA-TEST] OS=[Windows 6.1] Server=[Samba 4.5.0pre1-DEVELOPERBUILD]
>   file-1                              H      732  Wed Jun 15 19:16:43 2016
> 
>                 3905108792 blocks of size 1024. 3715276956 blocks available
> 
> new file has strange attribute state (expected just N for normal)

Here its hidden.

> UNEXPECTED(failure): samba3.blackbox.smbclient_tar (fileserver).create, normal files (interactive)(fileserver)
> REASON: Exception: Exception: create_file size=797, head -c 797 /dev/urandom | tee \/home\/aaptel\/prog\/samba\-git\/st\/fileserver\/share\/\.\/st\/file\-1 | md5sum
> /home/aaptel/prog/samba-git/bin/smbclient //FILESERVER/tmp -Uaaptel\%fileserver -I 127.0.0.35 --configfile\=/home/aaptel/prog/samba-git/st/client/client.conf -D ./st -c ls\ \"file-1\"
> Domain=[SAMBA-TEST] OS=[Windows 6.1] Server=[Samba 4.5.0pre1-DEVELOPERBUILD]
>   file-1                              A      797  Wed Jun 15 19:16:43 2016
> 
>                 3905108792 blocks of size 1024. 3715276956 blocks available
> 
> new file has strange attribute state (expected just N for normal)

Here its archived.

And now the best part: after removing the die() call and letting it run
further...

> REASON: Exception: Exception:
> create_file size=535, head -c 535 /dev/urandom | tee \/home\/aaptel\/prog\/samba\-git\/st\/fileserver\/share\/file0\.jpg | md5sum
> /home/aaptel/prog/samba-git/bin/smbclient //FILESERVER/tmp -Uaaptel\%fileserver -I 127.0.0.35 --configfile\=/home/aaptel/prog/samba-git/st/client/client.conf -c ls\ \"file0.jpg\"
> Domain=[SAMBA-TEST] OS=[Windows 6.1] Server=[Samba 4.5.0pre1-DEVELOPERBUILD]
>   file0.jpg                           D        0  Tue Jun 14 18:34:51 2016
> 
>                 3905108792 blocks of size 1024. 3714374024 blocks available

The file is reported as a directory! Then a bit later in the same run:

> /home/aaptel/prog/samba-git/bin/smbclient //FILESERVER/tmp -Uaaptel\%fileserver -I 127.0.0.35 --configfile\=/home/aaptel/prog/samba-git/st/client/client.conf -TcrX /tmp/zIE_TyzD6k/tarmode.tar \*.exe
> Domain=[SAMBA-TEST] OS=[Windows 6.1] Server=[Samba 4.5.0pre1-DEVELOPERBUILD]
> tar:712  Total bytes received: 635
> NT_STATUS_OBJECT_PATH_NOT_FOUND listing \file0.jpg\*

When calling the tar command to archive it, it tries to 'cd' into it
(and fails obviously).

I couldn't reproduce these problems out of the self-test environment so far.
I was able to make a simpler 40 lines self-test shell script (attached)
that triggers the weird attributes (not very often tho):

> testsuite: samba3.blackbox.smbclient_filedir (fileserver)
> progress: push
> time: 2016-06-15 18:16:46.000000Z
> test: samba3.blackbox.smbclient_filedir (fileserver).file reported as dir(fileserver)
> failure: samba3.blackbox.smbclient_filedir (fileserver).file reported as dir(fileserver) [
> Exception: Exception:
> 
> run #705 failed: got this attribute instead of N
> /home/aaptel/prog/samba-git/bin/smbclient: Can't load --configfile=/home/aaptel/prog/samba-git/st/client/client.conf - run testparm to debug it
> Domain=[SAMBA-TEST] OS=[Windows 6.1] Server=[Samba 4.5.0pre1-DEVELOPERBUILD]
>   file-25248                          S      700  Wed Jun 15 18:17:15 2016
> 
>                 3905108792 blocks of size 1024. 3715116580 blocks available
> 
> 
> 
> ]

After creating, checking, deleting a random file 705 times one system
flag showed up.

> Unix group domusers already mapped to SID S-1-5-21-1686286308-928999311-630493174-513
> [1(0)/1 at 0s] samba3.blackbox.smbclient_filedir (fileserver)
> teardown_env(fileserver)
> UNEXPECTED(failure): samba3.blackbox.smbclient_filedir (fileserver).file reported as dir(fileserver)
> REASON: Exception: Exception: run #214 failed: got this attribute instead of N
> /home/aaptel/prog/samba-git/bin/smbclient: Can't load --configfile=/home/aaptel/prog/samba-git/st/client/client.conf - run testparm to debug it
> Domain=[SAMBA-TEST] OS=[Windows 6.1] Server=[Samba 4.5.0pre1-DEVELOPERBUILD]
>   file-31365                          S      700  Wed Jun 15 18:38:21 2016
> 
>                 3905108792 blocks of size 1024. 3715125000 blocks available
> 
> FAILED (1 failures, 0 errors and 0 unexpected successes in 0 testsuites)

From another run, at the #214 try. Another mysterious system flag.

-- 
Aurélien Aptel / SUSE Labs Samba Team
GPG: 1839 CB5F 9F5B FB9B AA97  8C99 03C8 A49B 521B D5D3
SUSE Linux GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG
Nürnberg)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: test_smbclient_filedir.sh
Type: application/x-shellscript
Size: 772 bytes
Desc: not available
URL: <http://lists.samba.org/pipermail/samba-technical/attachments/20160615/24d6371a/test_smbclient_filedir.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: <http://lists.samba.org/pipermail/samba-technical/attachments/20160615/24d6371a/attachment.sig>