NT_STATUS_FILE_IS_A_DIRECTORY in nosymlinks test

[Andrew Bartlett]

On Mon, 2017-07-03 at 08:20 +0200, Andreas Schneider wrote:
> On Friday, 30 June 2017 07:03:17 CEST Andrew Bartlett via samba-technical 
> wrote:
> > G'Day Andreas,
> 
> Hi Andrew,
>  
> > I've been trying to make our test environments run in parallel, but
> > when I enable that on the Catalyst Cloud, using this branch:
> > 
> > https://git.samba.org/?p=abartlet/samba.git/.git;a=shortlog;h=refs/heads/par
> > allel-test
> > 
> > I (about 50% of the time) get failures in
> > test_nosymlinks() in test_smbclient_s3.sh
> > UNEXPECTED(failure): samba3.blackbox.smbclient_s3.SMB3.plain
> > (ad_member) member creds.follow symlinks = no(ad_member)
> > 
> > (sorry for the garbled output, something very odd is going on in the
> > output stream with the escaping)
> 
> I've seen the same and tried to find the issue but couldn't nail it down.
> 
> > REASON: Exception: Exception: CLI_FORCE_INTERACTIVE=yes
> > /home/ubuntu/autobuild/b20147/samba/bin/smbclient
> > -ULOCALADMEMBER/ubuntu%loCalMemberPass //LOCALADMEMBER/nosymlinks -I
> > 127.0.0.29
> > --configfile=/home/ubuntu/autobuild/b20147/samba/bin/ab/client/client.conf
> > -mSMB3 NT_STATUS_ACCESS_DENIED opening remote file     est\source
> > CLI_FORCE_INTERACTIVE=yes /home/ubuntu/autobuild/b20147/samba/bin/smbclient
> > -ULOCALADMEMBER/ubuntu%loCalMemberPass //LOCALADMEMBER/nosymlinks -I
> > 127.0.0.29
> > --configfile=/home/ubuntu/autobuild/b20147/samba/bin/ab/client/client.conf
> > -mSMB3 CLI_FORCE_INTERACTIVE=yes
> > /home/ubuntu/autobuild/b20147/samba/bin/smbclient
> > -ULOCALADMEMBER/ubuntu%loCalMemberPass //LOCALADMEMBER/nosymlinks -I
> > 127.0.0.29
> > --configfile=/home/ubuntu/autobuild/b20147/samba/bin/ab/client/client.conf
> > -mSMB3 NT_STATUS_FILE_IS_A_DIRECTORY opening remote
> > file       estooar       estfile Try "help" do get a list of possible
> > commands.
> > smb: \> cd testooar
> > smb:    estooar\> ls
> >   .                                   D        0  Fri Jun 30 03:27:08 2017
> >   ..                                  D        0  Fri Jun 30 03:27:08 2017
> >   testfile                            D        0  Fri Jun 30 03:27:08 2017
> > 
> >                 10287416 blocks of size 1024. 3260044 blocks available
> > smb:    estooar\> get testfile -
> > NT_STATUS_FILE_IS_A_DIRECTORY opening remote
> > file       estooar       estfile smb:    estooar\> quit
> > failed - NT_STATUS_XXXX doing cd fooar; get testfile on 
> > osymlinks
> > 
> > It would be great to make some progress on this, because in the
> > successful runs, I see autobuild times drop by 60mins (288 -> 218 on an
> > identical VM)!
> > 
> > I had assumed this was due to re-use of the dev/inode, but I tried the
> > attached and it didn't help.  (This happens when files are created or
> > deleted on the client side, not the server side).
> 
> I've sent a mail to Jeremy who wrote the code and test for the CVE regression, 
> but he asked for a reproducer. I can only reproduce it if I run the test 20 
> times in a loop then normally it happens once.
> 
> > Another idea would be to wipe the xattr.tdb file between each test.
> > 
> > My final idea is to have as much of the deltree and mkdir etc be done
> > over SMB, so they will get an entry in the xattr.tdb, but I've not had
> > time to do this yet.
> 
> Hmm. Jeremy any ideas?

I tried to do the deltree and mkdir, but it just became a time sink of
failed authentication and other random pain.

https://git.samba.org/?p=abartlet/samba.git/.git;a=shortlog;h=refs/heads/parallel-test

See

https://git.samba.org/?p=abartlet/samba.git/.git;a=commitdiff;h=c1988e2bebb04b3a8f17d08b2761cabecd235459
https://git.samba.org/?p=abartlet/samba.git/.git;a=commitdiff;h=7a67ac8b71271aeec4664ff341522569d7922abb

The key is to never do local rm, mkdir our file creation operations. 
ln is probably OK, but anything else risks dev/inode re-use.


> P.S. Metze and I started to improve selftest. However I couldn't get the 
> following patch working yet:
> 
> https://git.samba.org/?p=asn/samba.git;a=shortlog;h=refs/heads/master-selftest-ok
> 
> Some tests are failing but we couldn't nail it down to why and where.

OK.  I'll take a peek when I get a chance. 

Thanks,

Andrew Bartlett

-- 
Andrew Bartlett                       http://samba.org/~abartlet/
Authentication Developer, Samba Team  http://samba.org
Samba Developer, Catalyst IT          http://catalyst.net.nz/services/samba