[jcifs] Re: jcifs-1.2.4 stability fixes

[Lars heete]

Hello,
> > > Actually the threads are not waiting for each other, Thread-8 is
> > > waiting for itself. It called sessionSetup which set's it's state to 1
> > > (setting up) and then ends up calling logoff via doDisconnect
> > > (presumably because you pulled the network cable). Because it sees the
> > > state of 1 it waits which of course is futile because it's waiting for
> > > itself to complete the sessionSetup.
> > >
> > > This is pretty easy to fix actually. I had seen this in the Transport
> > > state machine and had a solution there but only now realized it
> > > can happen for all objects with explict locking for both setup and
> > > teardown. The solution is to make a note of the thread responsible for
> > > the state transition and then check for at the top of
> > > connect/disconnect, sessionSetup/logoff, and
> > > treeConnect/treeDisconnect. See the sentinel member of Transport,
> > > SmbSession, and SmbTree.
> > >
> > > Anyway, I think it's fixed. Please try jcifs-1.2.6test2.
> >
> > does not work when starting more than one threads on one session at the
> > same time (output from my SmbThreadTest class)
> >
> > creating 3 threads
> > SAE: Access is denied.
> > jcifs.smb.SmbAuthException: Access is denied.
> >         at jcifs.smb.SmbTransport.checkStatus(SmbTransport.java:488)
> >         at jcifs.smb.SmbTransport.send(SmbTransport.java:599)
> >         at jcifs.smb.SmbSession.sessionSetup(SmbSession.java:303)
> >         at jcifs.smb.SmbSession.send(SmbSession.java:237)
> >         at jcifs.smb.SmbTree.treeConnect(SmbTree.java:180)
> >         at jcifs.smb.SmbFile.connect(SmbFile.java:792)
> >         at jcifs.smb.SmbFile.connect0(SmbFile.java:762)
> >         at jcifs.smb.SmbFile.exists(SmbFile.java:1248)
> >         at SmbThreadTest.traverse(SmbThreadTest.java:41)
> >         at SmbThreadTest.run(SmbThreadTest.java:97)
> > SAE: Access is denied.
> > jcifs.smb.SmbAuthException: Access is denied.
> >         at jcifs.smb.SmbTransport.checkStatus(SmbTransport.java:488)
> >         at jcifs.smb.SmbTransport.send(SmbTransport.java:599)
> >         at jcifs.smb.SmbSession.sessionSetup(SmbSession.java:303)
> >         at jcifs.smb.SmbSession.send(SmbSession.java:237)
> >         at jcifs.smb.SmbTree.treeConnect(SmbTree.java:180)
> >         at jcifs.smb.SmbFile.connect(SmbFile.java:792)
> >         at jcifs.smb.SmbFile.connect0(SmbFile.java:762)
> >         at jcifs.smb.SmbFile.exists(SmbFile.java:1248)
> >         at SmbThreadTest.traverse(SmbThreadTest.java:41)
> >         at SmbThreadTest.run(SmbThreadTest.java:97)
> > SAE: Access is denied.
>
> How do you know this isn't normal? When I run the CrawlTest I get Access
> is denied trying to access files that I don't own and other special
> files like pagefile.sys, etc.
Because it works with when starting only one thread at once.

> > also with only one thread per session jcifs can block or throw similar
> > exceptions like above when transport becomes unresponsive for several
> > seconds (for example if you remove the network-cable from the server).
>
> Ok, well you have to give me more to go on than this. What
> exceptions? Give me a thread dump of this "block". Give me something.
actually you the "Access is denied" exceptions from above (won't help very 
much). But also

jcifs.smb.SmbException:
java.net.SocketException: Socket closed
        at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:99)
        at java.net.SocketOutputStream.write(SocketOutputStream.java:136)
        at jcifs.smb.SmbTransport.doSend(SmbTransport.java:407)
        at jcifs.util.transport.Transport.sendrecv(Transport.java:79)
        at jcifs.smb.SmbTransport.send(SmbTransport.java:591)
        at jcifs.smb.SmbSession.sessionSetup(SmbSession.java:303)
        at jcifs.smb.SmbSession.send(SmbSession.java:237)
        at jcifs.smb.SmbTree.treeConnect(SmbTree.java:180)
        at jcifs.smb.SmbFile.connect(SmbFile.java:792)
        at jcifs.smb.SmbFile.connect0(SmbFile.java:762)
        at jcifs.smb.SmbFile.exists(SmbFile.java:1248)
        at SmbSessionListTest.traverse(SmbSessionListTest.java:58)
        at SmbSessionListTest.run(SmbSessionListTest.java:43)
        at java.lang.Thread.run(Thread.java:534)

        at jcifs.smb.SmbTransport.send(SmbTransport.java:596)
        at jcifs.smb.SmbSession.sessionSetup(SmbSession.java:303)
        at jcifs.smb.SmbSession.send(SmbSession.java:237)
        at jcifs.smb.SmbTree.treeConnect(SmbTree.java:180)
        at jcifs.smb.SmbFile.connect(SmbFile.java:792)
        at jcifs.smb.SmbFile.connect0(SmbFile.java:762)
        at jcifs.smb.SmbFile.exists(SmbFile.java:1248)
        at SmbSessionListTest.traverse(SmbSessionListTest.java:58)
        at SmbSessionListTest.run(SmbSessionListTest.java:43)
        at java.lang.Thread.run(Thread.java:534)

for a long time when using my SessionList Test.
The thread dump of the blocked case showed no deadlock, but many threads 
waiting on transport.

> Also note that if you set soTimeout and responseTimeout to a very
> low value (e.g. 300) it is very likely that you will get many timeout
> exceptions at least until the server cache is hot.
soTimeout is 150000 and responseTimeout is 10000

> > > > Attached is the test to trigger these problems (SmbThreadTest).
> > > > To simulate many user sessions the session-match.diff patch is needed
> > > > and the "jcifs.smb.client.debug.compareSessionAuth" property has to
> > > > be set to "false". The test needs to be run on a share with a
> > > > sufficient large directory structure. If you disconnect transport on
> > > > the server side several times there will be no new sessions if the
> > > > bug is triggered.
> > >
> > > Actually I've been using the examples/CrawlTest.java example. It's
> > > pretty much like your test but I think it might stress things even more
> > > because it sets the soTimeout and responseTimeout just low enough so
> > > that sessions and transports timeout frequently which causes them to be
> > > setup and torn down a lot. I also have a commented clause in
> > > SmbTransport.getSmbSession that does what I think compareSessionAuth
> > > does. When you run it, hit return to add a thread to the crawl.
> > >
> > > Needless to say I ran CrawlTest quite a bit and stopped/started Samba
> > > and the client was able to recover ok. I think this release might go
> > > the distance. I'd really like to see how it stands up to your HTTP
> > > crawler though.
> >
> > but your CrawlTest only starts one thread at once.
>
> Huh? You can hit enter 15 times and get 15 threads all crawling the
> same directory at the same time.
It's about starting at the same time, probabliy calling sessionSetup in 
parralel.
> I get the feeling you're not particularly interesting in persuing this
> further. That's ok with me. To be honest, I don't actually use JCIFS
> anymore so I'm doing this 100% for other people at this point.
Im not using jcifs for own projects, but a customer had asked me to resolve 
the threading issues. My changes to jcifs seem to work quite well for his 
application so he is interested in getting them merged back.
I also have a new version pending, that simplifies my changes and alters the 
response wakeup-behavior to be more like jcifs-1.1, which was actually much 
faster on transports with many active requests. (in jcifs 1.2 the notify on 
response_map then causes many context switches).
I also have some ideas to resolve the tree/session/transport locking issues 
you a trying to address, but this is definitly more for a 2.0 version than a 
1.x.

Lars