[jcifs] Ntlm challenge fails with connection reset on WebSphere6.1

[Eric Gilbertson]

Hello Jcifs Experts:

I would be greatly appreciated if someone could help me with what appears to be an app server dependent problem with WebSphere 6.1 (funny how all our weird problems are somehow related to WebSphere). We are using jcifs-1.2.13 in our J2EE app to provide NTLM authentication using the servlet described in the jcifs docs and this works when deployed on WebSphere 5.1 and WebLogic 8.1. The problem is that the authentication response phase fails and the connection is reset when run on WebSphere 6.1 (see log snipette below). The headers in the fail scenario look the same as the headers on a good challenge. If have examined the event log on the domain controller there are no error messages in it. It appears the everything is normal except that someone is prematurely closing the socke connection. Tweaking the socket timeout and setting ssnLimit to 1 has no effect. The domain controller runs Windows 2003.

Any suggestions as to how to resolve this issue would be greatly appreciated.

TIA,

Eric Gilbertson

Config summary:
Jcifs version: 1.2.13
App server: WebSphere6.1 on Windows XP
Domain controller: Windows Server 2003R2
jcifs.http.domainController=<DC_IP_ADDRESS>
jcifs.netbios.wins=<WINS_IP_ADDRESS>
jcifs.smb.client.soTimeout=35000
jcifs.smb.client.ssnLimit=1
Jcifs.util.loglevel=5

Log info:
[3/21/07 13:25:54:833 PDT] 00000024 SystemErr     R SmbComNegotiate[command=SMB_COM_NEGOTIATE,received=false,errorCode=0,flags=0x0018,flags2=0xC003,signSeq=0,tid=0,pid=24941,uid=0,mid=1,wordCount=0,byteCount=12,wordCount=0,dialects=NT LM 0.12]
[3/21/07 13:25:54:833 PDT] 00000024 SystemErr     R New data read: Transport1[0.0.0.0<00>/10.10.100.198:0]
[3/21/07 13:25:54:833 PDT] 00000024 SystemErr     R 00000: FF 53 4D 42 72 00 00 00 00 98 03 C0 00 00 00 00  |±µ²SMBr......¶µ....|
00010: 00 00 00 00 00 00 00 00 00 00 6D 61 00 00 01 00  |..........ma....|
[3/21/07 13:25:54:833 PDT] 00000024 SystemErr     R 
[3/21/07 13:25:54:833 PDT] 00000024 SystemErr     R byteCount=32 but readBytesWireFormat returned 18
[3/21/07 13:25:54:833 PDT] 00000024 SystemErr     R SmbComNegotiateResponse[command=SMB_COM_NEGOTIATE,received=false,errorCode=0,flags=0x0098,flags2=0xC003,signSeq=0,tid=0,pid=24941,uid=0,mid=1,wordCount=17,byteCount=32,wordCount=17,dialectIndex=0,securityMode=0xF,security=user,encryptedPasswords=true,maxMpxCount=50,maxNumberVcs=1,maxBufferSize=16644,maxRawSize=65536,sessionKey=0x00000000,capabilities=0x0001F3FD,serverTime=Wed Mar 21 13:25:54 PDT 2007,serverTimeZone=420,encryptionKeyLength=8,byteCount=32,encryptionKey=0xDF2B4232DDF35F49,oemDomainName=SSEAL]
[3/21/07 13:25:54:849 PDT] 00000024 SystemErr     R NodeStatusRequest[nameTrnId=1,isResponse=false,opCode=QUERY,isAuthAnswer=false,isTruncated=false,isRecurAvailable=false,isRecurDesired=false,isBroadcast=false,resultCode=0,questionCount=1,answerCount=0,authorityCount=0,additionalCount=0,questionName=*              <00>,questionType=0x0021,questionClass=IN,recordName=null,recordType=0x0000,recordClass=0x0000,ttl=0,rDataLength=0]
[3/21/07 13:25:54:849 PDT] 00000024 SystemErr     R 00000: 00 01 00 00 00 01 00 00 00 00 00 00 20 43 4B 41  |............ CKA|
00010: 41 41 41 41 41 41 41 41 41 41 41 41 41 41 41 41  |AAAAAAAAAAAAAAAA|
00020: 41 41 41 41 41 41 41 41 41 41 41 41 41 00 00 21  |AAAAAAAAAAAAA..!|
00030: 00 01                                            |..              |
[3/21/07 13:25:54:849 PDT] 00000024 SystemErr     R 
[3/21/07 13:25:54:849 PDT] 00000025 SystemErr     R NetBIOS: new data read from socket
[3/21/07 13:25:54:849 PDT] 00000025 SystemErr     R NodeStatusResponse[nameTrnId=1,isResponse=true,opCode=QUERY,isAuthAnswer=true,isTruncated=false,isRecurAvailable=false,isRecurDesired=false,isBroadcast=false,resultCode=0,questionCount=0,answerCount=1,authorityCount=0,additionalCount=0,questionName=null,questionType=0x0000,questionClass=IN,recordName=*              <00>,recordType=0x0021,recordClass=IN,ttl=0,rDataLength=191]
[3/21/07 13:25:54:849 PDT] 00000025 SystemErr     R 00000: 00 01 84 00 00 00 00 01 00 00 00 00 20 43 4B 41  |............ CKA|
00010: 41 41 41 41 41 41 41 41 41 41 41 41 41 41 41 41  |AAAAAAAAAAAAAAAA|
....
00100: 00 00 00 00 00 00 00 00 00                       |.........       |
[3/21/07 13:25:54:849 PDT] 00000025 SystemErr     R 
[3/21/07 13:27:55:507 PDT] 00000024 SystemErr     R java.net.SocketException: Connection reset
[3/21/07 13:27:55:522 PDT] 00000024 SystemErr     R 	at java.net.SocketInputStream.read(SocketInputStream.java:197)
[3/21/07 13:27:55:522 PDT] 00000024 SystemErr     R 	at jcifs.util.transport.Transport.readn(Transport.java:29)
[3/21/07 13:27:55:522 PDT] 00000024 SystemErr     R 	at jcifs.smb.SmbTransport.peekKey(SmbTransport.java:355)
[3/21/07 13:27:55:522 PDT] 00000024 SystemErr     R 	at jcifs.util.transport.Transport.loop(Transport.java:100)
[3/21/07 13:27:55:522 PDT] 00000024 SystemErr     R 	at jcifs.util.transport.Transport.run(Transport.java:265)
[3/21/07 13:27:55:522 PDT] 00000024 SystemErr     R 	at java.lang.Thread.run(Thread.java:797)