[jcifs] Character Encioding Problems on OS/390
Bigham, Ted
TBigham at kmart.com
Fri Oct 25 06:29:13 EST 2002
Thanks for the input Allen.
I think you might be looking at things the same way I used to before I
started working with Java in the EBCDIC world. The big thing I learned is
that Java doesn't internally keep the format for strings that are created.
It always converts them to Unicode. You probably already knew that, but the
implication it causes is that toAscii( ) or toEbcdic( ) methods would have
to return byte arrays not strings, since ascii and ebcdic only relate to
bytes not string.
The strings that go into and out of jCIFS are actually perfectly valid
Unicode strings. The UniAddress.getByName() call actually works and finds
my domain controller. Everything is good up until jCIFS does a getBytes()
instead of a getBytes("UTF-8"), to create the SMB packet. I had hoped to
cause a different flow of logic and used a domain name instead (that's where
the true came from, I never took It back out) but it threw a host not found
exception. My problem isn't getting the right strings into jCIFS, it's
getting the strings and byte arrays it internally creates to add the "UTF-8"
encoding. I was digging around in the source and it looks like changing it
everywhere wouldn't be all that big of a chore. I just wish someone else
already did it.
Ted Bigham
-----Original Message-----
From: Allen, Michael B (RSCH) [mailto:Michael_B_Allen at ml.com]
Sent: Wednesday, October 23, 2002 7:40 PM
To: 'Bigham, Ted'; 'jcifs at samba.org'
Subject: RE: [jcifs] Character Encioding Problems on OS/390
> -----Original Message-----
> From: Bigham, Ted [SMTP:TBigham at kmart.com]
> Sent: Wednesday, October 23, 2002 12:32 PM
> To: 'jcifs at samba.org'
> Subject: [jcifs] Character Encioding Problems on OS/390
>
> (using version 7.0b5)
> I'm trying to authenticate my client connections against a domain
controller
> from a servlet. The server (servlet engine) runs on and OS/390 machine
with
> the file.encoding property set to Cp1047 which is IBM EBCDIC. This
setting
> is required to read the .jsp files in their native format, and changing
> isn't (and shouldn't have to be) be an option.
>
Unfortunately there are many places where the jCIFS code assumes
it's using an ASCII
compatible character encoding. This is largely because CIFS is
inherently an ASCII
compatible and UCS-2LE (also ASCII compatible) protocol. This is
illustrated clearly in
the jcifs.util.Log code and it's printHexDump method which is not
printing some of the
logging messages correctly. I cannot fix this. It would require very
pervasive changes. It
is very unusual to run the VM in the EBCDIC locale and yet do IO
with a non-EBCDIC
peers like the CIFS protocol. For webpages this isn't so bad because
you can just run
everything through a converter since HTTP is entirely text based.
You can try to convert
*all* strings going into and coming out of JCIFS. In theory that
*should* work. You could
write a little toEbcdic and toAscii methods easily and do:
public static void main(String[] args) throws Exception
{
jcifs.util.Log.setMask(jcifs.util.Log.ALL);
UniAddress domainController =
UniAddress.getByName(toAscii("148.162.36.216"));
byte[] challenge =
SmbSession.getChallenge(domainController);
System.out.println(toEbcdic(domainController));
for (int i=0; i < challenge.length; i++)
System.out.print(challenge[i] + ":");
System.out.println();
}
But this will probably require some judicious hacking and in the end
there's a good chance
it just wouldn't work because of some assumption made in the jCIFS
code. Also note, you
do not need the second 'true' parameter for getByName. You're not
looking up a workgroup
or domain name. You already have the IP.
> This is effecting the way the message blocks are being generated. Trying
to
> call SmbSession.getChallenge( ) fails. The message block in the log looks
> very corrupted (to me), and the server sends back a rejection message
> (called name not present).
>
> The following command line test class shows this problem well. All it's
> supposed to do is get a challenge from the domain controller...
>
> public static void main(String[] args) throws Exception
> {
> jcifs.util.Log.setMask(jcifs.util.Log.ALL);
> UniAddress domainController = UniAddress.getByName("148.162.36.216",
> true);
> byte[] challenge = SmbSession.getChallenge(domainController);
>
> System.out.println(domainController);
> for (int i=0; i < challenge.length; i++)
> System.out.print(challenge[i] + ":");
> System.out.println();
> }
>
> Running with the default file.encoding (Cp1047) gives the following output
> (trimmed for relevance). Notice the messed up host name in the
exception...
> ========================= BEGIN OUTPUT
> ================================================
> Oct 23 12:06:17.646 - name service address cache
> JCIFS175_2_??<?É> JCIFS175_2_??<?É>/148.162.175.2
> 0.0.0.0<ÉÉ> 0.0.0.0<ÉÉ>/0.0.0.0
>
> Oct 23 12:06:17.790 - smb negotiation warning
> requesting negotiation with 0.0.0.0<ÉÉ>/148.162.36.216
> Oct 23 12:06:17.941 - smb warning
> ERR_SSN_SRVC/Called name not present
> Oct 23 12:06:17.964 - nbt name service debugging
> new data read from socket
> Oct 23 12:06:17.967 - nbt name service packet sent
>
NodeStatusRequest[nameTrnId=1,isResponse=false,opCode=QUERY,isAuthAnswer=fal
> se,isTruncated=false,isRecurAvailable=false,isRecurDesir
> Oct 23 12:06:17.969 - datagram packet sent to: 148.162.36.216
> 00000: 00 01 00 00 00 01 00 00 00 00 00 00 20 43 4B 41 |............ CKA|
> 00010: 41 41 41 41 41 41 41 41 41 41 41 41 41 41 41 41 |AAAAAAAAAAAAAAAA|
> 00020: 41 41 41 41 41 41 41 41 41 41 41 41 41 00 00 21 |AAAAAAAAAAAAA..!|
> 00030: 00 01 |.. |
>
> Oct 23 12:06:17.972 - nbt name service packet receviced
>
NodeStatusResponse[nameTrnId=1,isResponse=true,opCode=QUERY,isAuthAnswer=tru
> e,isTruncated=false,isRecurAvailable=false,isRecurDesire...
> Oct 23 12:06:17.983 - datagram packet received from: 148.162.36.216
> 00000: 00 01 84 00 00 00 00 01 00 00 00 00 20 43 4B 41 |............ CKA|
> 00010: 41 41 41 41 41 41 41 41 41 41 41 41 41 41 41 41 |AAAAAAAAAAAAAAAA|
> 00020: 41 41 41 41 41 41 41 41 41 41 41 41 41 00 00 21 |AAAAAAAAAAAAA..!|
> 00030: 00 01 00 00 00 00 00 53 02 55 53 4B 49 48 53 56 |.......S.USKIHSV|
> 00040: 50 57 49 4E 53 43 4C 31 20 44 00 55 53 4B 49 48 |PWINSCL1 D.USKIH|
> 00050: 53 56 50 57 49 4E 53 43 4C 31 00 44 00 00 03 47 |SVPWINSCL1.D...G|
> 00060: 78 5A 73 00 00 00 00 00 00 00 00 00 00 00 00 00 |xZs.............|
> 00070: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
> 00080: 00 00 00 00 00 00 00 00 00 00 00 00 49 00 44 00 |............I.D.|
> 00090: 5C 00 7B 00 32 00 30 00 38 00 44 00 32 00 43 00 |\.{.2.0.8.D.2.C.|
> 000A0: 36 00 30 00 2D 00 33 00 41 00 45 00 41 00 2D 00 |6.0.-.3.A.E.A.-.|
> 000B0: 31 00 30 00 36 00 39 00 2D 00 41 00 32 00 44 00 |1.0.6.9.-.A.2.D.|
> 000C0: 37 00 2D 00 30 00 38 00 30 00 30 00 32 00 42 00 |7.-.0.8.0.0.2.B.|
> 000D0: 33 00 30 00 33 00 30 00 39 00 44 00 7D 00 5C 00 |3.0.3.0.9.D.}.\.|
> 000E0: 49 00 6E 00 50 00 72 00 6F 00 63 00 53 00 65 00 |I.n.P.r.o.c.S.e.|
> 000F0: 72 00 76 00 65 00 72 00 33 00 32 00 00 00 00 00 |r.v.e.r.3.2.....|
> 00100: 00 00 65 00 61 00 26 00 30 |..e.a.&.0 |
>
> Oct 23 12:06:17.991 - smb warning
> ERR_SSN_SRVC/Called name not present
> Exception in thread "main" jcifs.smb.SmbException: Failed to establish
> session with --.-¦-+&--+--<æ<ÉÉ>/148.162.36.216
> at jcifs.smb.SmbTransport.send(SmbTransport.java:478)
> at jcifs.smb.SmbTransport.negotiate(SmbTransport.java:666)
> at jcifs.smb.SmbSession.getChallenge(SmbSession.java:34)
> at Test.main(Test.java:11)
> ========================== END OUTPUT
> ============================================
>
>
>
> Running the same program with the java option -Dfile.encoding=UTF-8, or
> running it on windows machine gives the following correct output...
>
> ========================= BEGIN OUTPUT
> ================================================
> Oct 23 12:10:27.592 - name service address cache
> JCIFS175_2_BC<20> JCIFS175_2_BC<20>/148.162.175.2
> 0.0.0.0<00> 0.0.0.0<00>/0.0.0.0
>
> Oct 23 12:10:27.769 - smb negotiation warning
> requesting negotiation with 0.0.0.0<00>/148.162.36.216
> Oct 23 12:10:27.962 - session service warning
> session established ok with 0.0.0.0<00>/148.162.36.216
> Oct 23 12:10:27.993 - smb sent
>
SmbComNegotiate[command=SMB_COM_NEGOTIATE,received=false,errorCode=0x0000000
> 0,flags=0x0018,flags2=0x8001,tid=0,pid=35199,uid=0,mid=1...
>
> Oct 23 12:10:27.996 - smb sent
> 00000: FF 53 4D 42 72 00 00 00 00 18 01 80 00 00 00 00 |MBr...........|
> 00010: 00 00 00 00 00 00 00 00 00 00 7F 89 00 00 01 00 |................|
> 00020: 00 0C 00 02 4E 54 20 4C 4D 20 30 2E 31 32 00 |....NT LM 0.12. |
>
> Oct 23 12:10:28.007 - smb transport warning
> new data read from socket
> Oct 23 12:10:28.104 - smb packet format warning
> byteCount=54 but readBytesWireFormat returned 26
> Oct 23 12:10:28.146 - smb received
>
SmbComNegotiateResponse[command=SMB_COM_NEGOTIATE,received=true,errorCode=0x
> 00000000,flags=0x0098,flags2=0x8001,tid=0,pid=35199,uid=...
>
> Oct 23 12:10:28.162 - smb received
> 00000: FF 53 4D 42 72 00 00 00 00 98 01 80 00 00 00 00 |MBr...........|
> 00010: 00 00 00 00 00 00 00 00 00 00 7F 89 00 00 01 00 |................|
> 00020: 11 00 00 03 32 00 01 00 04 41 00 00 00 00 01 00 |....2....A......|
> 00030: 00 00 00 00 FD F3 00 00 8A 66 E0 1D AF 7A C2 01 |....²=...fa.»z-.|
> 00040: F0 00 08 36 00 75 B5 CB C0 C7 E1 6B 4E 4B 00 4D |=..6.u¦-+¦ßkNK.M|
> 00050: 00 53 00 45 00 52 00 56 00 45 00 52 00 53 00 00 |.S.E.R.V.E.R.S..|
> 00060: 00 55 00 53 00 4B 00 49 00 48 00 53 00 56 00 50 |.U.S.K.I.H.S.V.P|
> 00070: 00 50 00 52 00 54 00 31 00 00 00 |.P.R.T.1... |
>
> 0.0.0.0<00>/148.162.36.216
> 117:-75:-53:-64:-57:-31:107:78:
> ========================== END OUTPUT
> ============================================
>
>
> This shows that it's very possible to make it work in this environment. I
> only need to get the jcifs code to quit assuming it's in an ASCII
> environment.
>
> As stated before, changing the file encoding for the entire web server is
> not a viable option. Does anyone know how to make this work as is, or
will
> the internal jcifs code have to be modified to specify UTF-8 encoding when
> it converts its byte arrays to strings (if that's really what the problem
> is)?
>
> p.s.
> This also shows a small bug in the hex dump code where some of the
> characters that tried print weren't printable. That makes it looks like
the
> " SMB" header is incorrect, but it's actually right.
>
> Thanks to all in advance
> Ted Bigham
>
More information about the jcifs
mailing list