[jcifs] 500 Internal Server Error when NTLMv2 session security flag is set

[Michael B Allen]

On Thu, Aug 26, 2010 at 1:53 PM, Anton Prevosti Vives
<prevosti at vivisimo.com> wrote:
> Hi all,
>
> I am trying to use jCIFS NTLM capabilities to implement Apache HTTPClient's NTLMEngine. I am following the  instructions in http://hc.apache.org/httpcomponents-client/ntlm.html.
>
> It works with the following Windows registry flags:
> LMCompatibilityLevel in HKLM\SYSTEM\CurrentControlSet\Control\Lsa  set to 5

Hi Anton,

Registry values have no effect on JCIFS behavior.

> NtlmMinClientSec and NtlmMinServerSec in HKLM\SYSTEM\CurrentControlSet\Control\Lsa\MSV1_0 with the following flags:
>      0x10 Message integrity
>      0x20 Message confidentiality
> 0x20000000 128-bit encryption
>
> However, when I set the flag 0x80000 "NTLMv2 session security" I get the error:

There is no such thing as "NTLMv2 session security". I believe you
mean NTLM2 session security without a 'v' (because NTLM2 session
security can be used with NTLMv1).

> 500.0 0x80090302 SEC_E_UNSUPPORTED_FUNCTION
>
> I am using HTTPClient 4.0.1

HTTP does not use NTLM's session security (because it uses SSL for
encrypted transport instead).

> and JCIFS 1.3.14 and I am setting the following two properties:
>
> jcifs.Config.setProperty("jcifs.smb.NtlmContextlmCompatibility", "5");
> jcifs.Config.setProperty("jcifs.smb.client.useExtendedSecurity", "true");

JCIFS does NTLMv2 by default and thus It is not necessary to set these
properties. In fact the only reason to set these properties would be
to force JCIFS to use NTLMv1.

> The JCIFS documentation for jcifs.smb.lmCompatibility seems to imply that NTLMv2 session security is supported.

NTLM2 session security is partially supported (but again, you don't
need it) in that it will compute the NTLM2 session key used for
signatures and the NTLM2 session security flag is on by default. But I
do not believe NTLM2 confidentiality is supported.

Ultimately the problem is that the NTLM2 session security flag is hard
coded on by default in jcifs/smb/NtlmContext.java:49. If you just
delete that line I think you'll dodge the error.

Mike

-- 
Michael B Allen
Java Active Directory Integration
http://www.ioplex.com/