[Samba] Win7pro can't authenticate dcom identity

Chris Perry outtascope at gmail.com
Fri Oct 28 10:12:23 MDT 2011

On Wed, Oct 26, 2011 at 7:25 AM, Chris Perry <outtascope at gmail.com> wrote:

> I have a Samba 3.4.7 PDC set up (Ubuntu 10.10/OpenLDAP) and have configured
> a Windows 7 Pro 64 bit workstation as a domain member. Logins and shares
> work without a hitch.
> I'm running into a problem with ArcGIS 10 Server on Windows using domain
> accounts for the services.  ArcGIS post-install fails because it won't take
> the username/password combination of the domain account.  I initially
> thought this was an ArcGIS problem, but looking at the Samba logs, I find
> "check_ntlm_password: sam authentication for user [ArcGISSOC] FAILED with
> error NT_STATUS_WRONG_PASSWORD" entries whenever I try to configure the
> service.

Ok, I have fixed this problem on my site by patching Samba 3.4.7 on Ubuntu
10.04 (I incorrectly said 10.10 originally).  It isn't clear to me if this
is a Samba bug, a Windows bug, or a misconfiguration on either end of my
systems, but the following does fix my problem.

The issue appears to (my neophyte eyes) to be that login requests that fail
ntlmv2 authentication are not falling through gracefully to lm auth.  In
Samba, the code that determines whether or not to process is an ntlmv2
request checks the length parameter of the nt field in the RPC request.  If
this field is greater than or equal to 24, it processes as ntlmv2.  If it
processes the ntlmv2 check and it fails, it returns an
NT_STATUS_WRONG_PASSWORD to the client before it ever has a chance to check
Now I have played around with these auth settings on my Windows machine
endlessly and it does not seem to have an effect on what authentication
dcomcnfg uses when setting the identity of a component.
My instinct on this issue is that whatever initially processes the rpc
request in Samba is creating a zero-filled data element for both methods and
setting the length accordingly, regardless of whether the particular method
was requested or not (probably to avoid null pointer errors).  The ntlmv2
check assumes that the length should be zero for the nt data element if it
wasn't requested.  My patch additionally checks that the nt response
property is not just a zero-filled array before deciding that it is an
ntlmv2 request.

Again, I can't say that this is the right solution and I'm sure there are
some enormous deficits in my understanding of how this is supposed to work,
but this patch does work if you are willing to compile yourself.  This
section of code has been changed quite a bit in Samba 3.5 and up, so it may
not be an issue on those new versions, but I can't say for sure.

If anyone has any additional insight into this (ie., I have an obvious
configuration error, I have misunderstood something, or I have created a
gaping security hole with this patch) I would appreciate the feedback.


- Chris

Extract from log files of failing configuration (real data replaced with
01234, 0000... sequences are directly from log).

in.logon.network.nt and in.logon.network.lm properties of netr_LogonSamLogon
struct at rpc_server/srv_pipe.c:2327(api_rpcTNP)

>From request generated when setting Identity property of DCOM object in

nt: struct netr_ChallengeResponse
  length : 0x002c (44)
  size   : 0x002c (44)
  data   : *
    data :
lm: struct netr_ChallengeResponse
  length : 0x0018 (24)
  size   : 0x0018 (24)
  data   : *
    data : 012345678901234567890123456789010000000000000000

>From domain login on the same workstation

nt: struct netr_ChallengeResponse
  length : 0x010e (270)
  size   : 0x010e (270)
  data   : *
    data : 01234567890123456789012345 ... 01234567890123456789012345
lm: struct netr_ChallengeResponse
  length : 0x0018 (24)
  size   : 0x0018 (24)
  data   : *
    data : 000000000000000000000000000000000000000000000000

Patch to source3/libsmb/ntlm_check.c that makes this work:

--- samba-3.4.7~dfsg/source3/libsmb/ntlm_check.c        2011-10-28
11:05:38.000000000 -0400
+++ samba-3.4.7~dfsg-modified/source3/libsmb/ntlm_check.c       2011-10-28
11:41:12.000000000 -0400
@@ -28,6 +28,21 @@
  Core of smb password checking routine.
+static bool nt_response_empty(TALLOC_CTX *mem_ctx,
+                              const DATA_BLOB *nt_response)
+        if(nt_response->length == 0)
+                return true;
+        DATA_BLOB empty_response = data_blob_talloc_zero(mem_ctx,
+        if(memcmp(nt_response->data, empty_response.data,
nt_response->length)) {
+                data_blob_free(&empty_response);
+                return false;
+        }
+        data_blob_free(&empty_response);
+        return true;

 static bool smb_pwd_check_ntlmv1(const DATA_BLOB *nt_response,
                                 const uchar *part_passwd,
@@ -283,7 +298,7 @@
                         (unsigned long)nt_response->length, username));

-       if (nt_response->length >= 24 && nt_pw) {
+       if (nt_response->length >= 24 && nt_pw &&
!nt_response_empty(mem_ctx, nt_response)) {
                if (nt_response->length > 24) {
                        /* We have the NT MD4 hash challenge available - see
if we can
                           use it

More information about the samba mailing list