Intermittent failure to authenticate using NTLM with NT_STATUS_ACCESS_DENIED

[Richard Sharpe]

On Fri, Oct 9, 2015 at 9:24 AM, Richard Sharpe
<realrichardsharpe at gmail.com> wrote:
> On Thu, Oct 8, 2015 at 9:35 PM, Richard Sharpe
> <realrichardsharpe at gmail.com> wrote:
>> On Thu, Oct 8, 2015 at 9:22 PM, Stefan Metzmacher <metze at samba.org> wrote:
>>> Am 09.10.2015 um 01:19 schrieb Jeremy Allison:
>>>> On Thu, Oct 08, 2015 at 04:11:19PM -0700, Richard Sharpe wrote:
>>>>> Hi folks,
>>>>>
>>>>> We are intermittently seeing NTLM auth failing with
>>>>> NT_STATUS_ACCESS_DENIED and we see this message in winbindd.log:
>>>>>
>>>>> [2015/10/08 15:34:33.393987,  3, pid=3549, effective(0, 0), real(0,
>>>>> 0), class=winbind]
>>>>> ../source3/winbindd/winbindd_pam.c:1426(winbind_samlogon_retry_loop)
>>>>>   winbind_samlogon_retry_loop: sam_logon returned ACCESS_DENIED.
>>>>> Maybe the trust account password was changed and we didn't know it.
>>>>> Killing connections to domain SOMEDOM
>>>>>
>>>>> Now, the real reason seems to be that one of the DCs in that domain
>>>>> disallows NTLM authentication and whenever winbindd finds that DC we
>>>>> get this problem.
>>>>>
>>>>> Is there some way to tell Windindd not to use that DC?
>>>>>
>>>>> Also, I notice that in some instances in winbind_samlogon_retry_loop
>>>>> we move to another DC but not in this case. We simply retry with the
>>>>> same DC.
>>>>>
>>>>> I suspect that we should move to another DC in this case as well.
>>>>>
>>>>> Any comments?
>>>>
>>>> Yep - getting  ACCESS_DENIED should certainly trigger adding
>>>> the DC to the negative connection cache.
>>>
>>> But not an the first failure!
>>
>> Hmmm, why not. If it is returning ACCESS_DENIED either someone has
>> changed the machine account password without telling us or that DC
>> does not like NTLM passthrough ...
>
> OK, so there is a situation where we could get access denied because
> the machine account password has changed. Let's say we are already
> connected and someone changes it. In that case we would not want to
> black-list it, but just connect to another DC.
>
> Maybe what I need to do is to increase the retry count to three.
>
> However the failure I was seeing seemed to occur on the retry as well,
> because we found the same DC name again and connected to it.
>
> Maybe all I need to do is deprecate that name? Perhaps remove it from
> gencache ...

Having stared at that code a lot now, I think the thing to do is call
saf_delete(domain->name).

This will remove affinity to that name and allow us to try a different DC.

There could still be intermittent successes if there was one server
that allowed pass through auth while the others didn't.

-- 
Regards,
Richard Sharpe
(何以解憂？唯有杜康。--曹操)