[Samba] Failed to find [principal](kvno 4) in keytab MEMORY:cifs_srv_keytab (arcfour-hmac-md5)]

Jonathon Reinhart jonathon.reinhart at gmail.com
Mon Dec 16 20:22:33 UTC 2019

Hello everyone,

I have a FreeNAS server (9.10 running samba 4.3.11-GIT-UNKNOWN) that's recently
started emitting this error:

gss_accept_sec_context failed with [ Miscellaneous failure (see text):
Failed to find cifs/nas01 at EXAMPLE.COM(kvno 4) in keytab
MEMORY:cifs_srv_keytab (arcfour-hmac-md5)]

I've looked at bug 12262 [1], which is why I've cc'd Stefan Metzmacher.
I don't think this is is the same as that bug, and I will explain why.

I've looked at a lot of mailing list/forum posts, and the problem either
"goes away", or the conversation ends. [2][3][4][5][6]

Here's my understanding of the problem: The message "Starting GENSEC
submechanism gse_krb5" shows up every time a new connection comes in. That
means this happens each connection:

- gensec_gse_server_start
  - gse_init_server
    - gse_context_init
      - krb5_init_context(&gse_ctx->k5ctx)
    - gse_krb5_get_server_keytab

FreeNAS generates an smb4.conf without 'kerberos method', so it defaults to
'secrets only'. That means only fill_mem_keytab_from_secrets() is used.
That function gets the (current and previous) machine password from
secrets.tdb, which I can dump with:

tdbdump /var/db/system/samba4/private/secrets.tdb

- I've obviously tried restarting the smb service
- I've tried having users log-off/reboot
- I've performed an un-join and re-join (using FreeNAS interface), and I can
  see the machine password changing, as well as the kvno number in the error

Here's why I'm particularly confused:
- This is not tied to any particular end user. User A was unable to connect to
  the share from his machine, but was able to from a different machine.
- It seems to be tied to a client machine. Users A and B were unable to connect
  to the share from a problematic machine.
- I took a wireshark capture of a successful "Session Setup Request" from my
  machine, and an unsuccessful request from User A's machine. In both cases,
  the GSSAPI->SPNEGO->Kerberos AP-REQ ticket looked the same:

    tkt-vno: 5
    realm: EXAMPLE.COM
      name-type: kRB5-NT-SRV-INST (2)
      sname-string: 2 items
        SNameString: cifs
        SNameString: nas01.example.com
      etype: eTYPE-ARCFOUR-HMAC-MD5 (23)cifs
      kvno: 4
      cipher: [random]

Since both "good" and "bad" requests have the same principal, that implies that
the MEMORY:cifs_srv_keytab keytab ends up with different contents depending on
the client machine that connects O_o  -- This is why I don't think it is
(directly) related to 12262.

Unfortunately because the keytab is in memory, I have no (remotely easy) way of
seeing what's in it. And even with 'log level = 19', there is no debug output
in the path where the keytab is being built.

So I'm really hoping a developer can step in here and help understand a few

A) Am I correct in my understanding that the in-memory keytab is created on
   each request? How could it possibly be different for different client

B) How does fill_mem_keytab_from_secrets() work without ever setting kvno?

      krb5_kvno kvno = 0; /* FIXME: fetch current vno from KDC ? */

  This is particularly interesting because fill_keytab_from_password() is
  called with old(er) machine passwords, passing `kvno - 1` and `kvno - 2`
  which will will be (-1) and (-2) which seem completely invalid.

  There's a lot of confusing 'vno' logic in krb5_kt_get_entry().

C) Is there any other information I'm not looking at that would be helpful
   in understanding this?

Jonathon Reinhart

[1] https://bugzilla.samba.org/show_bug.cgi?id=12262
[2] https://www.google.com/search?q=%22Failed+to+find%22+kvno+%22keytab+MEMORY%3Acifs_srv_keytab%22
[3] https://lists.samba.org/archive/samba/2016-September/203338.html
[4] https://redmine.ixsystems.com/issues/6228
[5] https://www.ixsystems.com/community/threads/36439
[6] https://www.ixsystems.com/community/threads/61835

More information about the samba mailing list