[Samba] Samba AD/DC crashed again, third time in as many months

[Rowland penny]

On 02/03/16 20:46, IT Admin wrote:
> Any further information required?  Any troubleshooting steps?  I need to
> get this domain functional ASAP.
>
> Please advise.
>
> JS
> On Mar 2, 2016 11:39 AM, "IT Admin" <it at cliffbells.com> wrote:
>
>> Hello to the samba users list again.  I've got an emergency here, a Samba
>> AD DC I've got deployed has crashed again, this is the third time since
>> 12/17/2015 that this domain has failed completely.
>>
>> No power outages or unexpected shutdowns, samba simply fails to start, it
>> appears my database is yet again corrupt:
>>
>> [2016/03/02 11:27:17.025235,  0]
>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>    ldb: dsdb_get_schema: refresh_fn() failed
>> [2016/03/02 11:27:17.025346,  0]
>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>    ldb: schema_load_init: dsdb_get_schema failed
>> [2016/03/02 11:27:17.025377,  0]
>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>    ldb: module schema_load initialization failed : Operations error
>> [2016/03/02 11:27:17.025404,  0]
>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>    ldb: module rootdse initialization failed : Operations error
>> [2016/03/02 11:27:17.025431,  0]
>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>    ldb: module samba_dsdb initialization failed : Operations error
>> [2016/03/02 11:27:17.025460,  0]
>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>    ldb: Unable to load modules for /usr/local/samba/private/sam.ldb:
>> schema_load_init: dsdb_get_schema failed
>> [2016/03/02 11:27:17.032681,  0]
>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>    ldb: dsdb_get_schema: refresh_fn() failed
>> [2016/03/02 11:27:17.033626,  0]
>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>> [2016/03/02 11:27:17.033776,  0]
>> ../source4/smbd/service_task.c:35(task_server_terminate)
>>    task_server_terminate: [nbtd failed to open samdb]
>>    ldb: schema_load_init: dsdb_get_schema failed
>> [2016/03/02 11:27:17.034676,  0]
>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>    ldb: module schema_load initialization failed : Operations error
>> [2016/03/02 11:27:17.035224,  0]
>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>    ldb: module rootdse initialization failed : Operations error
>> [2016/03/02 11:27:17.035838,  0]
>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>    ldb: module samba_dsdb initialization failed : Operations error
>> [2016/03/02 11:27:17.036392,  0]
>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>    ldb: Unable to load modules for /usr/local/samba/private/sam.ldb:
>> schema_load_init: dsdb_get_schema failed
>> [2016/03/02 11:27:17.038675,  0]
>> ../source4/smbd/service_task.c:35(task_server_terminate)
>> [2016/03/02 11:27:17.040353,  0]
>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>> [2016/03/02 11:27:17.042764,  0]
>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>    ldb: dsdb_get_schema: refresh_fn() failed
>> [2016/03/02 11:27:17.042857,  0]
>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>    ldb: schema_load_init: dsdb_get_schema failed
>> [2016/03/02 11:27:17.042887,  0]
>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>    ldb: module schema_load initialization failed : Operations error
>> [2016/03/02 11:27:17.042917,  0]
>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>    ldb: module rootdse initialization failed : Operations error
>> [2016/03/02 11:27:17.042942,  0]
>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>    ldb: module samba_dsdb initialization failed : Operations error
>> [2016/03/02 11:27:17.042971,  0]
>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>    ldb: Unable to load modules for /usr/local/samba/private/sam.ldb:
>> schema_load_init: dsdb_get_schema failed
>>    ldb: dsdb_get_schema: refresh_fn() failed
>> [2016/03/02 11:27:17.045981,  0]
>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>    ldb: schema_load_init: dsdb_get_schema failed
>> [2016/03/02 11:27:17.046598,  0]
>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>    ldb: module schema_load initialization failed : Operations error
>> [2016/03/02 11:27:17.047302,  0]
>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>> [2016/03/02 11:27:17.047523,  0]
>> ../source4/smbd/service_task.c:35(task_server_terminate)
>>    task_server_terminate: [dreplsrv: Failed to connect to local samdb:
>> WERR_DS_UNAVAILABLE
>>    ]
>>    ldb: module rootdse initialization failed : Operations error
>> [2016/03/02 11:27:17.048503,  0]
>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>    ldb: module samba_dsdb initialization failed : Operations error
>> [2016/03/02 11:27:17.048540,  0]
>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>    ldb: Unable to load modules for /usr/local/samba/private/sam.ldb:
>> schema_load_init: dsdb_get_schema failed
>> [2016/03/02 11:27:17.050402,  0]
>> ../source4/smbd/service_task.c:35(task_server_terminate)
>>    task_server_terminate: [kdc: krb5_init_context samdb connect failed]
>>    task_server_terminate: [cldapd failed to open samdb]
>> [2016/03/02 11:27:17.072670,  0]
>> ../source4/smbd/server.c:211(samba_terminate)
>>    samba_terminate: dreplsrv: Failed to connect to local samdb:
>> WERR_DS_UNAVAILABLE
>>
>> [2016/03/02 11:27:17.073896,  0]
>> ../source4/smbd/server.c:211(samba_terminate)
>>    samba_terminate: nbtd failed to open samdb
>> [2016/03/02 11:27:17.075031,  0]
>> ../source4/smbd/server.c:211(samba_terminate)
>>    samba_terminate: cldapd failed to open samdb
>> [2016/03/02 11:27:17.075614,  0]
>> ../source4/smbd/server.c:211(samba_terminate)
>>    samba_terminate: kdc: krb5_init_context samdb connect failed
>> [2016/03/02 11:27:17.086051,  0]
>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>    ldb: dsdb_get_schema: refresh_fn() failed
>> [2016/03/02 11:27:17.091786,  0]
>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>    ldb: dsdb_get_schema: refresh_fn() failed
>> [2016/03/02 11:27:17.091860,  0]
>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>    ldb: schema_load_init: dsdb_get_schema failed
>> [2016/03/02 11:27:17.091890,  0]
>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>    ldb: module schema_load initialization failed : Operations error
>> [2016/03/02 11:27:17.091919,  0]
>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>    ldb: module rootdse initialization failed : Operations error
>> [2016/03/02 11:27:17.091946,  0]
>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>    ldb: module samba_dsdb initialization failed : Operations error
>> [2016/03/02 11:27:17.091974,  0]
>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>    ldb: Unable to load modules for /usr/local/samba/private/sam.ldb:
>> schema_load_init: dsdb_get_schema failed
>> [2016/03/02 11:27:17.093919,  0]
>> ../source4/smbd/service_task.c:35(task_server_terminate)
>>    task_server_terminate: [dns: samdb_connect failed]
>> [2016/03/02 11:27:17.095175,  0]
>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>    ldb: schema_load_init: dsdb_get_schema failed
>> [2016/03/02 11:27:17.095232,  0]
>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>    ldb: module schema_load initialization failed : Operations error
>> [2016/03/02 11:27:17.095261,  0]
>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>    ldb: module rootdse initialization failed : Operations error
>> [2016/03/02 11:27:17.095288,  0]
>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>    ldb: module samba_dsdb initialization failed : Operations error
>> [2016/03/02 11:27:17.095317,  0]
>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>    ldb: Unable to load modules for /usr/local/samba/private/sam.ldb:
>> schema_load_init: dsdb_get_schema failed
>> [2016/03/02 11:27:17.095940,  0]
>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>    ldb: dsdb_get_schema: refresh_fn() failed
>> [2016/03/02 11:27:17.097255,  0]
>> ../source4/smbd/service_task.c:35(task_server_terminate)
>> [2016/03/02 11:27:17.098404,  0]
>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>    task_server_terminate: [kccsrv: Failed to connect to local samdb:
>> WERR_DS_UNAVAILABLE
>>    ]
>> [2016/03/02 11:27:17.102052,  0]
>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>    ldb: schema_load_init: dsdb_get_schema failed
>> [2016/03/02 11:27:17.102608,  0]
>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>    ldb: module schema_load initialization failed : Operations error
>> [2016/03/02 11:27:17.103164,  0]
>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>    ldb: module rootdse initialization failed : Operations error
>> [2016/03/02 11:27:17.103210,  0]
>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>    ldb: module samba_dsdb initialization failed : Operations error
>> [2016/03/02 11:27:17.103241,  0]
>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>    ldb: Unable to load modules for /usr/local/samba/private/sam.ldb:
>> schema_load_init: dsdb_get_schema failed
>>    ldb: dsdb_get_schema: refresh_fn() failed
>> [2016/03/02 11:27:17.103939,  0]
>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>    ldb: schema_load_init: dsdb_get_schema failed
>> [2016/03/02 11:27:17.103974,  0]
>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>    ldb: module schema_load initialization failed : Operations error
>> [2016/03/02 11:27:17.104003,  0]
>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>    ldb: module rootdse initialization failed : Operations error
>> [2016/03/02 11:27:17.107759,  0]
>> ../source4/smbd/service_task.c:35(task_server_terminate)
>>    task_server_terminate: [dnsupdate: Failed to connect to local samdb
>>    ]
>> [2016/03/02 11:27:17.109612,  0]
>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>    ldb: module samba_dsdb initialization failed : Operations error
>> [2016/03/02 11:27:17.109672,  0]
>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>    ldb: Unable to load modules for /usr/local/samba/private/sam.ldb:
>> schema_load_init: dsdb_get_schema failed
>> [2016/03/02 11:27:17.114754,  0]
>> ../source4/smbd/service_task.c:35(task_server_terminate)
>>    task_server_terminate: [ntp_signd failed to open samdb]
>> [2016/03/02 11:27:17.731614,  0]
>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>    ldb: dsdb_get_schema: refresh_fn() failed
>> [2016/03/02 11:27:17.731721,  0]
>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>    ldb: schema_load_init: dsdb_get_schema failed
>> [2016/03/02 11:27:17.731750,  0]
>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>    ldb: module schema_load initialization failed : Operations error
>> [2016/03/02 11:27:17.731779,  0]
>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>    ldb: module rootdse initialization failed : Operations error
>> [2016/03/02 11:27:17.731807,  0]
>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>    ldb: module samba_dsdb initialization failed : Operations error
>> [2016/03/02 11:27:17.731835,  0]
>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>    ldb: Unable to load modules for /usr/local/samba/private/sam.ldb:
>> schema_load_init: dsdb_get_schema failed
>> [2016/03/02 11:27:17.762162,  0]
>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>    ldb: dsdb_get_schema: refresh_fn() failed
>> [2016/03/02 11:27:17.762247,  0]
>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>    ldb: schema_load_init: dsdb_get_schema failed
>> [2016/03/02 11:27:17.762276,  0]
>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>    ldb: module schema_load initialization failed : Operations error
>> [2016/03/02 11:27:17.762305,  0]
>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>    ldb: module rootdse initialization failed : Operations error
>> [2016/03/02 11:27:17.762332,  0]
>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>    ldb: module samba_dsdb initialization failed : Operations error
>> [2016/03/02 11:27:17.762361,  0]
>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>    ldb: Unable to load modules for /usr/local/samba/private/sam.ldb:
>> schema_load_init: dsdb_get_schema failed
>> [2016/03/02 11:27:17.763923,  0]
>> ../source4/smbd/service_task.c:35(task_server_terminate)
>>    task_server_terminate: [Failed to startup ldap server task]
>> [2016/03/02 11:27:18.247740,  0]
>> ../file_server/file_server.c:46(file_server_smbd_done)
>>    file_server smbd daemon died with exit status 1
>> [2016/03/02 11:27:18.247875,  0]
>> ../source4/smbd/service_task.c:35(task_server_terminate)
>>    task_server_terminate: [smbd child process exited]
>> [2016/03/02 11:27:23.987725,  0]
>> ../source4/winbind/winbindd.c:49(winbindd_done)
>>    winbindd daemon exited normally
>> [2016/03/02 11:27:23.989217,  0]
>> ../source4/smbd/service_task.c:35(task_server_terminate)
>>    task_server_terminate: [winbindd child process exited]
>>
>> Relevant info:
>>
>> itwerks at filer:~$ samba -V
>> Version 4.3.3
>> itwerks at filer:~$ lsb_release -a
>> No LSB modules are available.
>> Distributor ID: Ubuntu
>> Description:    Ubuntu 15.10
>> Release:        15.10
>> Codename:       wily
>> itwerks at filer:~$ uname -a
>> Linux filer 4.2.0-30-generic #36-Ubuntu SMP Fri Feb 26 00:58:07 UTC 2016
>> x86_64 x86_64 x86_64 GNU/Linux
>> itwerks at filer:~$
>>
>> Please advise.  It is end of year for this company, I simply cannot afford
>> failures like this every three to four weeks, and I cannot blow the entire
>> domain out again and rebuild from scratch.
>>
>> Frustrated.
>>
>> JS
>>

Have you considered that it may be a hardware problem ? or is your raid 
setup ok . You could also try turning Apparmor off as a test (Is there 
anything in syslog about Samba and Apparmor).

Your problem is not a normal one, or one that is common, it may help if 
we could see your smb.conf (you may have posted it before, but if so, 
please post it again)