[Samba] Samba AD/DC crashed again, third time in as many months

IT Admin it at cliffbells.com
Wed Mar 2 21:42:11 UTC 2016


I built this machine, and while it isn't the most robust box in the world
it has been stable otherwise.  The RAID array is configured RAID1, I can't
see how that could cause corruption issues and I haven't experienced any
other data corruption issues apart from SAMBA collapsing.  I did start
seeing samba failures about two weeks ago, restarting the machine brought
the domain up in those cases, associated errors revolved around samba dns
functions iirc.  I have intended to upgrade to the latest release (which
went public a week after the last rebuild), I had hoped that update would
eradicate the errors I was experiencing.

My smb.conf:

cat /etc/samba/smb.conf
# Global parameters
[global]
        workgroup = CB
        realm = CB.CLIFFBELLS.COM
        netbios name = FILER
        server role = active directory domain controller
        dns forwarder = 192.168.37.254
        allow dns updates = nonsecure and secure
        idmap_ldb:use rfc2307 = yes
        printing = CUPS
        printcap name = /dev/null
        # below added from dragon.org guide
        idmap config *:backend = tdb
        # This local range must not overlap the BLACK:range above
        idmap config *:range = 5000-9999
        idmap config CB:backend = ad
        idmap config CB:schema_mode = rfc2307
        idmap config CB:range = 10000-29999
         # Use home directory and shell information from AD
        winbind nss info = rfc2307

[netlogon]
        path = /usr/local/samba/var/locks/sysvol/cb.cliffbells.com/scripts
        read only = No

[sysvol]
        path = /usr/local/samba/var/locks/sysvol
        read only = No
[Users]
        directory_mode: parameter = 0700
        read only = no
        path = /storage/Users
        csc policy = documents

The above smb.conf has been in use since late January, and the following I
cannot explain... There should be declarations for "data" and "backups" in
that file but they have magically vanished.  I worked at this location last
weekend and know for a fact that those shares were live and accessible.

It appears an update mangled the smb.conf file on the 17th of february:

ls -la /etc/samba/
total 68
drwxr-xr-x   3 root root  4096 Mar  2 16:34 .
drwxr-xr-x 135 root root 12288 Mar  2 06:51 ..
-rw-r--r--   1 root root     8 Nov 12 13:07 gdbcommands
-rw-r--r--   1 root root   972 Feb 17 06:34 smb.conf
-rw-r--r--   1 root root   998 Dec 13 19:44 smb.conf.BAD
-rw-r--r--   1 root root   345 Jan 10 04:36 smb.conf.BADDY
-rw-r--r--   1 root root  1529 Jan  6 06:54 smb.conf.ONE.CLIFFBELLS.COM
-rw-r--r--   1 root root  9542 Dec  7 18:43 smb.conf.ORIG
-rw-r--r--   1 root root   345 Jan  8 21:24 smb.conf.SRC
-rw-r--r--   1 root root  9535 Feb 17 06:34 smb.conf.ucf-dist
drwxr-xr-x   2 root root  4096 Nov 12 13:06 tls

I'm going to restore it to an earlier version and reboot.

JS
On Mar 2, 2016 4:01 PM, "Rowland penny" <rpenny at samba.org> wrote:

> On 02/03/16 20:46, IT Admin wrote:
>
>> Any further information required?  Any troubleshooting steps?  I need to
>> get this domain functional ASAP.
>>
>> Please advise.
>>
>> JS
>> On Mar 2, 2016 11:39 AM, "IT Admin" <it at cliffbells.com> wrote:
>>
>> Hello to the samba users list again.  I've got an emergency here, a Samba
>>> AD DC I've got deployed has crashed again, this is the third time since
>>> 12/17/2015 that this domain has failed completely.
>>>
>>> No power outages or unexpected shutdowns, samba simply fails to start, it
>>> appears my database is yet again corrupt:
>>>
>>> [2016/03/02 11:27:17.025235,  0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>    ldb: dsdb_get_schema: refresh_fn() failed
>>> [2016/03/02 11:27:17.025346,  0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>    ldb: schema_load_init: dsdb_get_schema failed
>>> [2016/03/02 11:27:17.025377,  0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>    ldb: module schema_load initialization failed : Operations error
>>> [2016/03/02 11:27:17.025404,  0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>    ldb: module rootdse initialization failed : Operations error
>>> [2016/03/02 11:27:17.025431,  0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>    ldb: module samba_dsdb initialization failed : Operations error
>>> [2016/03/02 11:27:17.025460,  0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>    ldb: Unable to load modules for /usr/local/samba/private/sam.ldb:
>>> schema_load_init: dsdb_get_schema failed
>>> [2016/03/02 11:27:17.032681,  0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>    ldb: dsdb_get_schema: refresh_fn() failed
>>> [2016/03/02 11:27:17.033626,  0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>> [2016/03/02 11:27:17.033776,  0]
>>> ../source4/smbd/service_task.c:35(task_server_terminate)
>>>    task_server_terminate: [nbtd failed to open samdb]
>>>    ldb: schema_load_init: dsdb_get_schema failed
>>> [2016/03/02 11:27:17.034676,  0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>    ldb: module schema_load initialization failed : Operations error
>>> [2016/03/02 11:27:17.035224,  0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>    ldb: module rootdse initialization failed : Operations error
>>> [2016/03/02 11:27:17.035838,  0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>    ldb: module samba_dsdb initialization failed : Operations error
>>> [2016/03/02 11:27:17.036392,  0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>    ldb: Unable to load modules for /usr/local/samba/private/sam.ldb:
>>> schema_load_init: dsdb_get_schema failed
>>> [2016/03/02 11:27:17.038675,  0]
>>> ../source4/smbd/service_task.c:35(task_server_terminate)
>>> [2016/03/02 11:27:17.040353,  0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>> [2016/03/02 11:27:17.042764,  0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>    ldb: dsdb_get_schema: refresh_fn() failed
>>> [2016/03/02 11:27:17.042857,  0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>    ldb: schema_load_init: dsdb_get_schema failed
>>> [2016/03/02 11:27:17.042887,  0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>    ldb: module schema_load initialization failed : Operations error
>>> [2016/03/02 11:27:17.042917,  0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>    ldb: module rootdse initialization failed : Operations error
>>> [2016/03/02 11:27:17.042942,  0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>    ldb: module samba_dsdb initialization failed : Operations error
>>> [2016/03/02 11:27:17.042971,  0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>    ldb: Unable to load modules for /usr/local/samba/private/sam.ldb:
>>> schema_load_init: dsdb_get_schema failed
>>>    ldb: dsdb_get_schema: refresh_fn() failed
>>> [2016/03/02 11:27:17.045981,  0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>    ldb: schema_load_init: dsdb_get_schema failed
>>> [2016/03/02 11:27:17.046598,  0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>    ldb: module schema_load initialization failed : Operations error
>>> [2016/03/02 11:27:17.047302,  0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>> [2016/03/02 11:27:17.047523,  0]
>>> ../source4/smbd/service_task.c:35(task_server_terminate)
>>>    task_server_terminate: [dreplsrv: Failed to connect to local samdb:
>>> WERR_DS_UNAVAILABLE
>>>    ]
>>>    ldb: module rootdse initialization failed : Operations error
>>> [2016/03/02 11:27:17.048503,  0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>    ldb: module samba_dsdb initialization failed : Operations error
>>> [2016/03/02 11:27:17.048540,  0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>    ldb: Unable to load modules for /usr/local/samba/private/sam.ldb:
>>> schema_load_init: dsdb_get_schema failed
>>> [2016/03/02 11:27:17.050402,  0]
>>> ../source4/smbd/service_task.c:35(task_server_terminate)
>>>    task_server_terminate: [kdc: krb5_init_context samdb connect failed]
>>>    task_server_terminate: [cldapd failed to open samdb]
>>> [2016/03/02 11:27:17.072670,  0]
>>> ../source4/smbd/server.c:211(samba_terminate)
>>>    samba_terminate: dreplsrv: Failed to connect to local samdb:
>>> WERR_DS_UNAVAILABLE
>>>
>>> [2016/03/02 11:27:17.073896,  0]
>>> ../source4/smbd/server.c:211(samba_terminate)
>>>    samba_terminate: nbtd failed to open samdb
>>> [2016/03/02 11:27:17.075031,  0]
>>> ../source4/smbd/server.c:211(samba_terminate)
>>>    samba_terminate: cldapd failed to open samdb
>>> [2016/03/02 11:27:17.075614,  0]
>>> ../source4/smbd/server.c:211(samba_terminate)
>>>    samba_terminate: kdc: krb5_init_context samdb connect failed
>>> [2016/03/02 11:27:17.086051,  0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>    ldb: dsdb_get_schema: refresh_fn() failed
>>> [2016/03/02 11:27:17.091786,  0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>    ldb: dsdb_get_schema: refresh_fn() failed
>>> [2016/03/02 11:27:17.091860,  0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>    ldb: schema_load_init: dsdb_get_schema failed
>>> [2016/03/02 11:27:17.091890,  0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>    ldb: module schema_load initialization failed : Operations error
>>> [2016/03/02 11:27:17.091919,  0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>    ldb: module rootdse initialization failed : Operations error
>>> [2016/03/02 11:27:17.091946,  0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>    ldb: module samba_dsdb initialization failed : Operations error
>>> [2016/03/02 11:27:17.091974,  0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>    ldb: Unable to load modules for /usr/local/samba/private/sam.ldb:
>>> schema_load_init: dsdb_get_schema failed
>>> [2016/03/02 11:27:17.093919,  0]
>>> ../source4/smbd/service_task.c:35(task_server_terminate)
>>>    task_server_terminate: [dns: samdb_connect failed]
>>> [2016/03/02 11:27:17.095175,  0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>    ldb: schema_load_init: dsdb_get_schema failed
>>> [2016/03/02 11:27:17.095232,  0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>    ldb: module schema_load initialization failed : Operations error
>>> [2016/03/02 11:27:17.095261,  0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>    ldb: module rootdse initialization failed : Operations error
>>> [2016/03/02 11:27:17.095288,  0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>    ldb: module samba_dsdb initialization failed : Operations error
>>> [2016/03/02 11:27:17.095317,  0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>    ldb: Unable to load modules for /usr/local/samba/private/sam.ldb:
>>> schema_load_init: dsdb_get_schema failed
>>> [2016/03/02 11:27:17.095940,  0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>    ldb: dsdb_get_schema: refresh_fn() failed
>>> [2016/03/02 11:27:17.097255,  0]
>>> ../source4/smbd/service_task.c:35(task_server_terminate)
>>> [2016/03/02 11:27:17.098404,  0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>    task_server_terminate: [kccsrv: Failed to connect to local samdb:
>>> WERR_DS_UNAVAILABLE
>>>    ]
>>> [2016/03/02 11:27:17.102052,  0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>    ldb: schema_load_init: dsdb_get_schema failed
>>> [2016/03/02 11:27:17.102608,  0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>    ldb: module schema_load initialization failed : Operations error
>>> [2016/03/02 11:27:17.103164,  0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>    ldb: module rootdse initialization failed : Operations error
>>> [2016/03/02 11:27:17.103210,  0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>    ldb: module samba_dsdb initialization failed : Operations error
>>> [2016/03/02 11:27:17.103241,  0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>    ldb: Unable to load modules for /usr/local/samba/private/sam.ldb:
>>> schema_load_init: dsdb_get_schema failed
>>>    ldb: dsdb_get_schema: refresh_fn() failed
>>> [2016/03/02 11:27:17.103939,  0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>    ldb: schema_load_init: dsdb_get_schema failed
>>> [2016/03/02 11:27:17.103974,  0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>    ldb: module schema_load initialization failed : Operations error
>>> [2016/03/02 11:27:17.104003,  0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>    ldb: module rootdse initialization failed : Operations error
>>> [2016/03/02 11:27:17.107759,  0]
>>> ../source4/smbd/service_task.c:35(task_server_terminate)
>>>    task_server_terminate: [dnsupdate: Failed to connect to local samdb
>>>    ]
>>> [2016/03/02 11:27:17.109612,  0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>    ldb: module samba_dsdb initialization failed : Operations error
>>> [2016/03/02 11:27:17.109672,  0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>    ldb: Unable to load modules for /usr/local/samba/private/sam.ldb:
>>> schema_load_init: dsdb_get_schema failed
>>> [2016/03/02 11:27:17.114754,  0]
>>> ../source4/smbd/service_task.c:35(task_server_terminate)
>>>    task_server_terminate: [ntp_signd failed to open samdb]
>>> [2016/03/02 11:27:17.731614,  0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>    ldb: dsdb_get_schema: refresh_fn() failed
>>> [2016/03/02 11:27:17.731721,  0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>    ldb: schema_load_init: dsdb_get_schema failed
>>> [2016/03/02 11:27:17.731750,  0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>    ldb: module schema_load initialization failed : Operations error
>>> [2016/03/02 11:27:17.731779,  0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>    ldb: module rootdse initialization failed : Operations error
>>> [2016/03/02 11:27:17.731807,  0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>    ldb: module samba_dsdb initialization failed : Operations error
>>> [2016/03/02 11:27:17.731835,  0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>    ldb: Unable to load modules for /usr/local/samba/private/sam.ldb:
>>> schema_load_init: dsdb_get_schema failed
>>> [2016/03/02 11:27:17.762162,  0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>    ldb: dsdb_get_schema: refresh_fn() failed
>>> [2016/03/02 11:27:17.762247,  0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>    ldb: schema_load_init: dsdb_get_schema failed
>>> [2016/03/02 11:27:17.762276,  0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>    ldb: module schema_load initialization failed : Operations error
>>> [2016/03/02 11:27:17.762305,  0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>    ldb: module rootdse initialization failed : Operations error
>>> [2016/03/02 11:27:17.762332,  0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>    ldb: module samba_dsdb initialization failed : Operations error
>>> [2016/03/02 11:27:17.762361,  0]
>>> ../lib/ldb-samba/ldb_wrap.c:72(ldb_wrap_debug)
>>>    ldb: Unable to load modules for /usr/local/samba/private/sam.ldb:
>>> schema_load_init: dsdb_get_schema failed
>>> [2016/03/02 11:27:17.763923,  0]
>>> ../source4/smbd/service_task.c:35(task_server_terminate)
>>>    task_server_terminate: [Failed to startup ldap server task]
>>> [2016/03/02 11:27:18.247740,  0]
>>> ../file_server/file_server.c:46(file_server_smbd_done)
>>>    file_server smbd daemon died with exit status 1
>>> [2016/03/02 11:27:18.247875,  0]
>>> ../source4/smbd/service_task.c:35(task_server_terminate)
>>>    task_server_terminate: [smbd child process exited]
>>> [2016/03/02 11:27:23.987725,  0]
>>> ../source4/winbind/winbindd.c:49(winbindd_done)
>>>    winbindd daemon exited normally
>>> [2016/03/02 11:27:23.989217,  0]
>>> ../source4/smbd/service_task.c:35(task_server_terminate)
>>>    task_server_terminate: [winbindd child process exited]
>>>
>>> Relevant info:
>>>
>>> itwerks at filer:~$ samba -V
>>> Version 4.3.3
>>> itwerks at filer:~$ lsb_release -a
>>> No LSB modules are available.
>>> Distributor ID: Ubuntu
>>> Description:    Ubuntu 15.10
>>> Release:        15.10
>>> Codename:       wily
>>> itwerks at filer:~$ uname -a
>>> Linux filer 4.2.0-30-generic #36-Ubuntu SMP Fri Feb 26 00:58:07 UTC 2016
>>> x86_64 x86_64 x86_64 GNU/Linux
>>> itwerks at filer:~$
>>>
>>> Please advise.  It is end of year for this company, I simply cannot
>>> afford
>>> failures like this every three to four weeks, and I cannot blow the
>>> entire
>>> domain out again and rebuild from scratch.
>>>
>>> Frustrated.
>>>
>>> JS
>>>
>>>
> Have you considered that it may be a hardware problem ? or is your raid
> setup ok . You could also try turning Apparmor off as a test (Is there
> anything in syslog about Samba and Apparmor).
>
> Your problem is not a normal one, or one that is common, it may help if we
> could see your smb.conf (you may have posted it before, but if so, please
> post it again)
>
> --
> To unsubscribe from this list go to the following URL and read the
> instructions:  https://lists.samba.org/mailman/options/samba
>


More information about the samba mailing list