[Samba] Strange issue with Samba+CTDB+SELinux+GlusterFS
Thomas Cameron
thomas.cameron at camerontech.com
Wed Nov 16 03:04:18 UTC 2022
I'm wondering if something weird is happening like it creates the file
initially as /var/lib/ctdb/persistent/registry.tdb and then renames it
to /var/lib/ctdb/persistent/registry.tdb.1. The SELinux error could be
on the initial file it's creating or something like that.
And you say that, when you set SELinux to permissive, the problem goes
away completely, right?
Can you maybe run the server in permissive mode, then run through all of
the paces, THEN run audit2allow and see if it throws any errors?
I'm just brainstorming here. This is a weird problem. I am kinda
surprised that it worked for a while and then failed. Again, I wonder if
it's creating a file and then renaming it. What's the context of the
parent directory (ls -Z)?
Maybe you could do something like:
semanage fcontext -a -t ctdbd_var_lib_t
/var/lib/ctdb/persistent/account_policy.tdb
or even:
semanage fcontext -a -t ctdbd_var_lib_t /var/lib/ctdb/persistent(/.*)?
That would make any file created under /var/lib/ctdb/persistent/ labeled
as ctdbd_var_lib_t.
Thomas
On 11/15/22 15:47, Leszek Szczepanowski via samba wrote:
> Additionally:
>
> [root at fs01 symptoms]# ctdb getdbmap
> Number of databases:19
> dbid:0x4d2a432b name:g_lock.tdb path:/var/lib/ctdb/volatile/g_lock.tdb.0
> dbid:0x2d608c16 name:netlogon_creds_cli.tdb
> path:/var/lib/ctdb/volatile/netlogon_creds_cli.tdb.0
> dbid:0x521b7544 name:smbXsrv_version_global.tdb
> path:/var/lib/ctdb/volatile/smbXsrv_version_global.tdb.0
> dbid:0x477d2e20 name:smbXsrv_client_global.tdb
> path:/var/lib/ctdb/volatile/smbXsrv_client_global.tdb.0
> dbid:0x6b06a26d name:smbXsrv_session_global.tdb
> path:/var/lib/ctdb/volatile/smbXsrv_session_global.tdb.0
> dbid:0x68c12c2c name:smbXsrv_tcon_global.tdb
> path:/var/lib/ctdb/volatile/smbXsrv_tcon_global.tdb.0
> dbid:0x4e66c2b2 name:brlock.tdb path:/var/lib/ctdb/volatile/brlock.tdb.0
> dbid:0x7a19d84d name:locking.tdb path:/var/lib/ctdb/volatile/locking.tdb.0
> dbid:0x06916e77 name:leases.tdb path:/var/lib/ctdb/volatile/leases.tdb.0
> dbid:0x66f71b8c name:smbXsrv_open_global.tdb
> path:/var/lib/ctdb/volatile/smbXsrv_open_global.tdb.0
> dbid:0x1313cc83 name:autorid.tdb
> path:/var/lib/ctdb/persistent/autorid.tdb.0 PERSISTENT
> dbid:0x5bcfcbd7 name:printer_list.tdb
> path:/var/lib/ctdb/persistent/printer_list.tdb.0 PERSISTENT
> dbid:0x3ef19640 name:passdb.tdb path:/var/lib/ctdb/persistent/passdb.tdb.0
> PERSISTENT
> dbid:0x2ca251cf name:account_policy.tdb
> path:/var/lib/ctdb/persistent/account_policy.tdb.0 PERSISTENT
> dbid:0xa1413774 name:group_mapping.tdb
> path:/var/lib/ctdb/persistent/group_mapping.tdb.0 PERSISTENT
> dbid:0xc3078fba name:share_info.tdb
> path:/var/lib/ctdb/persistent/share_info.tdb.0 PERSISTENT
> dbid:0x6645c6c4 name:ctdb.tdb path:/var/lib/ctdb/persistent/ctdb.tdb.0
> PERSISTENT
> dbid:0x7132c184 name:secrets.tdb
> path:/var/lib/ctdb/persistent/secrets.tdb.0 PERSISTENT
> dbid:0x6cf2837d name:registry.tdb
> path:/var/lib/ctdb/persistent/registry.tdb.0 PERSISTENT
>
> It seems, it uses suffix of node number on each node, here node 3:
>
> [root at fs03 lszczepa]# ctdb getdbmap
> Number of databases:19
> dbid:0x66f71b8c name:smbXsrv_open_global.tdb
> path:/var/lib/ctdb/volatile/smbXsrv_open_global.tdb.2
> dbid:0x06916e77 name:leases.tdb path:/var/lib/ctdb/volatile/leases.tdb.2
> dbid:0x7a19d84d name:locking.tdb path:/var/lib/ctdb/volatile/locking.tdb.2
> dbid:0x4e66c2b2 name:brlock.tdb path:/var/lib/ctdb/volatile/brlock.tdb.2
> dbid:0x68c12c2c name:smbXsrv_tcon_global.tdb
> path:/var/lib/ctdb/volatile/smbXsrv_tcon_global.tdb.2
> dbid:0x6b06a26d name:smbXsrv_session_global.tdb
> path:/var/lib/ctdb/volatile/smbXsrv_session_global.tdb.2
> dbid:0x477d2e20 name:smbXsrv_client_global.tdb
> path:/var/lib/ctdb/volatile/smbXsrv_client_global.tdb.2
> dbid:0x521b7544 name:smbXsrv_version_global.tdb
> path:/var/lib/ctdb/volatile/smbXsrv_version_global.tdb.2
> dbid:0x2d608c16 name:netlogon_creds_cli.tdb
> path:/var/lib/ctdb/volatile/netlogon_creds_cli.tdb.2
> dbid:0x4d2a432b name:g_lock.tdb path:/var/lib/ctdb/volatile/g_lock.tdb.2
> dbid:0x1313cc83 name:autorid.tdb
> path:/var/lib/ctdb/persistent/autorid.tdb.2 PERSISTENT
> dbid:0x5bcfcbd7 name:printer_list.tdb
> path:/var/lib/ctdb/persistent/printer_list.tdb.2 PERSISTENT
> dbid:0x3ef19640 name:passdb.tdb path:/var/lib/ctdb/persistent/passdb.tdb.2
> PERSISTENT
> dbid:0x2ca251cf name:account_policy.tdb
> path:/var/lib/ctdb/persistent/account_policy.tdb.2 PERSISTENT
> dbid:0xa1413774 name:group_mapping.tdb
> path:/var/lib/ctdb/persistent/group_mapping.tdb.2 PERSISTENT
> dbid:0xc3078fba name:share_info.tdb
> path:/var/lib/ctdb/persistent/share_info.tdb.2 PERSISTENT
> dbid:0x6645c6c4 name:ctdb.tdb path:/var/lib/ctdb/persistent/ctdb.tdb.2
> PERSISTENT
> dbid:0x7132c184 name:secrets.tdb
> path:/var/lib/ctdb/persistent/secrets.tdb.2 PERSISTENT
> dbid:0x6cf2837d name:registry.tdb
> path:/var/lib/ctdb/persistent/registry.tdb.2 PERSISTENT
>
>
>
> wt., 15 lis 2022 o 22:44 Leszek Szczepanowski <twinsen at mspanc.net>
> napisał(a):
>
>> Hi,
>>
>> [root at fs01 symptoms]# ls -lZ /var/lib/ctdb/persistent/registry.tdb
>> ls: cannot access '/var/lib/ctdb/persistent/registry.tdb': No such file or
>> directory
>> [root at fs01 symptoms]# find / -name registry.tdb
>> [root at fs01 symptoms]#
>>
>> [root at fs01 symptoms]# ls -lZ /var/lib/ctdb/persistent/
>> total 20832
>> -rw-------. 1 root root system_u:object_r:ctdbd_var_lib_t:s0 7892992 Nov
>> 15 18:50 account_policy.tdb.0
>> -rw-r--r--. 1 root root system_u:object_r:ctdbd_var_lib_t:s0 1327104 Nov
>> 15 18:50 autorid.tdb.0
>> -rw-------. 1 root root system_u:object_r:ctdbd_var_lib_t:s0 1310720 Nov
>> 15 18:50 ctdb.tdb.0
>> -rw-------. 1 root root system_u:object_r:ctdbd_var_lib_t:s0 1310720 Nov
>> 15 18:50 group_mapping.tdb.0
>> -rw-------. 1 root root system_u:object_r:ctdbd_var_lib_t:s0 2560000 Nov
>> 15 18:50 passdb.tdb.0
>> -rw-r--r--. 1 root root system_u:object_r:ctdbd_var_lib_t:s0 1310720 Nov
>> 15 18:50 printer_list.tdb.0
>> -rw-------. 1 root root system_u:object_r:ctdbd_var_lib_t:s0 1736704 Nov
>> 15 18:50 registry.tdb.0
>> -rw-------. 1 root root system_u:object_r:ctdbd_var_lib_t:s0 2146304 Nov
>> 15 18:50 secrets.tdb.0
>> -rw-------. 1 root root system_u:object_r:ctdbd_var_lib_t:s0 1736704 Nov
>> 15 18:50 share_info.tdb.0
>>
>> [root at fs01 symptoms]# ls -lZ /var/lib/ctdb/persistent/registry.tdb.0
>> -rw-------. 1 root root system_u:object_r:ctdbd_var_lib_t:s0 1736704 Nov
>> 15 18:50 /var/lib/ctdb/persistent/registry.tdb.0
>>
>> That is strange. Why .0?
>>
>> wt., 15 lis 2022 o 21:28 Thomas Cameron <thomas.cameron at camerontech.com>
>> napisał(a):
>>
>>> What's the label for /var/lib/ctdb/persistent/registry.tdb.1? What does
>>> ls -lZ tell you?
>>>
>>> Thomas
>>>
>>> On 11/15/22 10:36, Leszek Szczepanowski wrote:
>>>
>>> I'm getting this:
>>>
>>> type=AVC msg=audit(1668528098.389:291): avc: denied { getattr } for
>>> pid=84190 comm="samba-dcerpcd"
>>> path="/var/lib/ctdb/persistent/registry.tdb.1" dev="dm-0" ino=117620565
>>> scontext=system_u:system_r:winbind_rpcd_t:s0
>>> tcontext=system_u:object_r:ctdbd_var_lib_t:s0 tclass=file permissive=1
>>> type=AVC msg=audit(1668528098.389:292): avc: denied { map } for
>>> pid=84190 comm="samba-dcerpcd"
>>> path="/var/lib/ctdb/persistent/registry.tdb.1" dev="dm-0" ino=117620565
>>> scontext=system_u:system_r:winbind_rpcd_t:s0
>>> tcontext=system_u:object_r:ctdbd_var_lib_t:s0 tclass=file permissive=1
>>> type=AVC msg=audit(1668528098.391:293): avc: denied { setattr } for
>>> pid=84190 comm="samba-dcerpcd" name="g_lock.tdb.1" dev="dm-0"
>>> ino=152097603 scontext=system_u:system_r:winbind_rpcd_t:s0
>>> tcontext=system_u:object_r:ctdbd_var_lib_t:s0 tclass=file permissive=1
>>> type=AVC msg=audit(1668529035.873:308): avc: denied { read write } for
>>> pid=89129 comm="samba-dcerpcd" name="registry.tdb.1" dev="dm-0"
>>> ino=117620565 scontext=system_u:system_r:winbind_rpcd_t:s0
>>> tcontext=system_u:object_r:ctdbd_var_lib_t:s0 tclass=file permissive=1
>>> type=AVC msg=audit(1668529035.873:308): avc: denied { open } for
>>> pid=89129 comm="samba-dcerpcd"
>>> path="/var/lib/ctdb/persistent/registry.tdb.1" dev="dm-0" ino=117620565
>>> scontext=system_u:system_r:winbind_rpcd_t:s0
>>> tcontext=system_u:object_r:ctdbd_var_lib_t:s0 tclass=file permissive=1
>>> type=AVC msg=audit(1668529035.873:309): avc: denied { lock } for
>>> pid=89129 comm="samba-dcerpcd"
>>> path="/var/lib/ctdb/persistent/registry.tdb.1" dev="dm-0" ino=117620565
>>> scontext=system_u:system_r:winbind_rpcd_t:s0
>>> tcontext=system_u:object_r:ctdbd_var_lib_t:s0 tclass=file permissive=1
>>> type=AVC msg=audit(1668529035.873:310): avc: denied { getattr } for
>>> pid=89129 comm="samba-dcerpcd"
>>> path="/var/lib/ctdb/persistent/registry.tdb.1" dev="dm-0" ino=117620565
>>> scontext=system_u:system_r:winbind_rpcd_t:s0
>>> tcontext=system_u:object_r:ctdbd_var_lib_t:s0 tclass=file permissive=1
>>> type=AVC msg=audit(1668529035.875:311): avc: denied { setattr } for
>>> pid=89129 comm="samba-dcerpcd" name="g_lock.tdb.1" dev="dm-0"
>>> ino=152097603 scontext=system_u:system_r:winbind_rpcd_t:s0
>>> tcontext=system_u:object_r:ctdbd_var_lib_t:s0 tclass=file permissive=1
>>>
>>> I did
>>> audit2allow -al -M dcerpcd
>>> semodule -i dcerpcd.pp
>>>
>>> It was working in Enforcing 1 mode for like 1 minute. After that, again
>>> not working. But this time:
>>>
>>> [root at fs02 samba]# audit2allow -al
>>> [root at fs02 samba]#
>>>
>>> So the module is active, nothing is denied (no new entries in
>>> /var/log/audit/audit.log), however it's again:
>>>
>>> [2022/11/15 17:33:13, 0]
>>> ../../source3/lib/dbwrap/dbwrap_ctdb.c:1926(db_open_ctdb)
>>> Could not open tdb /var/lib/ctdb/persistent/registry.tdb.1: Permission
>>> denied
>>> [2022/11/15 17:33:13, 0]
>>> ../../source3/lib/dbwrap/dbwrap_open.c:169(db_open)
>>> db_open: failed to attach to ctdb registry.tdb
>>> [2022/11/15 17:33:13, 0]
>>> ../../source3/lib/dbwrap/dbwrap_ctdb.c:1926(db_open_ctdb)
>>> Could not open tdb /var/lib/ctdb/persistent/registry.tdb.1: Permission
>>> denied
>>> [2022/11/15 17:33:13, 0]
>>> ../../source3/lib/dbwrap/dbwrap_open.c:169(db_open)
>>> db_open: failed to attach to ctdb registry.tdb
>>> [2022/11/15 17:33:13, 1]
>>> ../../source3/registry/reg_backend_db.c:759(regdb_init)
>>> regdb_init: Failed to open registry /var/lib/samba/registry.tdb
>>> (Permission denied)
>>> [2022/11/15 17:33:13, 0]
>>> ../../source3/registry/reg_init_basic.c:35(registry_init_common)
>>> Failed to initialize the registry: WERR_ACCESS_DENIED
>>> [2022/11/15 17:33:13, 1]
>>> ../../source3/param/loadparm.c:2157(lp_smbconf_ctx)
>>> error initializing registry configuration: SBC_ERR_BADFILE
>>> Can't load /etc/samba/smb.conf - run testparm to debug it
>>> samba-dcerpcd - Failed to load config file!
>>>
>>>
>>>
>>>
>>> wt., 15 lis 2022 o 16:09 Thomas Cameron via samba <samba at lists.samba.org>
>>> napisał(a):
>>>
>>>> As root, what does audit2allow -al tell you?
>>>>
>>>> Here's a video I did when I was at Red Hat, talking through SELinux. I
>>>> hope it's helpful. https://www.youtube.com/watch?v=_WOKRaM-HI4
>>>>
>>>> Thomas
>>>>
>>>> On 11/15/22 04:04, Leszek Szczepanowski via samba wrote:
>>>>> I think with security=user the rest is simply ignored, and the local
>>>> auth
>>>>> is working fine.
>>>>> I will comment out that option for now. The AD integration will be done
>>>>> later.
>>>>> The main problem is probably not related directly to CTDB, but to what
>>>>> Samba is trying to access with SELinux in Enforcing mode.
>>>>> As there are no errors in /var/log/messages or in /var/log/audit, I'm
>>>> lost.
>>>>> I forgot to say versions, so:
>>>>>
>>>>> [root at fs01 samba]# cat /etc/redhat-release
>>>>> CentOS Stream release 9
>>>>> [root at fs01 samba]# rpm -qa | grep samba
>>>>> samba-common-4.16.4-101.el9.noarch
>>>>> samba-client-libs-4.16.4-101.el9.x86_64
>>>>> samba-common-libs-4.16.4-101.el9.x86_64
>>>>> samba-libs-4.16.4-101.el9.x86_64
>>>>> python3-samba-4.16.4-101.el9.x86_64
>>>>> samba-common-tools-4.16.4-101.el9.x86_64
>>>>> samba-4.16.4-101.el9.x86_64
>>>>> samba-client-4.16.4-101.el9.x86_64
>>>>> samba-winbind-modules-4.16.4-101.el9.x86_64
>>>>> samba-winbind-4.16.4-101.el9.x86_64
>>>>> samba-winbind-krb5-locator-4.16.4-101.el9.x86_64
>>>>> samba-winbind-clients-4.16.4-101.el9.x86_64
>>>>> [root at fs01 samba]# rpm -qa | grep ctdb
>>>>> ctdb-4.16.4-101.el9.x86_64
>>>>> [root at fs01 samba]# uname -a
>>>>> Linux fs01.xxx 5.14.0-183.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Mon Oct 31
>>>>> 09:18:51 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
>>>>>
>>>>> Also, the provided errors were wrong, I was playing with permissive
>>>> mode.
>>>>> In enforcing it is:
>>>>>
>>>>> [2022/11/15 11:02:08, 0]
>>>>> ../../source3/lib/dbwrap/dbwrap_ctdb.c:1926(db_open_ctdb)
>>>>> Could not open tdb /var/lib/ctdb/persistent/registry.tdb.0:
>>>> Permission
>>>>> denied
>>>>> [2022/11/15 11:02:08, 0]
>>>>> ../../source3/lib/dbwrap/dbwrap_open.c:169(db_open)
>>>>> db_open: failed to attach to ctdb registry.tdb
>>>>> [2022/11/15 11:02:08, 0]
>>>>> ../../source3/lib/dbwrap/dbwrap_ctdb.c:1926(db_open_ctdb)
>>>>> Could not open tdb /var/lib/ctdb/persistent/registry.tdb.0:
>>>> Permission
>>>>> denied
>>>>> [2022/11/15 11:02:08, 0]
>>>>> ../../source3/lib/dbwrap/dbwrap_open.c:169(db_open)
>>>>> db_open: failed to attach to ctdb registry.tdb
>>>>> [2022/11/15 11:02:08, 1]
>>>>> ../../source3/registry/reg_backend_db.c:759(regdb_init)
>>>>> regdb_init: Failed to open registry /var/lib/samba/registry.tdb
>>>>> (Permission denied)
>>>>> [2022/11/15 11:02:08, 0]
>>>>> ../../source3/registry/reg_init_basic.c:35(registry_init_common)
>>>>> Failed to initialize the registry: WERR_ACCESS_DENIED
>>>>> [2022/11/15 11:02:08, 1]
>>>>> ../../source3/param/loadparm.c:2157(lp_smbconf_ctx)
>>>>> error initializing registry configuration: SBC_ERR_BADFILE
>>>>> Can't load /etc/samba/smb.conf - run testparm to debug it
>>>>> samba-dcerpcd - Failed to load config file!
>>>>>
>>>>> But in the same time, I can do testparm without any issues:
>>>>>
>>>>> [root at fs01 samba]# testparm
>>>>> Load smb config files from /etc/samba/smb.conf
>>>>> Loaded services file OK.
>>>>> Weak crypto is allowed
>>>>>
>>>>> Server role: ROLE_STANDALONE
>>>>>
>>>>> Press enter to see a dump of your service definitions
>>>>>
>>>>> # Global parameters
>>>>> [global]
>>>>> clustering = Yes
>>>>> logging = syslog
>>>>> netbios name = FS
>>>>> realm = FS.xxx
>>>>> registry shares = Yes
>>>>> security = USER
>>>>> workgroup = xxx
>>>>> idmap config * : range = 1000000-1999999
>>>>> ctdb:registry.tdb = yes
>>>>> idmap config * : backend = autorid
>>>>>
>>>>>
>>>>> [symptoms]
>>>>> path = /mnt/glusterfs/symptoms/
>>>>> read only = No
>>>>>
>>>>>
>>>>> wt., 15 lis 2022 o 10:47 Rowland Penny via samba <
>>>> samba at lists.samba.org>
>>>>> napisał(a):
>>>>>
>>>>>> On 15/11/2022 09:21, Leszek Szczepanowski via samba wrote:
>>>>>>> I have very simple config for HA Samba, using CTDB.
>>>>>>> I have set all possible SELinux options until "denied" messages
>>>> stopped
>>>>>>> appearch in /var/log/messages.
>>>>>>>
>>>>>>> All works flawlessly, just the problem is with browsing Samba shares
>>>> with
>>>>>>> enforcing setting.
>>>>>>>
>>>>>>> When I try to browse shares, I'm getting this:
>>>>>>>
>>>>>>> samba-dcerpcd version 4.16.4 started.
>>>>>>> Copyright Andrew Tridgell and the Samba Team 1992-2022
>>>>>>> [2022/11/15 10:10:57.674555, 1]
>>>>>>> ../../source3/rpc_client/cli_pipe.c:3014(rpc_pipe_open_ncalrpc)
>>>>>>> rpc_pipe_open_ncalrpc: connect(/run/samba/ncalrpc/EPMAPPER)
>>>> failed: No
>>>>>>> such file or directory
>>>>>>> [2022/11/15 10:10:57.820626, 1]
>>>>>>> ../../source3/rpc_server/rpc_host.c:1763(rpc_worker_exited)
>>>>>>> rpc_worker_exited: No worker with PID 3281
>>>>>>> [2022/11/15 10:10:58.040001, 1]
>>>>>>> ../../source3/rpc_server/rpc_host.c:1350(rpc_host_distribute_clients)
>>>>>>> rpc_host_distribute_clients: Sending new client
>>>>>>> /usr/libexec/samba/rpcd_winreg to 3294 with 0 clients
>>>>>>> [2022/11/15 10:10:58.048701, 1]
>>>>>>> ../../source3/rpc_server/rpc_host.c:1350(rpc_host_distribute_clients)
>>>>>>> rpc_host_distribute_clients: Sending new client
>>>>>>> /usr/libexec/samba/rpcd_winreg to 3294 with 0 clients
>>>>>>> [2022/11/15 10:10:58.049474, 1]
>>>>>>> ../../source3/rpc_server/rpc_host.c:1350(rpc_host_distribute_clients)
>>>>>>> rpc_host_distribute_clients: Sending new client
>>>>>>> /usr/libexec/samba/rpcd_classic to 3292 with 0 clients
>>>>>>> [2022/11/15 10:10:58.560868, 1]
>>>>>>> ../../source3/rpc_server/rpc_host.c:1350(rpc_host_distribute_clients)
>>>>>>> rpc_host_distribute_clients: Sending new client
>>>>>>> /usr/libexec/samba/rpcd_classic to 3292 with 0 clients
>>>>>>>
>>>>>>> Samba is in clustered mode + registry:
>>>>>>>
>>>>>>> [root at fs01 samba]# net conf list
>>>>>>> [global]
>>>>>>> logging = syslog
>>>>>>> log level = 1
>>>>>>> netbios name = fs
>>>>>>> workgroup = xxx
>>>>>>> realm = xxx
>>>>>>> idmap config * : backend = autorid
>>>>>>> idmap config * : range = 1000000-1999999
>>>>>>> security = user
>>>>>> Now I do not know a lot about CTDB, but I do know that you cannot use
>>>>>> 'idmap config' lines with 'security = user', they are are only used
>>>> with
>>>>>> a domain, so if this cluster is joined to a domain, I would start by
>>>>>> changing 'security = user' to 'security = ADS'
>>>>>>
>>>>>> Rowland
>>>>>>
>>>>>> --
>>>>>> To unsubscribe from this list go to the following URL and read the
>>>>>> instructions: https://lists.samba.org/mailman/options/samba
>>>>>>
>>>>
>>>> --
>>>> To unsubscribe from this list go to the following URL and read the
>>>> instructions: https://lists.samba.org/mailman/options/samba
>>>>
>>>
>>> --
>>> --
>>> Leszek A. Szczepanowski
>>> twinsen at mspanc.net
>>>
>>>
>>>
>> --
>> --
>> Leszek A. Szczepanowski
>> twinsen at mspanc.net
>>
>
More information about the samba
mailing list