[Samba] ctdb + gluster9 = not working
Anton Shevtsov
shevtsovay at basealt.ru
Tue Jan 14 07:19:58 UTC 2025
i fixed))
[root at samba1 ~]# ctdb event script list legacy
00.ctdb
01.reclock
05.system
06.nfs
10.interface
11.natgw
11.routing
13.per_ip_routing
20.multipathd
31.clamd
40.vsftpd
41.httpd
48.netbios
49.winbind
50.samba
60.nfs
70.iscsi
91.lvs
[root at samba1 ~]# ctdb event script enable legacy 10.interface
[root at samba1 ~]# ctdb event script list legacy
00.ctdb
01.reclock
05.system
06.nfs
* 10.interface
11.natgw
11.routing
13.per_ip_routing
20.multipathd
31.clamd
40.vsftpd
41.httpd
48.netbios
49.winbind
50.samba
60.nfs
70.iscsi
91.lvs
2: ens19: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel
state UP group default qlen 1000
altname enp0s19
inet 192.168.160.151/24 brd 192.168.160.255 scope global
noprefixroute ens19
valid_lft forever preferred_lft forever
inet 192.168.160.180/24 brd 192.168.160.255 scope global secondary
ens19
valid_lft forever preferred_lft forever
14.01.2025 12:15, Anton Shevtsov via samba пишет:
>
> 29.12.2024 04:25, Martin Schwenke via samba пишет:
>> Hi Anton,
>>
>> On Fri, 27 Dec 2024 20:04:07 +0500, Anton Shevtsov via samba
>> <samba at lists.samba.org> wrote:
>>
>>> Hi, i tried make a samba cluster (ctdb) with gluster9 (do not use vfs
>>> module)
>>>
>>> But i get an errors.
>>>
>>> [root at samba1 ctdb]# rpm -q samba samba-ctdb
>>> samba-4.19.9-alt3.x86_64
>>> samba-ctdb-4.19.9-alt3.x86_64
>>>
>>> [root at samba1 ctdb]# grep -v '^#' /etc/ctdb/ctdb.conf
>>> [cluster]
>>> recovery lock = "/mnt/gluster/ctdb/.ctdb.lock"
>> [...]
>>
>>> дек 27 19:46:54 samba1 systemd[1]: Starting CTDB...
>>> дек 27 19:46:54 samba1 ctdbd[25430]: CTDB logging to location
>>> file:/var/log/log.ctdb
>>> дек 27 19:46:54 samba1 systemd[1]: Started CTDB.
>>>
>>> [root at samba1 ctdb]# tail -f /var/log/log.ctdb
>>> [...]
>>> 2024-12-27T19:47:24.597406+05:00 samba1 ctdb-recoverd[25443]: Start
>>> election
>>> 2024-12-27T19:47:24.598139+05:00 samba1 ctdb-recoverd[25443]:
>>> Attempting to take cluster lock ("/mnt/gluster/ctdb/.ctdb.lock")
>>> 2024-12-27T19:47:24.611930+05:00 samba1 ctdbd[25431]:
>>> ctdb_mutex_fcntl_helper: Unable to open
>>> "/mnt/gluster/ctdb/.ctdb.lock" - (No such file or directory)
>>> 2024-12-27T19:47:24.612311+05:00 samba1 ctdb-recoverd[25443]: Unable
>>> to take cluster lock - unknown error
>>> [...]
>>> Why?
>> Good question! This took me a few guesses... because it certainly isn't
>> obvious! :-)
>>
>> You don't need the double-quotes around the recovery lock setting.
>> They are not inserted by the log messages, so must be maintained from
>> ctdb.conf and not stripped. Samba (and CTDB) log files can handle
>> values containing whitespace, so quoting isn't required.
>>
>> If your Linux distribution included the quotes in the sample
>> configuration file then you should report a bug because the example is
>> misleading. If not, you're just unlucky because you did something sane
>> but it didn't work.
>>
>> peace & happiness,
>> martin
>
> So simple! ;)))) Thanks!
>
> I can make a next step.
>
> [root at samba1 ~] systemctl start ctdb
>
> [root at samba2 ~] systemctl start ctdb
>
> [root at samba2 ~]# ctdb status
> Number of nodes:2
> pnn:0 192.168.160.151 OK
> pnn:1 192.168.160.152 OK (THIS NODE)
> Generation:1076686332
> Size:2
> hash:0 lmaster:0
> hash:1 lmaster:1
>
> [root at samba1 ctdb]# tail -f /var/log/log.ctdb
>
> [root at samba1 ~]# tail -f /var/log/log.ctdb
> 2025-01-14T12:02:34.731080+05:00 samba1 ctdbd[6033]: Freeze db:
> account_policy.tdb
> 2025-01-14T12:02:34.735230+05:00 samba1 ctdbd[6033]: Freeze db:
> group_mapping.tdb
> 2025-01-14T12:02:34.739394+05:00 samba1 ctdbd[6033]: Freeze db:
> share_info.tdb
> 2025-01-14T12:02:34.743590+05:00 samba1 ctdbd[6033]: Freeze db:
> registry.tdb
> 2025-01-14T12:02:34.747753+05:00 samba1 ctdbd[6033]: Set runstate to
> SETUP (2)
> 2025-01-14T12:02:34.763874+05:00 samba1 ctdbd[6033]: Keepalive
> monitoring has been started
> 2025-01-14T12:02:34.764010+05:00 samba1 ctdbd[6033]: Set runstate to
> FIRST_RECOVERY (3)
> 2025-01-14T12:02:34.764520+05:00 samba1 ctdb-recoverd[6045]:
> monitor_cluster starting
> 2025-01-14T12:02:35.764460+05:00 samba1 ctdbd[6033]:
> CTDB_WAIT_UNTIL_RECOVERED
> 2025-01-14T12:02:36.765546+05:00 samba1 ctdbd[6033]:
> CTDB_WAIT_UNTIL_RECOVERED
> 2025-01-14T12:02:37.766605+05:00 samba1 ctdbd[6033]:
> CTDB_WAIT_UNTIL_RECOVERED
> 2025-01-14T12:02:38.767048+05:00 samba1 ctdbd[6033]:
> CTDB_WAIT_UNTIL_RECOVERED
> 2025-01-14T12:02:39.767644+05:00 samba1 ctdbd[6033]:
> CTDB_WAIT_UNTIL_RECOVERED
> 2025-01-14T12:02:39.768239+05:00 samba1 ctdb-recoverd[6045]: Leader
> broadcast timeout
> 2025-01-14T12:02:39.768764+05:00 samba1 ctdb-recoverd[6045]: Start
> election
> 2025-01-14T12:02:39.769124+05:00 samba1 ctdb-recoverd[6045]:
> Attempting to take cluster lock (/mnt/gluster/ctdb.lock)
> 2025-01-14T12:02:39.769310+05:00 samba1 ctdb-recoverd[6045]: Set
> cluster mutex helper to "/usr/lib/ctdb/ctdb_mutex_fcntl_helper"
> 2025-01-14T12:02:39.789394+05:00 samba1 ctdb-recoverd[6045]: Cluster
> lock taken successfully
> 2025-01-14T12:02:39.789657+05:00 samba1 ctdb-recoverd[6045]: Took
> cluster lock, leader=0
> 2025-01-14T12:02:39.790853+05:00 samba1 ctdb-recoverd[6045]: Node:0
> was in recovery mode. Start recovery process
> 2025-01-14T12:02:39.791059+05:00 samba1 ctdb-recoverd[6045]:
> ../../ctdb/server/ctdb_recoverd.c:1314 Starting do_recovery
> 2025-01-14T12:02:39.791262+05:00 samba1 ctdb-recoverd[6045]:
> ../../ctdb/server/ctdb_recoverd.c:1358 Recovery initiated due to
> problem with node 0
> 2025-01-14T12:02:39.791741+05:00 samba1 ctdb-recoverd[6045]:
> ../../ctdb/server/ctdb_recoverd.c:1388 Recovery - updated flags
> 2025-01-14T12:02:39.791946+05:00 samba1 ctdb-recoverd[6045]: Set
> recovery_helper to "/usr/lib/ctdb/ctdb_recovery_helper"
> 2025-01-14T12:02:39.800805+05:00 samba1 ctdb-recovery[6051]: Set
> recovery mode to ACTIVE
> 2025-01-14T12:02:39.801001+05:00 samba1 ctdbd[6033]: Recovery has started
> 2025-01-14T12:02:39.801254+05:00 samba1 ctdb-recovery[6051]:
> start_recovery event finished
> 2025-01-14T12:02:39.801421+05:00 samba1 ctdb-recovery[6051]: updated
> VNNMAP
> 2025-01-14T12:02:39.801492+05:00 samba1 ctdb-recovery[6051]: recover
> database 0x7132c184
> 2025-01-14T12:02:39.801571+05:00 samba1 ctdb-recovery[6051]: recover
> database 0x3ef19640
> 2025-01-14T12:02:39.801678+05:00 samba1 ctdb-recovery[6051]: recover
> database 0x2ca251cf
> 2025-01-14T12:02:39.801742+05:00 samba1 ctdb-recovery[6051]: recover
> database 0xa1413774
> 2025-01-14T12:02:39.801800+05:00 samba1 ctdb-recovery[6051]: recover
> database 0xc3078fba
> 2025-01-14T12:02:39.801862+05:00 samba1 ctdb-recovery[6051]: recover
> database 0x6cf2837d
> 2025-01-14T12:02:39.802221+05:00 samba1 ctdbd[6033]: Freeze db:
> secrets.tdb frozen
> 2025-01-14T12:02:39.802309+05:00 samba1 ctdbd[6033]: Freeze db:
> passdb.tdb frozen
> 2025-01-14T12:02:39.802351+05:00 samba1 ctdbd[6033]: Freeze db:
> account_policy.tdb frozen
> 2025-01-14T12:02:39.802379+05:00 samba1 ctdbd[6033]: Freeze db:
> group_mapping.tdb frozen
> 2025-01-14T12:02:39.802403+05:00 samba1 ctdbd[6033]: Freeze db:
> share_info.tdb frozen
> 2025-01-14T12:02:39.802488+05:00 samba1 ctdbd[6033]: Freeze db:
> registry.tdb frozen
> 2025-01-14T12:02:40.652978+05:00 samba1 ctdbd[6033]: Thaw db:
> secrets.tdb generation 1827381565
> 2025-01-14T12:02:40.653088+05:00 samba1 ctdbd[6033]: Release freeze
> handle for db secrets.tdb
> 2025-01-14T12:02:40.653189+05:00 samba1 ctdbd[6033]: Thaw db:
> passdb.tdb generation 1827381565
> 2025-01-14T12:02:40.653226+05:00 samba1 ctdbd[6033]: Release freeze
> handle for db passdb.tdb
> 2025-01-14T12:02:40.653283+05:00 samba1 ctdbd[6033]: Thaw db:
> account_policy.tdb generation 1827381565
> 2025-01-14T12:02:40.653323+05:00 samba1 ctdbd[6033]: Release freeze
> handle for db account_policy.tdb
> 2025-01-14T12:02:40.653397+05:00 samba1 ctdbd[6033]: Thaw db:
> group_mapping.tdb generation 1827381565
> 2025-01-14T12:02:40.653441+05:00 samba1 ctdbd[6033]: Release freeze
> handle for db group_mapping.tdb
> 2025-01-14T12:02:40.653660+05:00 samba1 ctdbd[6033]: Thaw db:
> share_info.tdb generation 1827381565
> 2025-01-14T12:02:40.653730+05:00 samba1 ctdbd[6033]: Release freeze
> handle for db share_info.tdb
> 2025-01-14T12:02:40.655336+05:00 samba1 ctdbd[6033]: Thaw db:
> registry.tdb generation 1827381565
> 2025-01-14T12:02:40.655398+05:00 samba1 ctdbd[6033]: Release freeze
> handle for db registry.tdb
> 2025-01-14T12:02:40.658394+05:00 samba1 ctdb-recovery[6051]: 6 of 6
> databases recovered
> 2025-01-14T12:02:40.658577+05:00 samba1 ctdbd[6033]: Recovery mode set
> to NORMAL
> 2025-01-14T12:02:40.658666+05:00 samba1 ctdbd[6033]: Set cluster mutex
> helper to "/usr/lib/ctdb/ctdb_mutex_fcntl_helper"
> 2025-01-14T12:02:40.677257+05:00 samba1 ctdb-recovery[6051]: Set
> recovery mode to NORMAL
> 2025-01-14T12:02:40.677419+05:00 samba1 ctdbd[6033]: Recovery has
> finished
> 2025-01-14T12:02:40.678242+05:00 samba1 ctdbd[6033]: Set runstate to
> STARTUP (4)
> 2025-01-14T12:02:40.678505+05:00 samba1 ctdb-recovery[6051]: recovered
> event finished
> 2025-01-14T12:02:40.679156+05:00 samba1 ctdb-recoverd[6045]: Takeover
> run starting
> 2025-01-14T12:02:40.679362+05:00 samba1 ctdb-recoverd[6045]: Set
> takeover_helper to "/usr/lib/ctdb/ctdb_takeover_helper"
> 2025-01-14T12:02:40.692782+05:00 samba1 ctdb-takeover[6053]: No nodes
> available to host public IPs yet
> 2025-01-14T12:02:40.693404+05:00 samba1 ctdb-recoverd[6045]: Takeover
> run completed successfully
> 2025-01-14T12:02:40.693665+05:00 samba1 ctdb-recoverd[6045]:
> ../../ctdb/server/ctdb_recoverd.c:1406 Recovery complete
> 2025-01-14T12:02:40.693874+05:00 samba1 ctdb-recoverd[6045]: Resetting
> ban count to 0 for all nodes
> 2025-01-14T12:02:40.694146+05:00 samba1 ctdb-recoverd[6045]: Just
> finished a recovery. New recoveries will now be suppressed for the
> rerecovery timeout (10 seconds)
> 2025-01-14T12:02:40.694263+05:00 samba1 ctdb-recoverd[6045]: Disabling
> recoveries for 10 seconds
> 2025-01-14T12:02:40.767822+05:00 samba1 ctdbd[6033]:
> CTDB_WAIT_UNTIL_RECOVERED
> 2025-01-14T12:02:40.767888+05:00 samba1 ctdbd[6033]:
> ../../ctdb/server/ctdb_monitor.c:324 wait for pending recoveries to
> end. Wait one more second.
> 2025-01-14T12:02:40.792535+05:00 samba1 ctdb-recoverd[6045]: Initial
> interface fetched
> 2025-01-14T12:02:40.792732+05:00 samba1 ctdb-recoverd[6045]: Trigger
> takeoverrun
> 2025-01-14T12:02:40.792908+05:00 samba1 ctdb-recoverd[6045]: Takeover
> run starting
> 2025-01-14T12:02:40.805878+05:00 samba1 ctdb-takeover[6054]: No nodes
> available to host public IPs yet
> 2025-01-14T12:02:40.806450+05:00 samba1 ctdb-recoverd[6045]: Takeover
> run completed successfully
> 2025-01-14T12:02:41.768559+05:00 samba1 ctdbd[6033]:
> CTDB_WAIT_UNTIL_RECOVERED
> 2025-01-14T12:02:41.768659+05:00 samba1 ctdbd[6033]:
> ../../ctdb/server/ctdb_monitor.c:324 wait for pending recoveries to
> end. Wait one more second.
> 2025-01-14T12:02:42.769690+05:00 samba1 ctdbd[6033]:
> CTDB_WAIT_UNTIL_RECOVERED
> 2025-01-14T12:02:42.769777+05:00 samba1 ctdbd[6033]:
> ../../ctdb/server/ctdb_monitor.c:324 wait for pending recoveries to
> end. Wait one more second.
> 2025-01-14T12:02:43.769866+05:00 samba1 ctdbd[6033]:
> CTDB_WAIT_UNTIL_RECOVERED
> 2025-01-14T12:02:43.769971+05:00 samba1 ctdbd[6033]:
> ../../ctdb/server/ctdb_monitor.c:324 wait for pending recoveries to
> end. Wait one more second.
> 2025-01-14T12:02:44.770295+05:00 samba1 ctdbd[6033]:
> CTDB_WAIT_UNTIL_RECOVERED
> 2025-01-14T12:02:44.770385+05:00 samba1 ctdbd[6033]:
> ../../ctdb/server/ctdb_monitor.c:324 wait for pending recoveries to
> end. Wait one more second.
> 2025-01-14T12:02:45.770756+05:00 samba1 ctdbd[6033]:
> CTDB_WAIT_UNTIL_RECOVERED
> 2025-01-14T12:02:45.770847+05:00 samba1 ctdbd[6033]:
> ../../ctdb/server/ctdb_monitor.c:324 wait for pending recoveries to
> end. Wait one more second.
> 2025-01-14T12:02:46.771018+05:00 samba1 ctdbd[6033]:
> CTDB_WAIT_UNTIL_RECOVERED
> 2025-01-14T12:02:46.771115+05:00 samba1 ctdbd[6033]:
> ../../ctdb/server/ctdb_monitor.c:324 wait for pending recoveries to
> end. Wait one more second.
> 2025-01-14T12:02:47.771442+05:00 samba1 ctdbd[6033]:
> CTDB_WAIT_UNTIL_RECOVERED
> 2025-01-14T12:02:47.771529+05:00 samba1 ctdbd[6033]:
> ../../ctdb/server/ctdb_monitor.c:324 wait for pending recoveries to
> end. Wait one more second.
> 2025-01-14T12:02:48.772105+05:00 samba1 ctdbd[6033]:
> CTDB_WAIT_UNTIL_RECOVERED
> 2025-01-14T12:02:48.772185+05:00 samba1 ctdbd[6033]:
> ../../ctdb/server/ctdb_monitor.c:324 wait for pending recoveries to
> end. Wait one more second.
> 2025-01-14T12:02:49.772721+05:00 samba1 ctdbd[6033]:
> CTDB_WAIT_UNTIL_RECOVERED
> 2025-01-14T12:02:49.772832+05:00 samba1 ctdbd[6033]:
> ../../ctdb/server/ctdb_monitor.c:324 wait for pending recoveries to
> end. Wait one more second.
> 2025-01-14T12:02:50.695560+05:00 samba1 ctdb-recoverd[6045]:
> Reenabling recoveries after timeout
> 2025-01-14T12:02:50.773839+05:00 samba1 ctdbd[6033]:
> CTDB_WAIT_UNTIL_RECOVERED
> 2025-01-14T12:02:50.773907+05:00 samba1 ctdbd[6033]:
> ../../ctdb/server/ctdb_monitor.c:324 wait for pending recoveries to
> end. Wait one more second.
> 2025-01-14T12:02:51.774280+05:00 samba1 ctdbd[6033]:
> CTDB_WAIT_UNTIL_RECOVERED
> 2025-01-14T12:02:51.774384+05:00 samba1 ctdbd[6033]:
> ../../ctdb/server/ctdb_monitor.c:324 wait for pending recoveries to
> end. Wait one more second.
> 2025-01-14T12:02:52.774463+05:00 samba1 ctdbd[6033]:
> CTDB_WAIT_UNTIL_RECOVERED
> 2025-01-14T12:02:52.774558+05:00 samba1 ctdbd[6033]:
> ../../ctdb/server/ctdb_monitor.c:324 wait for pending recoveries to
> end. Wait one more second.
> 2025-01-14T12:02:53.775182+05:00 samba1 ctdbd[6033]:
> CTDB_WAIT_UNTIL_RECOVERED
> 2025-01-14T12:02:53.775367+05:00 samba1 ctdbd[6033]:
> ctdb_recheck_persistent_health: OK[6] FAIL[0]
> 2025-01-14T12:02:53.775413+05:00 samba1 ctdbd[6033]: Running the
> "startup" event.
> 2025-01-14T12:02:53.775832+05:00 samba1 ctdbd[6033]: startup event OK
> - enabling monitoring
> 2025-01-14T12:02:53.775876+05:00 samba1 ctdbd[6033]: Set runstate to
> RUNNING (5)
> 2025-01-14T12:02:55.777187+05:00 samba1 ctdbd[6033]: monitor event OK
> - node re-enabled
> 2025-01-14T12:02:55.778083+05:00 samba1 ctdbd[6033]: Node became
> HEALTHY. Ask recovery master to reallocate IPs
> 2025-01-14T12:02:55.802465+05:00 samba1 ctdb-recoverd[6045]:
> Unassigned IP 192.168.160.180 can be served by this node
> 2025-01-14T12:02:55.802591+05:00 samba1 ctdb-recoverd[6045]: Trigger
> takeoverrun
> 2025-01-14T12:02:55.802770+05:00 samba1 ctdb-recoverd[6045]: Takeover
> run starting
> 2025-01-14T12:02:55.813236+05:00 samba1 ctdbd[6033]: Takeover of IP
> 192.168.160.180/24 on interface ens19
> 2025-01-14T12:02:55.819797+05:00 samba1 ctdb-recoverd[6045]: Takeover
> run completed successfully
> 2025-01-14T12:02:56.802871+05:00 samba1 ctdb-recoverd[6045]: Assigned
> IP 192.168.160.180 not on an interface
> 2025-01-14T12:02:56.802933+05:00 samba1 ctdb-recoverd[6045]: Trigger
> takeoverrun
> 2025-01-14T12:02:56.803143+05:00 samba1 ctdb-recoverd[6045]: Takeover
> run starting
> 2025-01-14T12:02:56.813856+05:00 samba1 ctdbd[6033]: Takeover of IP
> 192.168.160.180/24 on interface ens19
> 2025-01-14T12:02:56.824962+05:00 samba1 ctdb-recoverd[6045]: Takeover
> run completed successfully
> 2025-01-14T12:02:57.803911+05:00 samba1 ctdb-recoverd[6045]: Assigned
> IP 192.168.160.180 not on an interface
> 2025-01-14T12:02:57.803980+05:00 samba1 ctdb-recoverd[6045]: Trigger
> takeoverrun
> 2025-01-14T12:02:57.804181+05:00 samba1 ctdb-recoverd[6045]: Takeover
> run starting
> 2025-01-14T12:02:57.815061+05:00 samba1 ctdbd[6033]: Takeover of IP
> 192.168.160.180/24 on interface ens19
> 2025-01-14T12:02:57.823951+05:00 samba1 ctdb-recoverd[6045]: Takeover
> run completed successfully
> 2025-01-14T12:02:58.804912+05:00 samba1 ctdb-recoverd[6045]: Assigned
> IP 192.168.160.180 not on an interface
> 2025-01-14T12:02:58.804980+05:00 samba1 ctdb-recoverd[6045]: Trigger
> takeoverrun
> 2025-01-14T12:02:58.805180+05:00 samba1 ctdb-recoverd[6045]: Takeover
> run starting
> 2025-01-14T12:02:58.816689+05:00 samba1 ctdbd[6033]: Takeover of IP
> 192.168.160.180/24 on interface ens19
> 2025-01-14T12:02:58.822922+05:00 samba1 ctdb-recoverd[6045]: Takeover
> run completed successfully
> .....
>
> and again and again repeated last lines (assign ip ..)
>
> [root at samba2 ~]# ctdb ip
> Public IPs on node 1
> 192.168.160.180 0
>
> but 192.168.160.180 not assign to ens19
>
> [root at samba1 ~]# ip -4 -f inet a show ens19
> 2: ens19: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel
> state UP group default qlen 1000
> altname enp0s19
> inet 192.168.160.151/24 brd 192.168.160.255 scope global
> noprefixroute ens19
> valid_lft forever preferred_lft forever
>
> [root at samba1 ~]# cat /etc/ctdb/public_addresses
> 192.168.160.180/24 ens19
>
> [root at samba2 ~]# cat /etc/ctdb/public_addresses
> 192.168.160.180/24 ens19
>
> [root at samba2 ~]# ip -4 -f inet a show ens19
> 2: ens19: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel
> state UP group default qlen 1000
> altname enp0s19
> inet 192.168.160.152/24 brd 192.168.160.255 scope global
> noprefixroute ens19
> valid_lft forever preferred_lft forever
>
> why 192.168.160.180 not assigned ?
>
--
basealt logo *Шевцов Антон Юрьевич*
Эксперт по инфраструктуре
Дирекция по проектам ООО Базальт СПО
мобильный : +79222651692
telegram : @anton_shevtsov
More information about the samba
mailing list