[Samba] ctdb issue: existing header for db_id 0xf2a58948 has larger RSN 1 than new RSN 1 in ctdb_persistent_store

[Nate Hardt]

Hello,
I have an issue with ctdb that I am not sure how to handle. I am running
ctdb-1.0.114.3-3. It looks like some sort of error occurs during the first
time initialization. The log looks like the following:

2012/05/11 04:33:16.881297 [16069]: Starting CTDBD as pid : 16069
2012/05/11 04:33:16.987610 [16069]: Starting service nfs
2012/05/11 04:33:17.522194 [16069]: Starting NFS statd: [  OK  ]
2012/05/11 04:33:17.584356 [16069]: Starting NFS services:  [  OK  ]
2012/05/11 04:33:17.599905 [16069]: Starting NFS quotas: [  OK  ]
2012/05/11 04:33:17.610284 [16069]: Starting NFS daemon: [  OK  ]
2012/05/11 04:33:17.614978 [16069]: Starting NFS mountd: [  OK  ]
2012/05/11 04:33:18.690755 [16069]: Freeze priority 1
2012/05/11 04:33:18.690797 [16069]: Freeze priority 2
2012/05/11 04:33:18.690823 [16069]: Freeze priority 3
2012/05/11 04:33:25.366763 [16310]: Taking out recovery lock from recovery
daemon
2012/05/11 04:33:25.366798 [16310]: Take the recovery lock
2012/05/11 04:33:25.384030 [16310]: Recovery lock taken successfully
2012/05/11 04:33:25.384082 [16310]: Recovery lock taken successfully by
recovery daemon
2012/05/11 04:33:25.385542 [16069]: Freeze priority 1
2012/05/11 04:33:25.385925 [16069]: Freeze priority 2
2012/05/11 04:33:25.386353 [16069]: Freeze priority 3
2012/05/11 04:33:25.502987 [16069]: Thawing priority 1
2012/05/11 04:33:25.503015 [16069]: Release freeze handler for prio 1
2012/05/11 04:33:25.503029 [16069]: Thawing priority 2
2012/05/11 04:33:25.503035 [16069]: Release freeze handler for prio 2
2012/05/11 04:33:25.503043 [16069]: Thawing priority 3
2012/05/11 04:33:25.503048 [16069]: Release freeze handler for prio 3
2012/05/11 04:33:25.628783 [16310]: Resetting ban count to 0 for all nodes
2012/05/11 04:33:36.630768 [16310]: Trigger takeoverrun
2012/05/11 04:33:40.311048 [16069]: Vacuuming is disabled for persistent
database secrets.tdb
2012/05/11 04:33:40.318936 [16069]: Starting Winbind services: [  OK  ]
2012/05/11 04:33:40.364715 [16069]: Register srvid 18302628885633695744
for client 65882
2012/05/11 04:33:40.651327 [16310]: Trigger takeoverrun
2012/05/11 04:33:40.715231 [16069]: Vacuuming is disabled for persistent
database registry.tdb
2012/05/11 04:33:40.716911 [16069]: Register srvid 18302628885633695744
for client 590225
2012/05/11 04:33:40.918393 [16069]: Deregister srvid 18302628885633695744
for client 65882
2012/05/11 04:33:41.272762 [16069]: Deregister srvid 18302628885633695744
for client 590225
2012/05/11 04:33:41.277908 [16069]: Register srvid 18302628885633695744
for client 590225
2012/05/11 04:33:41.278179 [16069]: Deregister srvid 18302628885633695744
for client 590225
2012/05/11 04:33:41.278363 [16069]: Register srvid 18302628885633695744
for client 590225
2012/05/11 04:33:41.665087 [16069]: Deregister srvid 18302628885633695744
for client 590225
2012/05/11 04:33:41.812788 [16069]: Starting SMB services: [  OK  ]
2012/05/11 04:33:41.823170 [16069]: Register srvid 18302628885633695744
for client 197066
2012/05/11 04:33:41.823470 [16069]: Deregister srvid 18302628885633695744
for client 197066
2012/05/11 04:33:41.830941 [16310]: Trigger takeoverrun
2012/05/11 04:33:41.868203 [16069]: Vacuuming is disabled for persistent
database idmap2.tdb
2012/05/11 04:33:42.295718 [16069]: Starting NFS statd: [  OK  ]
2012/05/11 04:33:42.356951 [16069]: Starting NFS services:  [  OK  ]
2012/05/11 04:33:42.360369 [16069]: Starting NFS quotas: [  OK  ]
2012/05/11 04:33:42.379459 [16069]: Starting NFS daemon: [  OK  ]
2012/05/11 04:33:42.384079 [16069]: Starting NFS mountd: [  OK  ]
2012/05/11 04:33:42.423498 [16069]: Vacuuming is disabled for persistent
database passdb.tdb
2012/05/11 04:33:43.128728 [16069]: Vacuuming is disabled for persistent
database account_policy.tdb
2012/05/11 04:33:43.130165 [16069]: Register srvid 18302628885633695744
for client 197066
2012/05/11 04:33:43.552421 [16069]: Deregister srvid 18302628885633695744
for client 197066
2012/05/11 04:33:43.553907 [16069]: Register srvid 18302628885633695744
for client 197066
2012/05/11 04:33:44.511739 [16069]: Deregister srvid 18302628885633695744
for client 197066
2012/05/11 04:33:44.650918 [17887]: existing header for db_id 0xf2a58948
has larger RSN 2 than new RSN 1 in ctdb_persistent_store
2012/05/11 04:33:44.650953 [17887]: server/ctdb_persistent.c:548 Failed to
write persistent data
2012/05/11 04:33:44.782324 [16069]: Register srvid 18302628885633695744
for client 197066
2012/05/11 04:33:44.939908 [16069]: Deregister srvid 18302628885633695744
for client 197066
2012/05/11 04:33:44.940054 [16069]: Register srvid 18302628885633695744
for client 197066
2012/05/11 04:33:45.531550 [16069]: Deregister srvid 18302628885633695744
for client 197066
2012/05/11 04:33:45.783438 [16069]: Register srvid 18302628885633695744
for client 197066
2012/05/11 04:33:45.922782 [16069]: Deregister srvid 18302628885633695744
for client 197066
2012/05/11 04:33:45.923314 [16069]: Register srvid 18302628885633695744
for client 197066
2012/05/11 04:33:46.493837 [16069]: Deregister srvid 18302628885633695744
for client 197066
2012/05/11 04:33:46.621968 [16069]: Register srvid 18302628885633695744
for client 197066
2012/05/11 04:33:46.798018 [16069]: Deregister srvid 18302628885633695744
for client 197066
2012/05/11 04:33:46.798741 [16069]: Register srvid 18302628885633695744
for client 197066
2012/05/11 04:33:47.148225 [16069]: Deregister srvid 18302628885633695744
for client 197066
2012/05/11 04:33:47.148612 [16069]: Register srvid 18302628885633695744
for client 197066
2012/05/11 04:33:47.284055 [17954]: existing header for db_id 0xf2a58948
has larger RSN 1 than new RSN 1 in ctdb_persistent_store
2012/05/11 04:33:47.284115 [17954]: server/ctdb_persistent.c:548 Failed to
write persistent data
2012/05/11 04:33:47.475576 [17967]: existing header for db_id 0xf2a58948
has larger RSN 1 than new RSN 1 in ctdb_persistent_store
2012/05/11 04:33:47.475622 [17967]: server/ctdb_persistent.c:548 Failed to
write persistent data
2012/05/11 04:33:47.476030 [17968]: existing header for db_id 0xf2a58948
has larger RSN 1 than new RSN 1 in ctdb_persistent_store
2012/05/11 04:33:47.476059 [17968]: server/ctdb_persistent.c:548 Failed to
write persistent data
2012/05/11 04:33:47.476547 [17969]: existing header for db_id 0xf2a58948
has larger RSN 1 than new RSN 1 in ctdb_persistent_store



These last two messages will repeat until ctdb is restarted or the /
partition fills up. I see it most often in automated regressions that
setup ctdb from scratch. Can anyone provide insight as to what might cause
this and how to avoid it?

Thanks for any help or info you can provide,
Nate Hardt