Setting up CTDB on OCFS2 and VMs ...

Ralph Böhme rb at sernet.de
Wed Dec 17 02:27:16 MST 2014


Hi Rowland,

On Tue, Dec 16, 2014 at 10:08:08PM +0000, Rowland Penny wrote:
> >can you share the bits from Debain to ocfs2? I'll set this up the next
> >day and see if I get ctdb to behave.
>
> OK, I based this on what Richard posted:
> 
> 1. Create two VirtualBox VMs with enough memory and disk for your
> Linux Distro. I used Debian 7.7 with 512MB and 8GB. You will also
> need an extra interface on each VM for the clustering private
> network. I set them to an internal type.
> 
> 2. Because you will need a shared disk, create one:
> 
> vboxmanage createhd --filename ~/VirtualBox\ VMs/SharedHD1 --size
> 10240 --variant Fixed --format VDI # Creates a 10GB fixed sized disk
> 
> vboxmanage modifyhd ~/VirtualBox\ VMs/SharedHD1.vdi --type shareable
> 
> Also, use the GUI to add the shared disk to both VMs.
> 
> 3. Install the OS on each of the VM's.
> 
> 4. Install clustering packages:
> 
> apt-get install openais corosync pacemaker ocfs2-tools-pacemaker dlm-pcmk
> 
> 5. Configure corosync
> 
> nano /etc/corosync/corosync.conf
> 
> Make sure that bindnetaddr is defined and points to your private
> interface. I set it to 192.168.1.0
> 
> # Copy the file to the other node.
> scp /etc/corosync/corosync.conf root at 192.168.0.7:/etc/corosync/
> 
> # ON BOTH NODES
> service pacemaker stop  # Stop these in case they were running
> service corosync stop # Same here
> 
> nano /etc/default/corosync
> Change:
> START=no
> 
> To:
> 
> START=yes
> 
> # now start the cluster
> service corosync start
> 
> # Also start it on the other node(s).
> 
> # Now check the status:
> root at cluster1:~# crm_mon -1
> ============
> Last updated: Mon Dec 15 10:46:20 2014
> Last change: Mon Dec 15 10:44:18 2014 via crmd on cluster1
> Stack: openais
> Current DC: cluster1 - partition with quorum
> Version: 1.1.7-ee0730e13d124c3d58f00016c3376a1de5323cff
> 2 Nodes configured, 2 expected votes
> 0 Resources configured.
> ============
> 
> Online: [ cluster1 cluster2 ]
> 
> If you do not see all the other nodes online, then you have to debug
> the problem.
> 
> 6. Configure the Oracle cluster
> 
> dpkg-reconfigure ocfs2-tools
> Configuring ocfs2-tools
> # Would you like to start an OCFS2 cluster (O2CB) at boot time?
> #
> # <Yes>
> #
> # Name of the cluster to start at boot time:
> #
> # ctdbdemo
> 
> # Create the ocfs2 cluster conf file
> o2cb_ctl -C -n ctdbdemo -t cluster
> o2cb_ctl -C -n cluster1 -t node -a number=1 -a
> ip_address=192.168.1.10 -a ip_port=7777 -a cluster=ctdbdemo
> o2cb_ctl -C -n cluster2 -t node -a number=2 -a
> ip_address=192.168.1.11 -a ip_port=7777 -a cluster=ctdbdemo
> 
> # ON BOTH NODES
> service corosync stop
> 
> # Copy files to the other node.
> scp /etc/default/o2cb  root at 192.168.0.7:/etc/default/
> scp /etc/ocfs2/cluster.conf  root at 192.168.0.7:/etc/ocfs2/
> 
> service o2cb start
> service corosync start
> 
> crm configure property stonith-enabled=false
> 
> 7. Create the shared shared file system on one node:
> 
> mkfs.ocfs2 -L CTDBdemocommon -T datafiles -N 4 /dev/sdb
> 
> 8. Mount it on both and ensure that you can create files/dirs on one
> node and see them on the other node.
> 
> mkdir /cluster
> mount -t ocfs2 /dev/sdb /cluster
> 
> This gets you a shared cluster??
> 
> For me it all goes pear shaped when I try to add CTDB.
> 
> If you find that I have missed something or done something wrong,
> then I will not be surprised, info is very hard to find.

ok, this looks like you're enabling the lock manager (DLM) which is a
pita. Configuring ctdb is a breeze compared to that, I swear! ;)

The problem is a lack of documentation for configuring OCSF2 with
Pacemaker and friends on Debian and derived distros. Here are some
pointers I bookmarked over the last days:

<http://blog.clusterlabs.org/blog/2012/pacemaker-and-cluster-filesystems/>
<http://www.hastexo.com/resources/hints-and-kinks/ocfs2-pacemaker-debianubuntu>

Afaict even the official Pacemaker guide for Ubuntu doesn't cover
enabling the DLM:
<http://clusterlabs.org/quickstart-ubuntu.html>

The next step on my todo list for this is to head over to
<http://oss.clusterlabs.org/mailman/listinfo/pacemaker>

and see if someone can point us the right direction.

I'll also look into enhancing ping_pong to provide a simple and
reliable test for custer-wide locking, stay tuned. :)

-Ralph

-- 
SerNet GmbH, Bahnhofsallee 1b, 37081 Göttingen
phone: +49-551-370000-0, fax: +49-551-370000-9
AG Göttingen, HRB 2816, GF: Dr. Johannes Loxen
http://www.sernet.de,mailto:kontakt@sernet.de


More information about the samba-technical mailing list