[clug] Clustering file systems, without excluding windows

[Robert Edwards]

Speaking of ATA over Ethernet (AoE), we are just in the process of
commissioning a Coraid SR1521 15-disk AoE rack (with 8 x 750GB drives).

I might be at a stage to lug it downstairs at next weeks CLUG meeting
if anyone wants an up-close squiz at the innards and a short talk about
how it all fits together.

At this stage, I am looking at Lustre and GFS as two candidates for
the clustered F/S. Andrew Tridgell spoke about IBMs proprietry GPFS
during his recent CLUG talk on Samba Clustering - we aren't interested
in going there at this stage. I have no interest in trying to make
mine "work with Windoze", so can't help on that at all.

Cheers,

Bob Edwards.

Paul TBBle Hampson wrote:
> [Disclaimer: This is work-related. Feel free to ignore it if you
> would expect that answering this should be paid-consultant work.]
> 
> Everyone,
> 
> I've got a few servers together running an ISP, and it's currently
> operating with basically RAID-1 and rdiff-backup across samba, at
> least for the actual files. (Database backups are handled differently)
> 
> I'd like to move to some sort of clustered/NAS filesystem, both to
> centralise at least one copy of the data someone I can back it up
> without pulling it across the LAN, and partly so I can cluster the
> various servers out.
> 
> I've been staring hard at this, and I'm not sure I've wrapped my head
> around it completely, ever since Michael Still mentioned
> ATA-over-Ethernet talking about a NSLU2 at CLUG one night.
> 
> If I'm understanding correctly, I put disks in a server, mdadm them into
> a raid away, partition it (prolly with mdp, but I guess LVM is possible
> too) and export the partitions with either AoE or iSCSI.
> 
> I then put a clustered file system on top of that, which runs on each
> machine accessing the exported block devices as local device names, but
> distributing locking using something like DLM.
> 
> For the mail spool, that looks quite safe and easy. I then throw vserver
> in front to handle POP3 (or simply an iptables rule + heartbeat would
> sort that one out) and I can run SMTP servers on as many machines as I
> like, at least for incoming email. And voila, distributed mail hosting.
> (Well, I dunno how I'm going to handle mailman yet... Luckily, mailman
> is always its own domain, so I can easily just have a dedicated mailman
> machine again with heartbeatd or something.)
> 
> Since I'm using postfix, and only one machine can use a specific spool,
> I can't do the same thing for outgoing email... I guess I could, and if
> one of the outgoing machines catches file, the risk is only to stuff in
> that one's spool. That'll save me having to try and split one DNS entry
> "mail" into "smtp" and "mx", given the swathe of hosted clients.
> 
> The problem arises from web service. Specifically, the IIS machine. What
> I'd like is to have it use the same file hierarchy as the linux web
> servers, so I can unify the various site-creation scripts and pathname
> expectations.
> 
> I haven't seen a free clustered file system that includes a Windows
> client, so I'm thinking I throw samba on one of the machines in the
> cluster, and export the tree to the windows machine.
> 
> Has anyone tried having a single samba exporting a cluster-shared file
> system like this, and/or run into any problems trying?
> 
> This'd also allow me to distribute FTP access to website directories,
> which'd be nice.
> 
> The other big advantage is I could test sites for migration off the IIS
> server without having to actually copy the data onto one of the linux
> webservers and try it, but just by adding the appropriate Apache config
> snippet.
> 
> If anyone's got any comments or suggetions, I'd love to hear them. As I
> say, I'm having a little trouble getting my head around this in the
> limited time I have to throw at such things.
> 
>