Greyhole VFS module(s)

[Adam Tauno Williams]

On Thu, 2010-11-25 at 11:56 -0800, Jeremy Allison wrote: 
> On Thu, Nov 25, 2010 at 02:53:08PM -0500, Guillaume Boudreau wrote:
> > Hi,
> > Jeremy contacted me today about a possible inclusion of the Greyhole
> > VFS module in mainline Samba.
> > I'm thrilled by the idea, so here's the VFS module in question.
> > For those who don't know Greyhole (that would be most if not all of
> > you), you can read about it here: http://code.google.com/p/greyhole/
> > "Greyhole is an application that uses Samba to create a storage pool
> > of all your available hard drives (whatever their size, however
> > they're connected), and allows you to create redundant copies of the
> > files you store, in order to prevent data loss when part of your
> > hardware fails."
> > I have two versions (one for 3.4, one for 3.5) because my 3.4 version
> > wouldn't compile on 3.5. Maybe there's a way to unify both into one
> > file, but I don't know how.
> > Suggestions are welcome. (Code reviews too!)
> > The module is quite simple; it simply logs to syslogd
> > (/var/log/messages) all file operations that Greyhole cares about.
> > Those logs are formated in *-seperated fields: yuck! I'm sure someone
> > can suggest a better approach than this to keep those logs
> > machine-readable!
> Do you have to log these to syslogd ? Is there a better
> way to communicate with Greyhole ? You might want to
> look at the code in modules/vfs_smb_traffic_analyzer.c
> that communicates over a UNIX domain socket to an external
> daemon that logs all Samba traffic. Could Greyhole be more
> efficient using this ?

I'm puzzled why more facilities like this don't just use proper
messaging systems like AMQP (RabbitMQ).  They are reliable, extremely
fast, scalable, and simple [for the application to use anyway].
<http://www.rabbitmq.com/>
<http://www.rabbitmq.com/getstarted.html>
<http://www.rabbitmq.com/tutorial-one-python.html>
<http://hg.rabbitmq.com/rabbitmq-c/file/default/README>

Just my two cents.

<aside>
Provisioning Rabbit even on an 'unfriendly' distro like CentOS is
trivial -
cd /tmp

curl -o epel-release-5-4.noarch.rpm
http://download.fedora.redhat.com/pub/epel/5/i386/epel-release-5-4.noarch.rpm

rpm -Uvh epel-release-5-4.noarch.rpm

yum -y install erlang

curl -o rabbitmq-server-1.7.2-1.i386.rpm
http://www.rabbitmq.com/releases/rabbitmq-server/v1.7.2/rabbitmq-server-1.7.2-1.i386.rpm

rpm -Uvh rabbitmq-server-1.7.2-1.i386.rpm

echo "export RABBITMQ_NODE_IP_ADDRESS=127.0.0.1"
>  /etc/sysconfig/rabbitmq

chmod u+x /etc/sysconfig/rabbitmq

/etc/init.d/rabbitmq-server start

chkconfig rabbitmq-server on
</aside>