[Samba] Custom VFS

[Andrew Scherpbier]

Hi Daniel,

Just a note of encouragement...
I have so far written 2 filesystems in Java that use Samba for 2 
different companies, so you're not alone!  :-)

The strategy I've used is to write a simple TCP protocol client (the VFS 
module) and server (a straight forward threaded Java server).
Works like a charm.  As long as the client side is abstracted enough so 
that its samba connection state is independent from the server 
connection state, there are no issues with restarting either.  (I 
started out using a statefull protocol, but ended up changing to a 
completely stateless one, where the individual messages contain enough 
information to establish context.  This way, if either end of the system 
goes down, recovery is the simple act of building a new TCP connection.)

I also attempted to use the Apache ActiveMQ C++ library for 
communication, but found it buggy and leaky.

I originally looked into hosting the JVM in the VFS module, but that was 
going to be a problem because each smbd process would have to start its 
own JVM.  The JVM startup time (especially the server JVM) is very high 
and the memory overhead would not make it scalable.

TCP through the loopback interface is very fast (at least on the linux 
system's I've developed for), so there was no need to implement some 
sort of shared memory interface.

The system I'm working on now manages PB class storage (currently up to 
10PB) with hundreds of concurrent clients and the VFS module does this 
without issues or much overhead.  We're regularly seeing write speeds in 
the 400-500MB/s range using 10GbE and multiple windows clients.

Good luck!

P.S.:  Blatant plug for my current project: 
http://www.cuttedge.com/psca/index.html

On 07/24/2012 01:15 PM, ivenhov wrote:
> Thanks Michael, great link.
> Exactly what I was looking for. Does not answer my JNI question (creating
> file system in Java is not that common),
> but it's a great starting point.
>
> Daniel
>
> On 24 July 2012 19:51, Michael Wood-8 [via Samba] <
> ml-node+s2283325n4634960h22 at n4.nabble.com> wrote:
>
>> Hi
>>
>> On 19 July 2012 11:39, Daniel Iwan <[hidden email]<http://user/SendEmail.jtp?type=node&node=4634960&i=0>>
>> wrote:
>>
>>> Hi
>>>
>>> I would like to  create custom VFS that would redirect all calls to my
>>> backend.
>>>
>>> Few questions:
>>>
>>> 1) is it possible to use Java with JNI wrapper to communicate with my
>>> backend or does it have to be pure C/C++ ?
>> As far as I know, a Samba VFS must be a shared library.  i.e. a .so
>> file, so I don't think Java would work (although I don't know much
>> about JNI.)
>>
>>> 2) are the notifications (file deletion/creation etc.) available in VFS,
>>> in
>>> other words notifying samba server and applications that directory has
>>> changed?
>>>
>>> 3) are there any examples/tutorials/walkthroughs or up to date
>>> documentation for VFS?
>> Try this:
>>
>> http://www.samba.org/~sharpe/The-Samba-VFS.pdf
>>
>> --
>> Michael Wood <[hidden email]<http://user/SendEmail.jtp?type=node&node=4634960&i=1>>
>>
>> --
>> To unsubscribe from this list go to the following URL and read the
>> instructions:  https://lists.samba.org/mailman/options/samba
>>
>>
>> ------------------------------
>>   If you reply to this email, your message will be added to the discussion
>> below:
>> http://samba.2283325.n4.nabble.com/Custom-VFS-tp4634738p4634960.html
>>   To unsubscribe from Custom VFS, click here<http://samba.2283325.n4.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=4634738&code=aXdhbi5kYW5pZWxAZ21haWwuY29tfDQ2MzQ3Mzh8LTYxNzE4NjQwNw==>
>> .
>> NAML<http://samba.2283325.n4.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>>
>
>
>
> --
> View this message in context: http://samba.2283325.n4.nabble.com/Custom-VFS-tp4634738p4634963.html
> Sent from the Samba - General mailing list archive at Nabble.com.

-- 
Andrew Scherpbier
andrew at scherpbier.org