NTVFS vs S3 file server

[Stefan (metze) Metzmacher]

Hi Pavel,

> While reading the big discussion about Samba 4.0 release, I was wandering what 
> exactly are the advantages and disadvantages of NTVFS fileserver when compared 
> to "old" s3 code?
> 
> Just the major points, or preferrably a link to an answer given in the past.

Samba4 started with new 'raw' smb client library which was able to test
every aspect of the smb protocol, which was mostly undocumented at that
time (2003).
Tridge and Jim Myers wrote a lot of torture tests in order to work out the
details of mostly every smb command and their parameters. A Windows 2000
(and later 2003)
Server was used as a reference implementation to test against.

At that time (2003) Samba-3.0.0 was just released.
It had a posix like vfs abstraction, but the layer above that posix
layer was mixing
the parsing of the requests and the logic of the smb semantics. Up to
that point
we mostly tested samba with windows clients and only implemented what we
saw on the
wire, when windows clients talked to windows or samba servers.
As the mapping from smb to posix was not abstracted it was impossible
for vendors,
which special file systems to extend the functionality of samba by just
implementing
a feature like real NTACL support in a vfs module.

With the new torture test allowed us to explore every aspect of the smb
protocol
and the aim was to provide an abstraction layer (ntvfs) with smb
semantics, which would allow
vendors to implement every smb feature they want in an ntvfs module if
they had a filesystem
which supports more than just posix.

As a first step we got an smb server that parsed every request into a
structure
and just called a ntvfs function, without doing any logic or mapping.
The request structures where shared between the raw client library and
the server.
In order to verify that the abstraction layer where correct and didn't
loose information,
the first ntvfs module was the 'cifs' backend. This 'cifs' backend
implemented a proxy module
that just passed the request structures to a remote server.

Then Tridge wrote his "big gun" for smb testing a tool called 'gentest'.
gentest takes
the unc pathes and credentials of two smb servers. It triggers random
requests to both of
them and compares the results, if it hits a difference, it tries the
work out the smallest
sequence of requests to trigger the difference. Pointing gentest to
windows server and
a samba server with the ntvfs_cifs backend (also pointing to a windows
server)
shows how good or bad the abstraction layer works. If it runs for hours
without finding
a difference the abstraction layer is in a good shape.

The next steps were adding a 'simple' ntvfs backend (mostly unused) and
a 'posix' backend
which is currently used in samba4 alpha releases, it does the mapping
from from smb to posix.
The posix seems to work quite good for simple things, but it doesn't
have a flexibility
posix vfs layer.

During the last years the "old" samba3 file server "smbd" has improved a
lot and it also
passes most of the complex torture tests. While the clustering efforts
in samba were started
in samba4, the production cluster code is only available in "smbd"
currently.
The samba3 posix like vfs layer was extended to also support smb like
features,
so that vendors can implement stuff in a better way, than the default
smb to posix mapping.
Samba3 also got SMB2 support in 3.6.x. There's also a lot of other
features which were
backported from samba4 to the samba3 parts, e.g. both use the same
tevent infrastructure
and a common dcerpc binding handle abstraction and a lot of other stuff.
During the same time the focus of samba4 was partly shifted away from
SMB to Active Directory.

The basic ideas of a NTVFS layer are really good and they make the code
a lot easier to
understand and debug, but implementation of the async parts are quite
complex and doesn't use
our generic tevent_req async infrastructure. Currently we're redesigning
the internal data models
in the smbd file server in order to implement SMB2.x features like
durable handles and other complex
stuff. Our current plan includes to add a NTFSA layer (very similar to
the NTVFS layer), but named
after the [MS-FSA] "File System Algorithms" document of the Microsoft
documentation, which
specifies the common file system layer below SMB and SMB2.

I hope that gives you an overview.

metze

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 262 bytes
Desc: OpenPGP digital signature
URL: <http://lists.samba.org/pipermail/samba-technical/attachments/20111208/56a12ab2/attachment.pgp>