[Samba] Looking for a set of definitive answers (long)

[Avery Payne]

Question:

We recently moved to a Samba-based file server, which holds mission-
critical data on it (.dbf files used by our Accounting software, etc.)  
The goal was to create a file server that had excellent performance while 
providing Volume Management, but we felt that something like Veritas was 
overkill for our needs.  

Design Goals:
- Redundant Hardware
- Manual Failover (this was an acceptable solution)
- Very large storage capacity (minimum 1 Terabyte)
- Better than 100Mbyte/sec throughput
- Volume Management, Journaled Filesystem
- Drop-In Replacement for aging Win2k file server
- Use existing admin tools to avoid retraining


The proposed solution was a Samba file server running on a pair of 
redundant servers, with one connected to an eSATA raid box, with LVM and 
Ext3 providing volume management and journaling.  Our transition was a 
bit rough, but in the end it has been very stable and fast.  We have been 
really pleased with the performance of the hardware/software combo, 
seeing sustained throughput of about 250Mbyte/sec with peaks as high as 
300Mbyte/sec.  But along the way, we encountered some oddities, and I 
have some remaining questions.

First, the oddities (long-time Samba devs and admins, take this with a 
grain of salt, when I say oddity I mean it from the perspective of an 
experienced Windows administrator):

- File permissions do not behave as expected (from the viewpoint of other 
staff working with the server).

The *nix permission bits cause a user, group, and "Everyone" entry to 
become permanent and persistent.  There was some initial grousing over 
this fact as our long-time Windows admin scratched his head over why he 
couldn't remove these entries as he saw fit.  After explaining that there 
would always be three settings no matter what, that they could never be 
deleted, and that they represented actual filesystem-level bits that 
wouldn't go away, it was accepted.  I didn't notice if this was in the 
docs or not, but I certainly didn't find it.  It also meant enabling ACLs 
on all of the filesystems and doing some creative thinking with the 
permissions.  The closest I could do was to map all files as owner root, 
group set to Domain Admins, and Everyone set to disallowed; members of 
the IT staff would be mapped with the "admin users" parameter; from 
there, any additional permissions would be mapped via ACLs.  We've found 
that this method has the closest behavior to a "real" Windows server and 
has satisfied everyone.

- Permissions don't propigate through the filesystem.

On a Real Windows Box(tm) you would be able to set permissions at the 
parent level of a directory and have them show up for each child object.  
Because the filesystem semantics are not the same in *nix-land, you need 
to go into the directly and manually propigate the permissions, or if 
you're stuck trying to administer permissions through a windows session 
(like the other IT staffers in my department), using the Advanced setting 
to force-reset all permissions on all child objects.  This has also 
caused a bit of grousing as we have several nested directories with a 
heiarchy of permissions; getting one parent directory wrong means 
rebuilding permissions for several child directories as well.  I have 
never been able to get a satisfactory answer as to how to resolve this 
issue, other than the process I described above (which I had to resolve 
for myself without documentation).

- To oplock or not to oplock: that is the question

The documentation is not entirely clear about when you should and 
shouldn't use oplocks on shared files.  It would have been much simplier 
(IMHO) to simply say "use your best judgement, BUT if you are using 
shared data files like Access or Excel or DBF's, you will want to disable 
them or you'll have problems!".  Yes those words show up on newsgroups, 
but it should also show up in the documentation clearly.

- Office file locking workaround(s) were not immediately obvious

Buried in the nice (but large) Official Samba Reference and HOWTO is a 
fix for sharing Word and Excel files through Samba, which involves using 
the sticky bit for group permissions.  While the fix was adequate and 
works well, it should have been I think a little more prominently 
displayed in the documentation.

- What?  You want me to unlock that file?

We have had recurring instances where a workstation on the network has 
seized a DBF file and held onto it, not allowing any other workstation or 
server to perform writes to the file.  This locking issue shows up in 
random intervals and always requires that we have the person quit the 
program we are using and log back in.  It is not an application issue 
that we can determine - the rest of the system continues to funciton, it 
just prevents one of our servers (or anyone else) from locking the file.

- Speaking of which - just WHO does have that file lock?

For some reason, using the computer management tool in a windows 
workstation shows stale information.  In our past arrangement, we were 
able to determine who would have the locked file by simply connecting the 
tool to the server, and sorting on the number of locks present; the tool 
would show the data file with a lock count greater than zero.  Apparently 
this doesn't fly when connecting to the Samba server - it shows files 
open, but the lock count is for ALL locks (including reads) and not just 
write locks.

- You sure you still have that file open?  It says you don't even have it!

The computer management tool also has an issue with data appearing to be 
stale.  Workstations that have been powered down still show a file open.  
Or in some cases, the workstation is working with the file, but no file 
handle appears in the tool!  This was (and still is) a major issue for 
our staff, and as a result of this they have learned not to trust this 
once-reliable tool, because from their point of view, it lies to them.  I 
have had to come up with some work-arounds and while they do work they 
are suboptimal in their eyes.

Now, the remaining question(s):

- The vendor initially set up our authentication via tdb files and 
Winbind.  We have been using this combination succesfully for some time, 
but in the Official Samba Guide it talks about regular maintenance of the 
tdb files via tdbbackup.  The department head has asked that I find the 
definitive answer on how to do this, as we cannot afford more than a few 
minutes of scheduled downtime.  The vendor's response was that tdb files 
should not be used because they can be corrupted when applying tdbbackup 
to them (despite the fact that it was the vendor that set us up to use 
them to begin with - go figure).  This has caused even more concern - 
millions of dollars in business and 50+ users are supported by this 
server, running 24/7/365.  So, if we were to loose our file server 
tomorrow, and had to activate the backup server (which we would do by 
plugging in the eSATA array into the new units and starting up the 
system), how could we guarantee that the GUIDs, etc. would be consistent 
and we wouldn't have a complete mess on our hands?  I have seen someone 
else recently mention that they should be using an LDAP authentication 
backend.  So who's correct, the vendor's original setup which uses tdb 
files, or the 2nd vendor response which says don't use them, or should we 
be on LDAP authentication connected to our Win2k3 domain controllers?

- Is there a way to get the Computer Mangement tool to not "lie" to us 
about the state of file handles and locks?  It would be a godsend to not 
have to watch everyone roll their eyes because the "expected" way of 
locating file issues simply doesn't work for them.  This is an ongoing 
issue and I can't really provide a satisfactory answer - even SWAT 
appears to have this issue as well!  I have resorted to shell scripting 
to provide us with the answers we need but this is hardly a long-term 
solution.  Is there some magic setting I need to flip in the registry for 
our Windows XP clients to make this issue just go away?  Is it related to 
protocol changes in the newer versions of the Windows redirectors?  Just 
why does this happen?

A parting thought as well:

It would have been nice to have had a reasonably generic template or 
example somewhere that pointed this out, instead of wasting an incredible 
amount of time (a month!) reading many many many pages of documentation, 
sometimes scattered into different chapters, as well as contacting the 
vendor twice with over a half-dozen messages.  I would think that a 
simple, single smb.conf file named "smb.conf.dom-member.filesserver" with 
a generic looking setup would have resolved many of these issues, by 
having the appropriate settings and comments already in the file while 
pointing out what parts of the template needed to be changed.  Everyone 
that I have talked to outside of my work looks at Samba as a drop-in 
replacement for existing Win2k(3) file servers, and a template with 
settings that come closest to emulating that same behavior would go a 
long ways towards adoption.  I'm not trying to be overly critical, but 
rather, I'm trying to point out a missing part of the software package 
that I think administrators would like to have, especially as more and 
more companies start to operate in mixed environments.