[Samba] Re: Looking for a set of definitive answers (long)

[Avery Payne]

On Wed, 21 May 2008 12:33:34 -0700, Jeremy Allison wrote:

> On Wed, May 21, 2008 at 06:47:52PM +0000, Avery Payne wrote:
>> Question:
>> 
>> We recently moved to a Samba-based file server, which holds mission-
>> critical data on it (.dbf files used by our Accounting software, etc.)
>> [big snip]

>> But along the way, we encountered some oddities,
>> and I have some remaining questions.
>> 
>> First, the oddities (long-time Samba devs and admins, take this with a
>> grain of salt, when I say oddity I mean it from the perspective of an
>> experienced Windows administrator):
> 
> Great post, thanks for writing it !
> 
> I always appreciate it when users come and tell us about their
> experiences, and where we can improve.
> 
> Now onto the specifics:
> 
>> - File permissions do not behave as expected (from the viewpoint of
>> other staff working with the server).
> 
> Yes, ACLs are just different between UNIX & Windows. We map Windows ACLs
> onto POSIX as best as we can, but the mapping is not perfect. The goal
> is to make the two common cases : "these groups and user fred have
> access", and "these groups but *not* user fred have access" as intuitive
> as possible.
> 
> For 3.3 we're planning to overlay a Windows ACL model that will allow
> perfect Windows ACL restrictions to be added to Samba, but not perfect
> Windows ACL allowances (ie. we'll store the Windows ACLs and use them to
> restrict access early on access denied returns, but still map down to
> POSIX to allow the underlying file permissions to take effect).
> 
> Hopefully this might help you.

I think it will. :)

> 
>> - To oplock or not to oplock: that is the question
>> 
>> [snip]
> 
> Ok, I believe we are *identical* w.r.t. Windows as far as oplocks go. If
> the vendor says disable oplocks with Windows, disable them with Samba
> also. If not, leave them in place.

I was in a hurry to write all of this (as I am always pressed for time) 
but what I was trying to convey is that the documentation could probably 
be a bit clearer on this.  Yes, I will be happy to contribute some 
documentation to this specific issue. :)
> 
>> - Office file locking workaround(s) were not immediately obvious
>> 
>> Buried in the nice (but large) Official Samba Reference and HOWTO is a
>> fix for sharing Word and Excel files through Samba, which involves
>> using the sticky bit for group permissions. [snip]
> 
> Can you point that out to me. 

Sure. "The Official Samba-3 HOWTO and Reference Guide", Second Edition, 
(c) 2006 John H. Terpstra, printed by Prentice-Hall, Professional 
Technical Reference.  Turn to page 264, last 4 paragraphs on the page 
(including 1 inset caption).  Heading is 15.6.3, "MS Word with Samba 
Changes Owner of File".  The description of what to do provided a 
significant fix for us.

> We've done more work on ACL compatibility
> with 3.0.28a and I believe that fix may not now be needed.

The vendor's version is samba-3.0.25b-1.el5_1.4.  I can't mention the 
vendor but I think you could probably guess it by looking at that version 
number. ;)

>> - What?  You want me to unlock that file?
>> 
>> We have had recurring instances where a workstation on the network has
>> seized a DBF file and held onto it, not allowing any other workstation
>> or server to perform writes to the file. [snip]
> 
> Sounds like a bug to me. Not sure where, client app or Samba. Need more
> info on this.

The application is ACCPAC Pro Series 7.2 (now relabeled Sage Pro 
Series).  It is written in Visual FoxPro 8, and uses VFP DBF, CDX, DBC, 
VCX, etc. files, almost all of which are really just mutant dBase tables, 
so any record or file locking semantics that apply to DBF files will 
apply to those files as well.

I used to see this behavior years ago with Win95 workstations talking 
with a Win2k file server.  The issue disappeared when we migrated the 
workstations to Win2k.  It might not be so much a Samba bug as it is an 
issue with the settings for the workstation redirectors.

> 
>> - Speaking of which - just WHO does have that file lock?
>> 
>> For some reason, using the computer management tool in a windows
>> workstation shows stale information. [snip]
> 
> This seems like a Samba dificiency with that tool. You should be able to
> get that info by running smbstatus on the Samba box.

There's the rub.  The existing staff expect to use the in-place GUI 
toolset and have no interest in learning command line tools (including 
the department head).  Yeah, I know, nothing you can do about that 
directly, but indirectly, it might be worth looking into making minor 
tweaks to function with the Computer Management snap-in.

You can test this easily by openning several files (Excel, Word, Access, 
doesn't matter) at several workstations, start making changes but *don't 
save them*...let the workstations and server sit for a bit...then open 
the Comp. Mgmt. snap-in, right-click on the top of tree, select "Connect 
to another computer...", place the server name in the dialog that appears 
and click on [OK].  Navigate to Computer Management -> System Tools -> 
Shared Folders -> Open Files.  You'll see a list of files that are 
"open".  Note that after a long period of time - say, a few days - that 
the file you openned *doesn't appear in the list* after a refresh (press 
F5 to refresh).  Nothing wrong with that, as I understand that the client 
or server may close the connection - but if you do this same operation on 
a "stock" Windows box, the file handle is persisent for as long as you 
have the file open and both the server and workstation running.  In short 
- the file semantics being reported are different between a Samba server 
and a Windows server when using this tool.  If this is a design decision, 
you might want to have a small blurb in the docs someplace about how 
those semantics differ, so administrators don't feel like they're going 
insane when they can't figure out why a file is still in use but it isn't 
reported by the snap-in.

Another way to look at this is to open a shell and compare the output of 
lsof against smbstatus - you'll see that, over time, there are occasional 
discrepencies between the two as far as what files are open, etc.

Using smbstatus was (unfortunately) not effective for us - it showed 
different results from the snap-in I mentioned above, but like the snap-
in, it would have discrepencies between what lsof reports and what it 
reports.  We would also see "no one has that file open" but when we try 
to obtain an exclusive lock on the DBF, it would report that it is in use 
by someone (and indeed, it was).  The final solution in locating a locked 
file was to place this in a shell script named 'find-lock':

lsof | grep 'vp' | grep 'u[wW]  '

As crude as it is, it works.  The "vp" refers to a fixed pathname that is 
unique to our installation, so we can narrow down the number of files to 
just those files that may contain DBF data.  The grep looks for write 
locks, either at a table (file) level or a record level.  It has been 
100% accurate in the 8 times that we have had to use it.

> 
>> - You sure you still have that file open?  It says you don't even have
>> it!
>> 
>> The computer management tool also has an issue with data appearing to
>> be stale.  [snip]
> 
> Same as above. smbstatus should give you this info.

Addressed above.

> 
>> Now, the remaining question(s):
>> 
>> - The vendor initially set up our authentication via tdb files and
>> Winbind. [snip]
>> So who's correct, the
>> vendor's original setup which uses tdb files, or the 2nd vendor
>> response which says don't use them, or should we be on LDAP
>> authentication connected to our Win2k3 domain controllers?
> 
> It is safe to use tdbbackup on a live system. The vendor is mistaken.

THANK YOU THANK YOU THANK YOU THANK YOU THANK YOU....

That single reply was worth its weight in gold.  I can now sleep a little 
better at night.

>> - Is there a way to get the Computer Mangement tool to not "lie" to us
>> about the state of file handles and locks? [snip]
> 
> Needs more work in Samba on supporting the Computer management tool.
> Currently smbstatus works better for this.

Also addressed above.

> 
>> A parting thought as well:
>> 
>> It would have been nice to have had a reasonably generic template or
>> example somewhere that pointed this out... [snipped long frustration]
> 
> Yep, this is true. We depend on the Linux vendors to do this as we don't
> have the resources (or more accurately haven't spent the resources we
> have) on doing this. Sometimes they do a good job, sometimes not. This
> is why server appliance vendors often are a better choice than a full
> blown Linux solution, as they sell a pre-set up solution.

Long story short, it wasn't on the radar due to other issues - but I'll 
keep that in mind. :)

> 
> Thanks for your thoughts !

Thanks for the reply!