[Samba] Re: Looking for a set of definitive answers (long)

[John H Terpstra]

On Thursday 22 May 2008 02:48:14 pm Avery Payne wrote:
> On Wed, 21 May 2008 15:31:48 -0500, John H Terpstra wrote:
> > Avery,
> >
> > OK - I'll respond too.  I see Jeremy has beaten me to it.
> >
> > Let me tell you up front, if you want the documentation to be improved
> > the best thing you can do is contribute changes and updates.  Making us
> > aware of docuentation problems is a good start, but please take this a
> > step further - send us your updates and changes.
>
> More than happy to  as soon as I can find the time. :)

Good. Thanks.
 
>
> > One other thing, before I get too far into answer or commenting is this:
> > The Official Samba3 HOWTO and Reference Guide (TOSHARG) is a document
> > (book) that sets out how specific parts of Samba function.  It was never
> > intended to provide a working template or a scripted recipe.
>
> Understood.  I am using it as a tech reference.

Good.

> > I did write the Samba3-ByExample book with the specific objective to
> > provide detailed step-by-step, fully worked, examples of real working
> > networks, did you consult that document at any time?
>
> I didn't even know it existed.  The majority of web queries resulted in
> the online version of TOSHARG being displayed.  Thanks for pointing that
> out, I'll look for it.

Please check it out and let me know what is missing.  Also, since you did not 
know it exists, what should we do to make it more prominent?  Is it worth 
making it more prominent?

You can download it from:
http://www.samba.org/samba/docs/Samba3-ByExample.pdf

> > Are you offering to improve its value and utility by contributing your
> > experiences and recommendations?
>
> Yes, as time permits.

Great. This sort of contribution is necessary.

> ...
> More on that later.  I have some ideas that I think would be worthwhile,
> and I'm willing to type them up.
>
> >> - To oplock or not to oplock: that is the question
> >>
> >> The documentation is not entirely clear about when you should and
> >> shouldn't use oplocks on shared files. [frustrations snipped]
> >
> > I have not seen a single installation where DBF files and MDB file
> > sharing works acceptably with more than 3-5 concurrent users.
>
> In our prior version of the software we had 50 workstations, of which
> half were concurrently active, updating/reading DBF files every day.
> Locking is kept to a minimum, which contributes to a very low latency.
> The general "gut feel" we have here is that the limit of our system is
> probably around 50 concurrent users; on busier days with more than 30
> workstations active, the system started to visibly slow, although
> performance degraded gracefully.  The setup (at the time) was a Win2k
> Server with 50 Win2k Workstations on switched 100BaseT, with gigabit
> going into the server.

DBF files are a special case.  While the client can cache any file, DBF's have 
an internal locking method. A DBF application will set a record lock flag and 
is responsible for resetting it on completion of write.  If the client does 
not flush its cache this info can get lost with the result that the record 
remains locked.  There are some apps that will scan the DBF file for stale 
locks and reset them.  Corruption of DBFs is usually not as big a problem as 
with MDB files.

> > Best advice is do not use file sharing for multiple concurrent access
> > database files.  Instead, use a SQL backend database.
>
> The ERP package in use allows us to do so for a very prohibitive price.
> It would actually be cheaper to employ me for an entire year to develop
> an object wrapper to intercept all calls to the DB API and send them to a
> PostgreSQL or MySQL backend, working full time on this project and doing
> none of my other duties (networking, sysadmin, security, email services,
> data warehouse, end-user support, customization of the accounting
> package, etc.), while struggling to emulate all of the features with only
> the API interface as a document to guide me (there is no source
> available).  Of course, that isn't going to happen. :(
>
> >> - Office file locking workaround(s) were not immediately obvious
> >>
> >> [snip]
> >
> > Where in the book would you prefer to see this documented? What changes
> > would you make to the documentation to make this more prominent and more
> > readily capble of being found?
>
> The largest issue encountered revolved around locking semantics and how
> the system would behave.  This I will fully address later when I write up
> my docs and send them to you.

My purpose in asking is to find a better way to present the info we have so 
that people will more rapidly find the answer.  How can we do that?

> >> - What?  You want me to unlock that file?
> >>
> >> We have had recurring instances where a workstation on the network has
> >> seized a DBF file and held onto it, not allowing any other workstation
> >> or server to perform writes to the file.  This locking issue shows up
> >> in random intervals and always requires that we have the person quit
> >> the program we are using and log back in.  It is not an application
> >> issue that we can determine - the rest of the system continues to
> >> funciton, it just prevents one of our servers (or anyone else) from
> >> locking the file.
> >
> > This is a client-side caching issue. Samba does not know that the file
> > has been released until the client notifies Samba that it has released
> > the file. Windows clients can go down without ever releasing the file.
> > There are Microsoft KB articles on how to disable client-side caching.
> > Should this be more vigorously documented in the HOWTO?
>
> Yes, although I would prefer the term "slightly expanded article". :)
>
> > If so, where should it be documented in TOSHARGs?
>
> The viewpoint comes from a small-to-medium sized LAN installation, using
> shared-file databases (dBase, FoxPro, Paradox, Access).  This may not be
> very common in larger installations, but I can attest (from personal
> experience) that it is frequently used in smaller setups.
>
> I think the real issue is that there is a juncture between file locking
> and file permissions that are critical to having those environments work
> correctly.  A small paragraph about shared-file databases would probably
> go a long ways towards providing clarity to the implementor as to how
> things will work, etc.  Again, I'll try to get something to you later.

I do believe that is in the HOWTO somewhere.  Bear in mind that locking and 
client-side caching are orthogonal issues.

> >> - Speaking of which - just WHO does have that file lock?
> >>
> >> [snip]
> >
> > What tool are you using to explore this?
>
> Computer Management Snap-In.

OK - understood.

>...
>> > You need to set your client 
> > registry correctly to stop client-side caching for MDB and DBF files.  I
> > do believe that is documented somewhere in TOSHARG.
>
> I am using both global and share-level "veto oplocks" settings, with a
> very detailed match string. :)

This does not stop a windows client from doing client-side cacheing.  The only 
way to do that is to change the registry settings on the client.

> >> A parting thought as well:
> >>
> >> It would have been nice to have had a reasonably generic template or
> >> example somewhere that pointed this out [snip]
> >
> > Where would you search for this information? Which chapter? Which book?
> > How should it be documented?  Are you willing to write this up in a
> > usable form?
>
> 1) it should be part of a section on implementing replacement file
> services for Win2k(3) boxen.

Really?  Client-side caching is done by Windows 95 and later, and all Windows 
NT-based systems.  I'm not convinced that title will do.

> 2) undetermined.  I will have to look at that.  I am already almost over
> my time limits on this reply today (forgoing lunch to reply).

Thanks for answering.

> 3) again, undetermined.
> 4) Yes.

Cheers,
John T.