Samba login

[Jeremy Allison]

okuyamak at dd.iij4u.or.jp wrote:
> 
> The very fact that stat()->open() code exist, means someone have
> thought that this pattern works. The very fact that stat()->open()
> code STILL exist even after some are taken off, means someone who
> took it off, didn't check entire code.

True - I only checked the area of the main smb code processing
path, as that is what the profiler showed to be a problem.

> As I mentioned before, stat()->open() code does not work.  stat()
> does not proove open()'s validness. And once you find that, what you
> should do is check the entire code, looking for the same pattern and
> replace all of them, so that, someone will not copy that part and
> put them into major path again.
> 
> # 'DESTROY THEM ALL' is one of the very important way in tuning(^^;)

The stat()/open() pair is inevitible due to the case insensitivity
code. Imagine the following scenario : 

1). On disk is the share relative pathname "An/upperCase/file/paTH"

2). Client sends an SMB open call for the pathname "AN/UPPERCASE/FILE/PATH".
Now the statcache tries to convert this into the mixed case path on disk, it
does this by trying to do a stat() on the (hopefully) cached pathname
"An/upperCase/file/paTH".

3). If this succeeds, then the open is done.

Note that the stat() is not removable, as the conversion between
the case the client sends, and the case as stored on disk depends
upon it. If the pathname "An/upperCase/file/" exists, and not the
final component then you end up with two stats(), as the statcache
code walks back in the path from the end to find the longest match.

Now in a case insensitive filesystem then you can bypass the stat(),
but in standard POSIX such a thing doesn't exist, and Samba is coded
for the common POSIX case.

> readdir() with stat() information, is something that might worth,
> but will make readdir() call lot heavyer, for simply looking at
> directory is log lighter than locking and checking i-node. So, I
> don't recommend doing that.

Don't make it the same call - make it an additional one, only used
by Samba (after all we're imagining a Samba appliance here, not a
generic O/S :-).

> And for case insensitive filename lookup, is something you should do
> on process side, using readdir(). It's nothing more then regular
> expression search against list of file/directory names.  You can
> always cache list of directories, make case insensitive lookup of
> individual path. And you can share those information among smbds
> using shared memory. There's no need of stat() for filename
> matching.

Not true - someone has to do it. I agree the statcache should
be shared between smbds then it will be done less often, but
it will still be done. With the new tdb code this is much easier,
but consider this - a tdb lookup contains an fcntl lock, walking
an mmapped region (with possibly page faults from disk) followed
by an fcntl unlock. Is this quicker than a single stat() call ?
We need to profile this.

> You can control open() flag to get the same result.  If you could
> open file without O_CREAT option, it was there.  If you couldn't,
> file wasn't there.  And if you could open the file, fstat() will
> give you the filesize.

But you already need to do the stat() for the case insensitivity...
 
> WHY DO YOU NEED TO CHECK THE EXISTANCE OF FILE, WHEN open() DO CHECK
> THOSE FOR YOU??? That's point I can't understand.

CASE INSENSITIVITY on POSIX (sorry for shouting, but you were
shouting first :-) :-).

> I don't think what I'm pointing out have something to do with
> complex system emulation. 'Complex system' is one thing, 'Messy
> system' is something else. And I'm pointing out that Samba is messy.
> I said nothing about Complexity of Samba, though I do think samba is
> too complex against what it should do, too, but that's something else.

If you can help us simplify Samba I would be very happy. Current
functionality must keep working though....

> This tells you lot more about where samba is running. It's major
> path, and true heavy points. If you look at result of vmstat while
> running samba with high load, you'll find that most of CPU time are
> spent in kernel mode, not in user mode.

A lot of this is in the TCP stack though - which is as it should be.

> Was there any way you can take profiler of even child process?  At
> least Linux seems to mask signal used for Profiler. And since so,
> can't really find the major path using gprof. If you know any tips,
> please let me know. For now I know what's happening outside samba,
> I now have to know what's happening inside.

I must confess I've been doing this on SGI/Irix at the SGI
labs, so haven't run into this problem.

Comments anyone ?

Jeremy.

-- 
--------------------------------------------------------
Buying an operating system without source is like buying
a self-assembly Space Shuttle with no instructions.
--------------------------------------------------------