Samba login

[okuyamak at dd.iij4u.or.jp]

Dear Jeremy,

>>>>> "JA" == Jeremy Allison <jeremy at valinux.com> writes:
JA> Ok - but to be honest these calls are not a performance
JA> problem. load_client_codepage() and load_unicode_map()
JA> are done once on startup - I simply don't really care that
JA> much about the efficiency of these functions.

I know. It's "EASY TO FIND" area.

I don't think it's  efficient too, but I do care about 'how things
are being coded, and fixed', and these tells me the tipical.

The very fact that stat()->open() code exist, means someone have
thought that this pattern works. The very fact that stat()->open()
code STILL exist even after some are taken off, means someone who
took it off, didn't check entire code.

As I mentioned before, stat()->open() code does not work.  stat()
does not proove open()'s validness. And once you find that, what you
should do is check the entire code, looking for the same pattern and
replace all of them, so that, someone will not copy that part and
put them into major path again.

# 'DESTROY THEM ALL' is one of the very important way in tuning(^^;)


JA> Andrew's position is that the two things a kernel could
JA> do to help Samba efficiency are (a) provide a readdir()
JA> call that also returns stat information (as we are *always*
JA> going to do a stat once we've done a readdir()) and (b)
JA> a case insensitive filename lookup setting.

readdir() with stat() information, is something that might worth,
but will make readdir() call lot heavyer, for simply looking at
directory is log lighter than locking and checking i-node. So, I
don't recommend doing that.
# If you are facing FAT, this doesn't matter, for directory
# information contains filesize as well. But most of the unix system
# does not.


And for case insensitive filename lookup, is something you should do
on process side, using readdir(). It's nothing more then regular
expression search against list of file/directory names.  You can
always cache list of directories, make case insensitive lookup of
individual path. And you can share those information among smbds
using shared memory. There's no need of stat() for filename
matching.


Well... what I'll wish to have from kernel, is file's time
management with accuracy of nano-second ( pico second will be lot
better ). If we have this, we can always know whether our
file/directory list cache is valid or not from changed time.
milli-second is too rough for now-a-day machines. For this purpose,
I'd like to use stat() ... well fstat() really.


JA> Note that we must do a stat or similar call before doing an
JA> smb open as part of the return flags tell the client if the
JA> file was created as a result of this operation - there is
JA> no way to do that without doing a stat before the open (and
JA> yes I know this is a race condition if done in user space,
JA> there's nothing we can do about that).

?

You can control open() flag to get the same result.  If you could
open file without O_CREAT option, it was there.  If you couldn't,
file wasn't there.  And if you could open the file, fstat() will
give you the filesize.

WHY DO YOU NEED TO CHECK THE EXISTANCE OF FILE, WHEN open() DO CHECK
THOSE FOR YOU??? That's point I can't understand.

And, If I may say, calling stat() and calling open() is nearly equal
against file that does not exist(*1). And if it does, what's matter
about open() succeeding, if you have to open() it anyway.

*1) This is because filename->inode is really what's taking time
   for both open() and stat(). Unless you request exclusive open,
   or if file were not being opened exclusively, time required for
   treating inode itself is small against name lookup.


>> It's "Interface between units" that's causing hard to tune. Not
>> simply individual units. And because of this Interface unclearness,
>> fix to them is being hard, especially when code is changing.
>> 
>> I think we should first face this fact.
JA> Yes, but you also have to remember that a lot of what
JA> Samba has to do is quite complex, simply because it is
JA> emulating a complex system (NT Server) which has many
JA> legacy rules we have to obey.

I don't think what I'm pointing out have something to do with
complex system emulation. 'Complex system' is one thing, 'Messy
system' is something else. And I'm pointing out that Samba is messy.
I said nothing about Complexity of Samba, though I do think samba is
too complex against what it should do, too, but that's something else.

# Managing complexity is something we have to keep on working.
# But to do that, we must first divide complexity from messyness.
# And reduce messy part first. Then re-design for complexity.


JA> I would prefer you run a profiler on
JA> smbd and then let's concentrate on the code paths that
JA> suck up most of the time (that's what I've been doing
JA> with Herb in SGI's lab).

That's what I was doing few month ago. But since Samba continuously
fork(), and child process can't be profiled on Linux, this wasn't
easy way to solve problem.

Now, I'm adding hook to Linux system call entry. It counts up CPU
time ( of both realtime, and CPU time ) used for individual system
call, how many times it's being called, and time used for
interruption while the system call is being treated, plus how many
times interruption occured. And then, I run netbench of 660 sec course.

This tells you lot more about where samba is running. It's major
path, and true heavy points. If you look at result of vmstat while
running samba with high load, you'll find that most of CPU time are
spent in kernel mode, not in user mode. This tells you that it's
"How the system call are being called" which should be optimized.
It's not really how you treat bitmaps(^^;) that will speed up the
samba.

This will tell you that profiling will not give you what you need,
truely. The most heavyest point is outside what you're measuring
with profiler.


Was there any way you can take profiler of even child process?  At
least Linux seems to mask signal used for Profiler. And since so,
can't really find the major path using gprof. If you know any tips,
please let me know. For now I know what's happening outside samba,
I now have to know what's happening inside.


best regards,
---- 
Kenichi Okuyama at Tokyo Research Lab. IBM-Japan, Co.