Samba improvements needed (was: WinXP<->Linux samba server test)

[John E. Malmberg]

In article <1095770041.2360.23.camel at bgpc.dymaxion.ca>,
   BG - Ben Armstrong <BArmstrong at dymaxion.ca> writes:
> On Mon, 2004-09-20 at 18:32 -0400, John E. Malmberg wrote:
>> What version of SAMBA is running on the LINUX system?
>
> Samba 3.0.7 + smbfs 3.0.7.
>
>> Other than that, I am not set up to take advantage of your data at the
>> moment.  I do not have any way of reading the etherreal data.
>
> If it is because you don't have a Linux system, the ethereal data is not
> specific to ethereal or Linux.  A number of utilities can read it, some
> of which run on OpenVMS, and many of which run on Windows.
>
> Tcpdump, which is available for TCPIP 5.4, can read it.  For example:
>
> $ tcpdump=="$sys$system:tcpip$tcpdump"
> $ tcpdump -vr test.cap

I do not have a LINUX system running at this time.  Three years in my
new house, and I have not had time to organize that section of the basement
to get one of now ancient 100Mhz/90 Mhz sytems wired up to the LAN
let alone running LINUX.

> Also, tcptrace, which, like tcpdump, is based on libpcap, can read it.
> See:
>
> http://jarok.cs.ohiou.edu/software/tcptrace/download.html

That may be useful, once I get 2.2.11 or 3.0.7 or the SAMBA4 to build
and run, if the problem still remains.

> I find it rather interesting that Linux can negotiate Write AndX to
> write large buffers at a time to OpenVMS, and Windows can negotiate
> Write AndX to write large buffers at a time to Linux, but Windows can
> only negotiate Write to write 1K blocks at a time to OpenVMS.

It may be a difference between Samba 2.2.8 and later versions.

> Also, if pre-allocation is an expensive operation in OpenVMS, doesn't
> that indicate there is room for improvement here in the current Samba
> implementation for OpenVMS?

That is very much an understatement.  Volunteers are needed for an almost
unlimited number of tasks, some big, some small.

And it is not preallocation that SAMBA is doing as noted below.

>  Since the Windows host insists on
> preallocating, and since OpenVMS Samba refuses (or is unable) to send
> back EOF and allocation responses to the file info requests, it appears
> that Windows thinks it can only pre-allocate a small amount ahead of
> where it is writing, instead of further ahead, as it does in the Windows
> -> Linux case.  Wouldn't sending back EOF and allocation figures allow
> the Windows client to write further ahead, resulting in fewer
> preallocation requests?  As it stands, after the first 64K are written,
> one file info + one extra preallocation write are done per 1024 byte
> block written!  That is exorbitantly expensive, and totally unnecessary,
> if only the server would give the client the info necessary to make
> larger, and therefore fewer preallocation writes.

The client does tell the SAMBA server how big to make the file, but not
when VMS can do it efficiently.

First it has the server open the file for write access, and then it uses
ftruncate() or other means to extend it.  Most of these move the highwater
mark of the file, unlike just allocating space.  And that means that the
now empty file must be totally written to disk.

>From looking at the current structure of the UNIX SAMBA code, it looks like
the way to improve performance is to write VFS modules specific to ODS-2
and ODS5.  The ODS5 module would not need any filename mangling.

In that module, when a open +write access is done to create a new file,
the VFS would delay actually opening the file until either the first
data is actualy written, or the client (as per usual observed practice),
sends down the actual size of the file.

More buffering may also help.  The VFS approach may allow more efficient
application cache management and tuning.

As it is, I am trying to do what I can.  J.Y.C. Has done a tremendous amount
of work with the 2.2.8 port, and I am trying to understand what changes that
he made and why, and if any of the issues that I posted to this list before
his port appeared have been resolved.

At this point I am trying to optimize the 2.2.8 port for the new features
found in OpenVMS 8.2 EFT, while still making it build and work on older
versions as a secondary feature.  Then I will look at doing a quick merge
of the 2.2.11 Samba version.

I am also trying to get involved with the SAMBA4 project, as they seem to have
some buildable code.  That way I may be able to find a way to get some changes
in the code that can help VMS performance, especially if I can show them to
help UNIX performance to.

I have not yet been able to scope out a complete TODO list.

At a minimum, current versions of VMS include LDAP and Kerberos, but the
VMS Samba builds do not use them.

SSL is also available as an option to VMS.

The documentation package DOCBOOK?, rsync, subversion, need ports.  I am
working on rsync also, and have a almost functional client.

CUPS and SYSLOG replacement libraries would probably help.

Test suites need to be written, especially for the wrapper functions.  The
setuser.mar needs to be replaced with Persona Services for the applicable
versions of VMS.

-John
wb8tyw at qsl.net
Personal Opinion Only