CRLF / LF conversion - What you must deal with.

[John E. Malmberg]

Been there, had to do that, and know where all of the landmines are.

Maybe someone can condense this and put it in a SAMBA FAQ.

On Thu, 21 Feb 2002, Ries van twisk wrote:

> Hi All,
> 
> please stop this pointless discussion. Samba well never ever touch your
> file structure, it only SERVES them to a client. There is no change that
> samba will change the internals unless you write a VFS module to do just
> that.

The issue is not pointless when you are running SAMBA on a NON-UNIX but
POSIX compliant platform that has different text file internal formats.

UNIX may not differentate between text files and other file types, but
other operating systems do.

And to support SAMBA running on those platforms, the files must be
converted from the internal text format to LF format when serving the
files.  The reason that they are not converted to CRLF will be explained
in a section below.

The adding of VFS support may allow those platforms to operate more
efficiently than in the past.

The solution to the directory issue is to report the size seen on the
server. Actual experience has shown that nothing actually seems to believe
it, it is just used as an estimate on the file size.

The same is the case when reading the file size from the clients.

The biggest problem with on the fly file conversion is that Microsoft
Windows does not always send the file in sequential order.  It can send
the chunks of the file in random order.

If SAMBA does not store the file in the exact binary form as the Microsoft
system sent it, the file will be corrupt.  This can not be easily fixed
with a VFS or other system.

This will not be noticed in a SAMBA to SAMBA file transfer as SAMBA always
sends the file in order.

So while you can easily convert and send files on the host system to a
Microsoft Windows client, you can not do so on receive.

The situation really becomes bad if the Microsoft Windows Client wants to
do a random access partial update on the file.  It appears that some
common programs on that platform do that.

Now the reason that native host text formats are converted to LF and not
CR/LF on translation, is that a number of Microsoft Windows and DOS based
programs understand LF format, but not CR/LF.  In actual experience the
number of Microsoft Windows programs that can not tolerate LF format is
extremely small compared to the ones that can not tolerate CR/LF format.

The majority of the Microsoft Windows programs that I have encountered
will handle both formats equally well.

There is also no way to reliably tell if the Microsoft Windows program is
sending a text file that should be translated and a document file that
should not be.  Many of the file extensions are the same.  Especially if
you are still using older DOS programs.

This information was discovered while porting SAMBA to OpenVMS, and the
OpenVMS port can not avoid the issue and be a useful file server.
Unfortunately it still does not have the conversion 100% correct yet.

-John
wb8tyw at qsl.network
Personal Opinion Only