CRLF / LF conversion, all over again

[John Malmberg]

Ok, I finally got a chance to look up your references.

Timothee Besset wrote:
> I'm new to the list, so please bear with me .. I think this discussion
> belongs to the samba technical though. Some URLs to freshen things:
> 
> [1] http://de.samba.org/samba/ftp/docs/textdocs/CRLF-LF-Conversions.txt

This pretty much explains things, but the explaination is incomplete.
If you see my other post in a thread related to this one, it fills in 
the other missing details.

It also indicates that if you want to do anything, you are on your own 
to implement and test.

With open source if you want a change:

1. Do it your self
2. Pay someone to do it for you.
3. Ask and hope someone will.

If you are going to start on this, you might want to find the version of 
that explaination file in the HEAD source of SAMBA, update it with the 
missing information, and submit it.


> [2] http://marc.theaimsgroup.com/?l=samba&m=94907661120899&w=2
> [3] http://marc.theaimsgroup.com/?l=samba&m=98045359626997&w=2
> [4] http://www.tigris.org/files/documents/15/48/svn-design.html
>     (Text vs binary issues)
> 
> my problem is very similar to what's described in [2]

With the current SAMBA, the VFS is the way to implement this as you have 
been told.  You now know that after you identify the criterion as to 
convert a file or not, you have to deal with random access.  If you can 
find a way to do this with out killing the CPU, you can post it on the 
SAMBA-VMS at LISTS.SAMBA.ORG.

Even if there is no interest elsewhere, it is a real problem that can 
not be avoided.  Only on OpenVMS, it is not just a matter of replacing 
CR-LFs, there are many other ways that a text file can be represented.
The operating system in most cases will do the translation, but not when 
random access is involved.

And if the normal mode of a text file on OpenVMS does not show up as a 
text file on a share, the inconvenience is far worse than what you are 
encountering.

I am currently concentrating on RSYNC, and will have the same issue 
there.  I have not yet determined the best way to solve it.  But 
personally I will not use duplicate files or special directories.  That 
is just asking for a maintenance nightmare.

The issue with CVS is different, as it usually knows that it is dealing 
with text files, and it is not doing random access updates.  So 
converting as files are going in and out to a host friendly format is 
not a big deal.

> I want to work on a share that has a mix of binary files and text files. I
> don't want to manipulate LF on win32 and CRLF on unix occasionaly, because
> it is a half-assed solution. I want to have text files in the right line
> ending format wherever I go.

Again, most Microsoft Windows applications that deal with text files can 
automatically deal with CR or CR/LF.

While you are inconvenienced, the number of Windows applications that 
care is dropping.  If you have the source to the LINUX applications, you 
can change it to be neutral also.  The maintainer of that package may 
appreciate the work.

> My proposal would be something like this: 
> 
> Each shared directory would hold a .crlf/ directory

You can do this, but it would be very inefficient and is intruding on 
users file name and directory selection.

And reserving filenames or directory names is not something that an 
application like SAMBA can arbitrarily do.  All directory and control 
filenames need to be site selectable.  The same with UIDs, usernames, 
and group names.

Your own private VFS can be better taylored, and others may find it useful.

> Ok, so hopefully you can tell me wether this is an important issue for
> Samba, wether it is being worked on.

Your first URL reference answered that part of the question.  It is not 
being worked on in the UNIX versions.

>how hard it would be to do etc

Only one of the stumbling blocks is mentioned in that reference.  The 
smaller one.  The bigger one is the random access updates.
It is very hard to do and get it correct.  But I have no choice but to try.

I have it mostly working for files of less than 64K in size.  After that 
  I hit the point where the Microsoft client starts transfering it out 
of order.  I also have issues where sometimes Notepad writes the file 
back into the original instead of deleting the file and creating a new one.

-John
wb8tyw at qsl.network
Personal Opinion Only

Those that do not want to see ocasional discussions of CR-LF to LF 
conversions on this mailing list have an incentive to submit an update 
to the documentation covering these issues better.

Samba exists on more platforms than UNIX.  And while the main Samba team 
has their hands full working on the platforms that they know, and can 
not be expected to track the non-UNIX issues, I feel that it is on topic 
to discuss cross platform SAMBA issues here.  And this is one.


Timothee did take the inititive into doing some research on the subject 
before posting his proposal.  That is a lot more than a lot of people do.