MIDLC IDL Compiler

Andrew Tridgell tridge at osdl.org
Sun Jan 16 03:54:26 GMT 2005


Mike,

 > Arrays attributed with [string] are treated specially by MIDLC as they are
 > with MIDL. They *are* considered to be strings. When a [string] object
 > is encountered it is passed to the {enc,dec}_ndr_string function to be
 > converted between the server's encoding (e.g. UTF-16LE) and the locale
 > dependant character encoding (e.g. UTF-8) of the calling process. I
 > believe this is consistent with MIDL.

nope. [string] in MIDL means "array terminated by a zero element". You
can use [string] on _any_ type, not just uint16. MIDL does no
character conversion (see the NDR spec for full details).

Even if you assumed that [string] in MIDL was only ever used for
uint16 arrays and was always convertable to char*, then you still
wouldn't have enough information to get it right, as you also need to
encode the termination type. Microsoft neglected to encode this in the
IDL, but if you don't then you are going to hit major problems. Many
Microsoft interfaces only work if you use _exactly_ the same
termination as the Microsoft code does. So, if you send a string null
terminated when the server doesn't expect it to be then you may very
well find that the call fails with a mysterious error.

Putting this knowledge into every piece of code that calls the API is
just amazingly silly (thats what Microsoft does). The only sane thing
to do is to encode this knowledge into the IDL, and that means an
extension.

 > If the object in question is NOT really a string then the user would
 > just have to decode it as a [size_is(size),length_is(length)] array
 > explicitly and NOT use [string].

thats different on the wire, so it won't work

 > I'm not saying I'll *never* implement extensions. I just don't
 > want to go diving into all sorts of non-standard stuff right now. Maybe
 > when the code matures and it's been determined that the base stuff is
 > working well I'll go back and think about that stuff (provided it
 > doesn't break MIDL compatibility).

it doesn't have to break MIDL compatibility, but what I've been trying
to do is save you from having to throw out too much when you come to
add the extensions. The extensions are vital, but some of them change
the basic structures in your compiler, so if you add them later you're
just making a lot of work for yourself.

 > On the other hand, there are some things that I will never support. For
 > example, PIDL just ignores the conformance DWORD before a structure with
 > an embedded conformat array.

where do you get these claims about pidl from? We handle (and
validate) the conformance value. Look for NDR_ERR_CONFORMANT_SIZE in
our generated code. It's all over the place.

 > This would require that all Samba code that operates on dom_sid2 be
 > converted together. Not sure if that's a big deal for you but we
 > might as well get this stuff on the table now.

This is another example of where Microsoft goofed up, and we fixed
it. Two different Microsoft programmers encoded the one structure, a
dom_sid, in two different ways, making them incompatible on the wire. 

That means that Microsoft programmers have to deal with translating
these two different structures bewteen each other in their code. They
can't just use generic dom_sid handling functions everywhere.

To fix this we define dom_sid and dom_sid2 as the same structure using
cpp, then use a manual dom_sid2 parser in ndr_sec.c. You are right
that we weren't checking the conformant header in that hand written
function, but I've now fixed that with a 3 line patch. It did not
require a wholesale change to our code.

 > At the moment it's very important (to me) that MIDLC be totally compatible
 > with MIDL wherever possible [1]. I want people to be able to use their
 > existing IDL and knowledge. I'm thinking way beyond Samba or Java or CIFS.

yes, that's fine, but it doesn't preclude adding extensions to make
life easier.

The weird thing about this whole debate is it is the opposite of the
debate I'm also having with Ronnie Sahlberg. Ronnie wants more IDL
extensions for his use of IDL in ethereal, so he has decided to write
his own IDL compiler. You want less extensions, so you are writing a
new IDL compiler. Meanwhile, we've been adding the extensions Ronnie
needs to pidl (see the recent work on enums and bit fields).

What I _really_ can't understand is why you don't spend a little time
at least learning from the mistakes in pidl. Pidl has some major
flaws, but unless you at least spend a couple of days trying to
understand how it works you will just make the same mistakes again.

I spent weeks looking at all the existing IDL compilers I could find
before starting on pidl. I didn't use them, but at least I had good
reasons not to after having actually studied their generated code and
how they work. As far as I can see you've just blindly assumed
everything about pidl is worthless without even taking a look. Why? 

Cheers, Tridge


More information about the samba-technical mailing list