MIDLC IDL Compiler

[Andrew Tridgell]

Mike,

 > Actually I have to use offsets too or the realloc is going to really mess up
 > my pointers. So I'm NOT going to use the buf_t thing. I'm going to end up
 > with the same arrangement as your code but with a small change. So far my
 > protos are going to be like:
 > 
 > int enc_TYPE(struct ndr *ndr, TYPE *obj, size_t *off);

this is starting to approach what we use. A big difference is that we
pass an additional flags which is used for NDR_BUFFERS or
NDR_SCALARS. That allows us to use the same encoding routine for a
structure that is in a function and one that is embedded in another
structure. 

NDR_BUFFERS == we are encoding the deferred buffers portion of the packet
NDR_SCALARS == we are encoding the scalars portion of the packet

Then you can combine these. For example, in a function encoder we pass
NDR_BUFFERS|NDR_SCALARS for each element, as functions in NDR encode
both at once for each element. For a sub-structure we first use
NDR_SCALARS on each element, then on the 2nd pass we use NDR_BUFFERS
to encode any embedded pointers.

Another difference is in the error handling. You seem to use -1 as
"something bad happened". But what happened? Did you run out of
memory? Did a array overflow? Was a union case out of bounds? I think
you need more than a -1 return, otherwise you can't map the faults
correctly in the fault PDU (pidl doesn't get this completely right yet
either, but we do have a mechanism for getting it right when someone
goes in and fills in the details).

 > Unique pointers are a salted address' and I have a very simple map for DCE
 > pointers which gives properly incremented referents.

ok, as long as you handle it somehow

 > Right now I handle this in the leaf routines but to be honest I've never
 > even seen NDR with bigendian in it so for all I know it could be totally
 > wrong. 

just run a Samba4 server with "rpc big endian = yes". It will send all
its replies PDUs as big-endian, just like Suns cascade server on
sparc. I'd also highly recommend you run all your test suites in both
big and little endian. It's a quick way of finding lots of types of
errors in your IDL, as it allows you to differentiate between (for
example) two uint16s and a uint32. All MS servers accept both big
endian and little endian, but only generate little endian. They accept
big-endian very slowly, but it does work.

 >     dst = ndr->data + *off; 
 >     if (ndr->bigendian) {
 >         dst[0] = (val >> 24) & 0xFF; 
 >         dst[1] = (val >> 16) & 0xFF; 
 >         dst[2] = (val >> 8) & 0xFF; 
 >         dst[3] = val & 0xFF; 
 >     } else {
 >         dst[0] = val & 0xFF; 
 >         dst[1] = (val >> 8) & 0xFF; 
 >         dst[2] = (val >> 16) & 0xFF; 
 >         dst[3] = (val >> 24) & 0xFF; 
 >     }

For Samba this would need to be changed to use our IVAL/SVAL
macros. The reason is that these macros use inline assembly to make it
fast on platforms that have builtin support for these types (for
example, PPC can do little-endian word pulls in a single
instruction). It would be a significant performance loss to not take
advantage of the CPUs builtin instructions if it has them.

I'm really starting to suspect that MIDLC won't work out for Samba4,
at least at the moment. We just have aims that are too different. We
want lots of extensions to make coding IDL easier, and we want the
generated code to integrate tightly with the rest of Samba.

I'll still be very interested to follow the development of MIDLC, and
would be delighted to discuss IDL compiler strategies with you as you
develop it, but I think the idea that this is going to replace pidl in
the near future is very unlikely.

I would like to encourage you to test against our IDL files though. We
have working IDL and test suites for a huge number of functions. It
would just be silly for you not to be able to interoperate with our
IDL files.

Cheers, Tridge