MIDLC IDL Compiler

[Andrew Tridgell]

Michael,

 > However, as you anticipated there is a problem with the different I/O
 > strategies. PIDL stubs read/write data directly from/to the named pipe
 > in leaf operations.

no it doesn't!

The generated NDR layer is completely independent of the RPC code. In
fact, we use the generated NDR functions to also write to databases
and filesystem xattrs. 

You're probably confusing the ndr_XXX_c.c files, which are the
generated client calls, which is quite separate from the ndr_XXX.c
marshalling/unmarshalling routines.

 > MIDLC stubs currently expect to be provided with the
 > entire raw NDR data buffer and a buffer into which to write the raw NDR
 > response [1]

pidl generates code that allocates the NDR buffers on request for
marshalling. See librpc/ndr/ndr.c and librpc/ndr/ndr_basic.c for the
ndr library routines we use to do the basic work. 

 > First, the problem with the PIDL stubs. Consider the following IDL:
 > 
 >     typedef struct {
 >         [length_is(len)] char buf[BIGMAX];
 >         int len;
 >     } foo0;

yes, that works fine.

 >     int SomeOp([in,out,size_is(*len)] char *buf, [in,out] int *len);

that also works fine

 > I don't know exactly how PIDL would handle these but I think I know in
 > advance that it just doesn't.

it does handle them. 

 > The point here is that to decode this we must temporarily skip the array
 > to get to 'len' and *then go back* to decode the elements.

yes, pidl generates code that is either 1, 2 or 3 passes over the
structure depending on the IDL. Very simple IDL can be done in 1
pass. Most IDL takes 2 passes. Some takes 3.

 > To the best of my knowledge this is the only way to do it. That
 > means that when reading data you might need to skip stuff and then
 > go back. AFAICT this is not practical with the current approach in
 > Samba.

it handles it fine. Look for the ndr_token_*() code and things like
ndr_check_array_size().

 > Any thoughts? Problems?

read/test the existing generated pidl code in more detail before
trying to replace it. There are a _lot_ of subtleties in generating
code from IDL that I suspect you haven't come across yet.

I suspect the easiest way for us to use your compiler would be if you
also use the same librpc/ndr/ndr.c and librpc/ndr/ndr_basic.c NDR base
library. It's pretty well thought out, and it would keep your
generated code compatible with ours.

 > One possible solution to remedy this problem would be to simply reuse the
 > same buffer for all marshalling input and output and resize the buffer
 > to be larger on demand.

yikes! no way. This would interact horribly with the sign/seal code,
and the async nature of our rpc libraries. A single smbd daemon can be
processing hundreds of rpc queries simultaneously, some of which are
being fed into the input side, some being de-fragmented, some being
worked on as complete PDUs, and some being fed out as fragmented
replies. We've gone to a lot of effort to get all this right.

Cheers, Tridge