MIDLC IDL Compiler

Michael B Allen mba2000 at ioplex.com
Fri Jan 14 02:59:42 GMT 2005


Andrew Tridgell said:
> Michael,
>
>  > However, as you anticipated there is a problem with the different I/O
>  > strategies. PIDL stubs read/write data directly from/to the named pipe
>  > in leaf operations.
>
> no it doesn't!

I thought you said before you read data from the pipe piece-meal rather
than using complete buffers because an RPC could be several MB?

> You're probably confusing the ndr_XXX_c.c files, which are the
> generated client calls, which is quite separate from the ndr_XXX.c
> marshalling/unmarshalling routines.

The code I looked at was the ndr_XXX.c files but once it got down to the
IVAL macros I didn't look further (wasn't really sure where to look
actually).

>  > I don't know exactly how PIDL would handle these but I think I know in
>  > advance that it just doesn't.
>
> it does handle them.

Ok. Sorry for the confusion.

>  > Any thoughts? Problems?
>
> read/test the existing generated pidl code in more detail before
> trying to replace it. There are a _lot_ of subtleties in generating
> code from IDL that I suspect you haven't come across yet.

Just for the record, the reason I chose not to use PIDL in the first place
was because I wanted MIDL compatibility and language/application
independance. So there's no point in reading or testing anything that PIDL
generates because no part of that could be used in a stand alone
alternative.

As for the "subtleties of generating code from IDL", I can't answer. All I
know is that MIDLC stubs handle some relatively complex RPCs so far and
the code that generates it is still pretty clear and small. Of course
there's always that one IDL construct that could prove me wrong.

> I suspect the easiest way for us to use your compiler would be if you
> also use the same librpc/ndr/ndr.c and librpc/ndr/ndr_basic.c NDR base
> library. It's pretty well thought out, and it would keep your
> generated code compatible with ours.

I can't do that Andrew. I'm not doing this just for Samba. Top-level
"glue" is one thing but the leaf routines are generated by some internal
functions that need to be application independent. Maybe when all is said
and done we'll find a common set of leaf-ops that can be abstracted but
for the first pass I want to splice in at a higher level interface where
*everything* below it MIDLC generated.

>  > One possible solution to remedy this problem would be to simply reuse
> the
>  > same buffer for all marshalling input and output and resize the buffer
>  > to be larger on demand.
>
> yikes! no way. This would interact horribly with the sign/seal code,
> and the async nature of our rpc libraries.

How exactly would it interact horribly? If you're not doing I/O in the
marshalling routines, encoding/decoding/signing/sealing is all CPU bound
stuff so breaking it up won't make it any faster. From what I looked at it
seems your async code just splits up the RPC into decoding and encoding
functions so you can basically decode() -> schedule() -> encode(). Is that
right? If so I can generate *separate* decode() and encode() routines.
That's not good enough?

> A single smbd daemon can be
> processing hundreds of rpc queries simultaneously, some of which are
> being fed into the input side, some being de-fragmented, some being
> worked on as complete PDUs, and some being fed out as fragmented
> replies. We've gone to a lot of effort to get all this right.

Pardon me for being ignorant about this stuff. I was really hoping I
didn't have to get into the internals of Samba.

When an RPC is decoded do you a) decode everything because the entire NDR
buffer is available, b) do some kind of iterative decoding (if so for what
reason), c) trap a low-watermark in the marshalling routines and fill a
buffer, or d) ???

I don't see how you cannot decode (or encode) in a single call. Otherwise
if you're breaking it up so you can do I/O in between then you're going to
have the 'foo0' array problem I originally pointed out because you could
not know how big the buffer needs to be to get to 'len' and you can't
discard data to get there without loosing data needed in the "2nd pass".

Mike


More information about the samba-technical mailing list