FW: RPC restructuring finished (mostly).

Luke Kenneth Casson Leighton lkcl at samba-tng.org
Thu Mar 1 10:10:48 GMT 2001

hi seiichi, nice to see you're around.

On Wed, 28 Feb 2001, Tatsukawa, Seiichi wrote:

> Just FYI. (It's only a background info and has no contribution to Samba.)
> > btw IDL is going to be a bitch for spoolss.  it's a two-level affair,
> > with the data "wrapped" and typecast to a void*, length, info_level.
> > the top-level doesn't even know what the void* _is_.
> >
> > which is a stupid, horrible and wasteful way to do dce/rpc
> > programming.
> Not really.

> Both OSF/DCE IDL compiler and Microsoft MIDL compiler support
> "pickling", which generates a client stub code for
> marshalling/unmarshalling data structure into/from a buffer (could be
> incremental), instead of sending serialized data onto the wire. (It is

seiichi, do you _happen_ to know if the OSF IDL compiler can do
self-relative pickling?  i tried looking at it, and couldn't work out if
it did this or not.

in particular, do you have, or know where i can find, some coding
examples?  it would be _fantastic_ if we could do a SID pickling API, for

doing self-relative pickling in hand-coded marshalling / unmarshalling is
a real pain!

> analogous to the use of xdrmem_create()/xdr_destroy() with other
> xdr_*() filters in ONC/RPC. But, you don't need to write your own
> filter because IDL compiler takes care of it.) The pickling is
> typically used to store marshalled data in a persistent storage, e.g.,
> file, in a portable way (i.e., byte order, alignment, etc., are all
> taken care of). However, it is also often used with RPC.

you're right: i even started using this for storage of SAM database
information in tdb databases.

i noticed that microsoft was using pickling to store SAM data in the
registry, so i mimicked this method.

> OSF/DCE Security system necessitated IDL compiler's pickling support
> because it had to serialize data structure before sign/encrypts
> it. (In the initial release of OSF/DCE without pickling,
> marshalling/unmarshalling of such data structures was done by
> hand-coding it, like the use of xdrmem_create(), which was cumbersome
> for so many complex data structures.) A signed/encrypted data was sent
> in RPC (or in Kerberos ticket) or stored in a file.


> > then, at the second-level, they use internal
> > marshalling/unmarshalling, just like is currently done with security
> > descriptors and SERVICE_INFO.
> This enables Microsoft to augment new info structures (i.e., levels)
> without updating the RPC interface.

*bing* - of course.

okay, i like this method, now :)

despite the fact that their RPC interfaces are used extremely poorly:
there is a way to crash spoolss.exe by telling it that the return buffer
size of one of the RPC functions is over 1mb.
> > btw it is also worthwhile investigating whether receiving data from
> > non-intel-byte-order systems affects these second-level-marshalled
> > structures or not.
> It shouldn't because a serialized data supposedly includes a header
> describing its encoding method. (Well, at least, OSF/DCE IDL compiler
> does that. Otherwise, data portability is at a loss.)

well, is the internal picking told to use the same byte order as
over-the-wire in the header?

... now that i think of it, *duur* - of course, it will use the same
byte-order, the compiler generates simpler code in native-byte-order: it's
the receiver that has to have the [slightly more complex] dynamic

> > remember, dce/rpc can pass memory structures over-the-wire that can be
> > used beyond the scope of one single dce/rpc function call, at both
> > client end and server end.
> It's called "context handle" in DCE/RPC term and heavily used
> throughout samr, lsar and other interfaces. (It's fairly easy to
> detect a client's death over TCP or SMB pipe, well, after a long
> period of timeout. But, it's not so easy to do it reliably over
> UDP. Microsoft had tough time implementing MSRPC over UDP and their
> initial release didn't support the connection-less transports.)


> > so, fragile or not, memory leaks are better than freeing memory and
> > still using it!
> Until a process hits the memory limit...


good to hear from you, seiichi.


 ----- Luke Kenneth Casson Leighton <lkcl at samba-tng.org> -----

"i want a world of dreams, run by near-sighted visionaries"
"good.  that's them sorted out.  now, on _this_ world..."

More information about the samba-technical mailing list