info regarding dce/rpc runtime library

[Luke Kenneth Casson Leighton]

this is a report tying in previously planned developments for samba's
dce/rpc code which i am now no longer interested in pursuing for samba's
benefit, with observations on the open group's dce/rpc reference
implementation.  it is my hope - but not my wish - that these observations
will be noted for their technical merit.


1) handles.

the server-side implementation of dce/rpc functions _always_ take the
first argument - of every single function - as a handle [that must include
\PIPE\srvsvc].  there is an .acf file (attribute configuration file) that
specifies whether handles are "explicity" in the .idl file - with a tag of
[context_handle] - or whether the idl compiler is to "implicitly" add them
as the first argument to every single dce/rpc function in the .idl file.

why is this done?  the handle argument allows both client and server to
reference state information associated with the connection.  this
includes:

- a per-connection memory system, which in the dce/rpc rtenv library is
sophisticated enough to be thread-safe and it _Also_ has some server-side
functions that allow one thread to do marshalling / unmarshalling whilst
the other thread does the processing, but this server-side function is
called to wait until the processing-thread is finished with any memory
allocated and any stack-based arguments used by the marshalling /
unmarshalling thread.  or, it could be the other way round: i am uncertain
on this point, but you get the idea :) :)

- a means to associate a void* with a handle, to be returned back to the
caller.  this is best illustrated with the following pseudocode call
walk-through:

server-side marshalling stub code:

struct handle_t
{
	rpc_connection_state_info *state;
	RPC_20_BYTE_OVER_WIRE_STRUCTURE remote;
	void *local;
};

server_side_stub_of_some_idl_function()
{
	handle_t h;
	unmarshall_handle_from_wire(&h.remote);
	h.local = reference_remote_handle_to_local_one(h.remote);
	
	some_idl_function(h.local, ...);

	/* marshall arguments */
}

so, there is a level of indirection.  the 20-byte RPC_HANDLE *never* gets
seen by either the client-side or server-side application code: instead,
it sees a void*:

typedef void* some_handle_t;

take a look at the spoolss server-side code.  there is an array which
associates handles with a structure.  the implementation in samba exposes
PRINTER_HANDLE directly up to the server-side code.  the "array" that
associates PRINTER_HANDLE with the structure _should_ be hidden, for
example in a function called prs_handle().

i had been planning for some time to implement prs_handle() in this
fashion.

in this way, the presence, and necessity, of global variables to maintain
_any_ state information can be, and is, removed.


additionally, the idl compiler is responsible for generating a structure -
idlfilename_vN_N_c_ifspec - which contains [yep, you guessed it] a handle
which can be used to maintain the client-side state information.

this structure contains sufficient information to tell the IDL libraries
how to make a connection to a server: for example, it contains the
abstract syntax identifier as one of its members.

the details of this structure are opaque and hidden from prying eyes.


- a per-connection security / credentials information.

this allows a server to determine the security settings negotiated by the
client, potentially allowing the server to reject a connection on the
basis that the security settings are too weak.

it also allows the server to make decisions based on the identity of the
user making the connection.

- per-connection state information provides the means to simplify the code
by hiding the details of how and where to transfer the marshalled /
unmarshalled arguments behind another API layer, with an opaque "handle"
as the only state information transferred between the API and its use.

this in and of itself mandates the requirement that every single dce/rpc
function MUST have one "handle" argument.


2) packed data representation.

in the dce/rpc header, there are 4 bytes that indicate the data
representation types.

the first is currently being used: the byte order.  0 indicates
intel-word-order.  1 indicates the reverse.

the second byte is for floating-point representation.  no dce/rpc
functions in nt domains code have yet seen the use of a floating point
number.

the third byte is for character - char* representation.  0 indicates
ascii.  1 indicates ebdic (agh!!!!).


3) ndr "offsets" as pointers

the dce/rpc 1.1 rtenv has no code that can generate or understand relative
offsets as pointers.

some data structures over-the-wire have been observed to contain offsets,
with the offsets all relative to some specific point in the over-the-wire
data.

EVERY single occurrence where these "offsets" are observed it is also
noted that there is an additional "length" field.

in other words, it is suspected that EVERY single occurrence where
"offsets" are used is in fact a self-contained, independent representation
of a data structure which has been "typecast" to a [void*, int length]
"blob".

in other words, the data structures over-the-wire are completely opaque to
the top-level IDL files in which these structures are used.  this has two
implications:

- that there is a need to separate every single one of these structures
into a two-tier marshalling / unmarshalling architecture.

- that there needs to be an investigation into whether the ndr "packed
data representation" carries over from the DCE/RPC header into the opaque
blobs.  i suspect not, which means that samba's current codebase -
including TNG - will fall over when presented with non-intel-word-order
PDU requests: it will reverse the byte order _unnecessarily_.

i suspect that microsoft's first implementation of these was to simply
dereference the offsets into pointers, byte-swap if necessary and then
typecast the [void*] to the required data structure!!!!!  this relies not
only on the compiler doing some very specific data packing but also on the
target OS being 32-bit.  very bad.

examples of data structures where this is done:

- security descriptors

- almost every single info level in spoolss

- service control manager's ENUM_SRVC_STATUS structure

- the NTLMSSP code used in bind/bind-ack and CAP_EXTENDED_SECURITY.


my suspicions are that microsoft uses [and they in fact document the
methods and advise its use] the rpc_ndr_xxx routines _directly_ to do
this.  having suspected this for quite some time, i use this to good
effect in samba TNG code: see tdb_lookup_user() as an example in
samrd/srv_samr_usr_tdb.c.  this takes a TDB_CONTEXT, looks up a
flat-marshalled-data-stream and unmarshalls it into a SAMR_USER_INFO_21
structure.  tdb_store_user() does the reverse.

the byte order, whilst being intel-word-order, is irrelevant [unless the
tdb files are copied to another server] as the flattened data is only used
locally.

the implications of this two-tier approach are that it is conceivable that
the dce/rpc 1.1 rtenv library - or even just its idl compiler - can be
used unmodified in combination with rpc_parse code: the idl compiler to do
the first layer, using rpc_parse code to deal with the opaque,
self-referencing [void*, length] blobs.


4) error_status_t

the dce/rpc spec mentions that the .acf file can be used to specify that
certain arguments are to have special status, through a tag named
[error_status].

if an argument, which must be a 32-bit integer, has this attribute, then
if there are any communications errors of the underlying dce/rpc
transport, they are placed in the specified argument.

otherwise, any communications errors are NOT reported to the function.

in this way, an application writer can maintain some degree of control
over what happens should a dce/rpc service fail.

at the moment, none of the samba code is capable of detecting error
conditions, such that winbindd's development has required the addition of
some primitive detection for communications failures that are propagated
up to the top level into the winbindd application, and force a total
reestablishment of all connections to the NT services being used.

it is interesting to me to note that the dce/rpc designers considered this
problem, and solved it in what i consider to be a satisfactory solution:
explicit control over which parameters shall return communications error
codes.


that concludes a summary of my observations so far on the dce/rpc 1.1
rtenv reference implementation.  i have been speaking with people here
about it but considered it best to put these observations into words.

lukes