NamedPipe DCE/RPC struct split

[Luke Kenneth Casson Leighton]

> Of course. The idea is to split off the named pipe handling
> so that support for any specific pipe is loaded dynamicaly
> at runtime. This includes the current "built-in" pipes that
> Samba handles.

great.  that's what the patch i sent in begins to do: make
it easier for "builtin" dce/rpc services to be split / loaded
dynamically.

> The way this is intended to work is that smbd will scan a directory
> containing named pipe .so's and attempt to load them in and
> initialise them in turn. Each .so will get the opportunity
> to register a name, which will be the name of the pipe it will
> handle.
 
 Wez Furlong wrote some code that does _exactly_ this,
 in freedce.

 the attempt-to-load is by doing a dlsym() on a function
 named "rpc__module_init", which is void rpc__module_init(void),
 if it exists, call it.

 the function is expected to know exactly what to do, has
 access to all global variables it thinks it may need, etc etc,
 if in fact it _does_ need them, etc. etc.

> Then incoming data willbe marshalled by the RPC code in smbd
> until a complete RPC request is received, then this will be
> passed to the relevent .so. A full RPC reply will then be
> handed back to the RPC code in smbd for return to the client.
 
 well... okay, are you referring to PDUs, here?

 are you referring to marshalling as in actual _decoding_
 of the NDR data _in_ the PDU, as performed for request
 PDUs and response PDUs specifically and exclusively?

 or do you actually mean buffering, rather than marshalling?

 i'm going to assume that you mean buffering, rather than
 marshalling, because marshalling doesn't make sense, here,
 in this context.

 in either case, i think that you will find that neither 
 marshalling nor buffering is strictly necessary, as is
 made very clear from the patch i submitted.

 the things you _do_ need to know, however, i will
 explain here.
 
 the first is that a TransactNamedPipes function call is a sort
 of "botched" combination of [from a client-side viewpoint]:

 - a guaranteed up-to-a-fixed-buffer-size write()
 [as emulated by util_sock.c's write_socket()]

 followed by

 - a guaranteed up-to-a-fixed-buffer-size read()
 [as emulated by util_sock.c's read_socket()]

 followed by

 - a select() with a timeout of zero, and if
 there is data outstanding, an error code
 ERROR_MORE_DATA is returned (or NT status
 code equivalent).

 [exactly why microsoft chose this approach, i _do_ not know.
 anyway, leaving that aside, and moving swiftly on...]

repetition of information derived and known:

- in microsoft's MSDN description of the NamedPipes API,
nowhere do the function descriptions mention, in any
way, shape or form, DCE/RPC.

- in TNG, the smbd server does not even _link_ to any
DCE/RPC server-side.  the smbd server does not perform
any post- or pre-processing of the SMB requests it
receives from an SMB client on the NamedPipe channel:
it merely calls read(), write() and occasionally
select(), at the appropriate points, on a unix domain
proxying socket.

[oh, and it works, and has worked for the last two years,
with no known bug reports relating to this area.]

- in microsoft's copious MSDN description of MSRPC,
there is no reference to special cases of buffering
[or marshalling], as best i can tell, other than mentioning
the existence of "ncacn_np" as a transport syntax available
for use.

implications:

 - buffering in the smbd-side of the NamedPipe API
 is unnecessary.

 - buffering on the DCE/RPC-side of the NamedPipe API -
 the server-side, as implemented in samba exclusively
 in what jeremy refers to as "built-in" pipes, _is_
 necessary, however it is only necessary inasmuch
 as _any_ DCE/RPC implementation needs buffering
 of its NDR data in pre-unmarshalled form when receiving
 request PDUs and post-marshalled form when sending
 response PDUs, as part of removing or adding DCE/RPC
 16-byte "headers" etc. as is expected of any
 DCE/RPC server implementation.

 - no special cases or considerations need be given
 to the fact that a DCE/RPC implementation is using
 NamedPipes as a transport, whether this be via a
 .so "glue" interface, whether that NamedPipe
 transport API be implemented via unix domain
 sockets, shared memory, files, floppy disks, carrier
 pigeons, handwritten notes passed around a classroom,
 or temporary buffers in which individual PDUs
 may be stored prior to decoding request PDUs (inside the
 DCE/RPC server implementation "proper" as described
 above) or sending response PDUs (as created inside 
 the DCE/RPC server implementation "proper" as described
 above).


i'm sorry to be quite so long-winded about this: it's actually
a real simple statement: you don't need buffering, because
smbd's NamedPipe transport need know nothing about DCE/RPC.

that simple statement requires a technical justification,
and i am explaining the details on the assumed basis that
people may not necessarily know _why_ any arbitrary
implementation of NamedPipes need know nothing about DCE/RPC.

so, i'll say it again.

The NamedPipes API need only be the following functions,
where the following functions are exposed by the ".so"
through the NamedPipe "glue" interface, and here are
two examples of said functions, one is derived very
quickly from approximately two hours of careful application
of a scalpel at the right points in SAMBA 3 current CVS,
the other is a cut/paste job from samba tng's existing
code.


internal "built-in" pipes:

pipes_struct *make_internal_rpc_pipe_p(char *pipe_name, 
			      connection_struct *conn, uint16 vuid);

ssize_t transact_internal_pipe(pipes_struct *p, char *wdata, size_t wlen,
				char *rdata, size_t rlen,
				BOOL *is_data_outstanding);
				/* call write_to_pipe, 
				   check data written len == data requested
				   to be written len.  if
				   successful, follow up by return read_from_pipe().

				   that's all there is to this function.
				*/

ssize_t write_to_internal_pipe(pipes_struct *p, char *data, size_t n);

ssize_t read_from_internal_pipe(pipes_struct *p, char *data, size_t n,
				BOOL *is_data_outstanding);

					/* 
					 * NOTE:
					 * when smbd/ipc.c uses transact_internal_pipe,
					 * then the is_data_outstanding argument
					 * may be removed from read_from_internal_pipe.
					 *
					 * not until.
					 */

BOOL close_internal_rpc_pipe_hnd(pipes_struct *p);


external "inter-process" pipes:


/*******************************************************************
 * returns a filedescriptor as connected to the
 * proxy redirector socket.
 *
 * an error of -1 is returned, otherwise filedescriptor is returned.
 * [help, help, what to do if a filedescriptor of 0 is returned?]
 *
 ********************************************************************/
int namedpipe_proxy_open(char *pipe_name, 
			      connection_struct *conn, uint16 vuid);

/*******************************************************************
 * transaction
 ********************************************************************/
ssize_t namedpipe_proxy_transact(int fd, char *data, int len,
		    char *rdata, int rlen, BOOL *pipe_outstanding);

/****************************************************************************
writes data to a pipe.
****************************************************************************/
ssize_t namedpipe_proxy_write(int proxy_fd, char *data, size_t n);

/****************************************************************************
 reads data from a proxy pipe.
 ****************************************************************************/
ssize_t namedpipe_proxy_read(int proxy_fd, char *data, int max_len)

/****************************************************************************
 closes a proxy pipe.
 ****************************************************************************/
BOOL namedpipe_proxy_close(proxy_fd);



> Two things though. This functionality will not make it into
> 2.2.x to keep that codebase stable, this will be a 3.0.x
> feature - which is moving more and more to a "plug-in" style
> of architecture.

fine by me.