libmsrpc for Samba 3

[tridge at samba.org]

Jerry,

 > I agree about the importance of IDL.  But I'm still
 > thinking about your proposal.  You are calling the
 > client API the point at which structures get [un]marshalled.

not quite :-)

In Samba3 the link between the marshalling and these structures is
quite strong, but its much weaker in Samba4. A lot happens in the
generated code between those structures and the wire format (just
think about the NDR deferral mechanisms, arrays, ref pointers, strings
etc).

So I don't think its true to say that I'm proposing a "closer to the
wire" approach. It would be just as easy to make pidl generate a win32
style function interface, but I chose not to as I think it is a much
worse interface.

btw, a good example of the wire/structure separation is strings. In
Samba4 all strings in the IDL files appear as "const char *" in the
generated interface structures, no matter what form they take on the
wire. This is so that the user doesn't have to care about the wire
form - its just a unix string to them. In Samba3 they appear as little
"UNISTR2" structures which comes from the marshalling code, and the
caller has to care about the null termination rules. Similarly, array
sizes that are just NDR marshalling artifacts appear in the exposed
API structures in Samba3, but not in Samba4.

So the general rule we have in Samba4 is that only parameters that the
user should be setting appear in the structures in the API. The pidl
generated code worries about the details of keeping enough state to
synthesise all the bits and pieces of other NDR stuff.

 > The make_spoolss_q_open_printer_ex() call does what you
 > do by assigning the structure elements necessary for
 > the outgoing call.  So what you call the client interface
 > the the CLI_DO_RPC macro I've used above.

I can see similarities, but they are really only skin deep. The skin
does matter though :-)

 > What I'm still considering is whether or not this
 > low level API is really best for application writers.

I'm not sure it really is any more 'low level' than the win32 style
interface. Both have the same number of things the user has to set,
its just that in one the caller must name the elements specifically,
whereas in the other the naming comes from the ordering of the
parameters.

It certainly is more verbose, and leads to a higher line count in the
code, but I think the code is vastly more readable, and requires much
less of "go and lookup the docs for this function to see what
parameter 7 is".

 > I'm just not totally convinced yet, but am still thinking
 > about it.  Parts of it I like but parts of it I'm still
 > pondering.  At least now I do know that I understand your
 > original suggestion.

One thing to consider is scripting languages. I found it _much_ easier
to generate the js bindings for the Samba4 rpc code by following the
same pattern as the C interface. The only concession I had to make was
to use io.input.* instead of io.in.* as 'in' is a reserved word in js.

What I ended up with is code like this:

    /*
       get the sid for a domain
    */
    function samrLookupDomain(conn, handle, domain)
    {
	    var io = irpcObj();
	    io.input.connect_handle = handle;
	    io.input.domain_name = domain;
	    var status = conn.samr.samr_LookupDomain(conn, io);
	    check_status_ok(status);
	    return io.output.sid;
    }


which I find very readable, and so closely maps to the C code that
moving between the two is trivial.

See scripting/libjs/samr.js for more examples.

I also think you should not discount the advantage of the way of
writing the code matching the natural structure print format, and the
way ethereal displays structures. 

For example, with the above call and the [print] attribute on the rpc
binding string we get this:

    samr_LookupDomain: struct samr_LookupDomain
        in: struct samr_LookupDomain
            connect_handle           : *
                connect_handle: struct policy_handle
                    handle_type              : 0x00000000 (0)
                    uuid                     : 23b4b87c-fe62-408f-8d10-0e9384068065
            domain_name              : *
                domain_name: struct lsa_String
                    length                   : 0x000e (14)
                    size                     : 0x000e (14)
                    string                   : *
                        string                   : 'Builtin'
    samr_LookupDomain: struct samr_LookupDomain
        out: struct samr_LookupDomain
            sid                      : *
                sid                      : S-1-5-32
            result                   : NT_STATUS_OK

which is very easy to map to the calling API. Certainly much easier
than trying to make your eyes map a list of function parameters to the
structures.

There is also a place for a OO style API that works at a very high
level, and does things like "here is the name of a server, tell me the
home directory of all the users on it". That is a very good match for
a scripting language like python or js, but is much nastier in C. I
think we first need to get the "all parameters exposed" API done, then
people can build higher level APIs on top of that, possibly in
scripting languages.

The method I have chosen to do this in Samba4 is to auto-generate ejs
bindings for the full RPC API using the in.*, out.* style, then to
build higher level calls directly in the scripting language. That
allows the features of the scripting language to be properly utilised,
without running the risk that some important parameter has been left
out.

Cheers, Tridge