[SCM] Samba Shared Repository - branch master updated
Martin Schwenke
martins at samba.org
Sun Jan 21 10:49:01 UTC 2018
The branch, master has been updated
via ac9d528 docs: Remove prog_guide4.txt
via c6999a2 vfs_fileid: Fix the 32-bit build
via 3904c26 Added smbc_SetConfiguration which lets the user set the smb.conf for libsmbclient code
from 7c1c8c6 mit-kdb: support MIT Kerberos 1.16 KDB API changes
https://git.samba.org/?p=samba.git;a=shortlog;h=master
- Log -----------------------------------------------------------------
commit ac9d528b823aa89b3f3144f9377cc540f4b692e4
Author: Volker Lendecke <vl at samba.org>
Date: Thu Jan 18 10:57:23 2018 +0100
docs: Remove prog_guide4.txt
Move the still relevant parts elsewhere
Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>
Autobuild-User(master): Martin Schwenke <martins at samba.org>
Autobuild-Date(master): Sun Jan 21 11:48:01 CET 2018 on sn-devel-144
commit c6999a248ad78f75cbfcc0f461298021b20905b4
Author: Volker Lendecke <vl at samba.org>
Date: Fri Jan 19 12:15:58 2018 +0000
vfs_fileid: Fix the 32-bit build
Signed-off-by: Volker Lendecke <vl at samba.org>
Reviewed-by: Jeremy Allison <jra at samba.org>
commit 3904c26ac860038518d4ffae781ddbbf23746715
Author: Puran Chand <pchand at vmware.com>
Date: Wed Jan 10 13:13:44 2018 +0530
Added smbc_SetConfiguration which lets the user set the smb.conf for libsmbclient code
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13229
Signed-off-by: Puran Chand <pchand at vmware.com>
Reviewed-by: Jeremy Allison <jra at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
-----------------------------------------------------------------------
Summary of changes:
librpc/binding-strings.txt | 55 ++
prog_guide4.txt | 777 ---------------------
source3/include/libsmbclient.h | 4 +
.../{smbclient-0.3.1.sigs => smbclient-0.3.2.sigs} | 1 +
source3/libsmb/libsmb_setget.c | 15 +
source3/libsmb/wscript | 2 +-
source3/modules/vfs_fileid.c | 4 +-
source3/selftest/tests.py | 3 +
source4/rpc_server/dcerpc_server.h | 26 +
source4/selftest/tests.py | 3 +-
source4/torture/libsmbclient/libsmbclient.c | 65 ++
testdata/samba3/smb_new.conf | 7 +
12 files changed, 181 insertions(+), 781 deletions(-)
create mode 100644 librpc/binding-strings.txt
delete mode 100644 prog_guide4.txt
copy source3/libsmb/ABI/{smbclient-0.3.1.sigs => smbclient-0.3.2.sigs} (99%)
create mode 100644 testdata/samba3/smb_new.conf
Changeset truncated at 500 lines:
diff --git a/librpc/binding-strings.txt b/librpc/binding-strings.txt
new file mode 100644
index 0000000..5503da1
--- /dev/null
+++ b/librpc/binding-strings.txt
@@ -0,0 +1,55 @@
+DCERPC binding strings
+----------------------
+
+When connecting to a dcerpc service you need to specify a binding
+string.
+
+The format is:
+
+ TRANSPORT:host[flags]
+
+where TRANSPORT is either ncacn_np for SMB or ncacn_ip_tcp for RPC/TCP
+
+"host" is an IP or hostname or netbios name. If the binding string
+identifies the server side of an endpoint, "host" may be an empty
+string.
+
+"flags" can include a SMB pipe name if using the ncacn_np transport or
+a TCP port number if using the ncacn_ip_tcp transport, otherwise they
+will be auto-determined.
+
+other recognised flags are:
+
+ sign : enable ntlmssp signing
+ seal : enable ntlmssp sealing
+ spnego : use SPNEGO instead of NTLMSSP authentication
+ krb5 : use KRB5 instead of NTLMSSP authentication
+ connect : enable rpc connect level auth (auth, but no sign or seal)
+ validate : enable the NDR validator
+ print : enable debugging of the packets
+ bigendian : use bigendian RPC
+ padcheck : check reply data for non-zero pad bytes
+
+
+Here are some examples:
+
+ ncacn_np:myserver
+ ncacn_np:myserver[samr]
+ ncacn_np:myserver[\pipe\samr]
+ ncacn_np:myserver[/pipe/samr]
+ ncacn_np:myserver[samr,sign,print]
+ ncacn_np:myserver[sign,spnego]
+ ncacn_np:myserver[\pipe\samr,sign,seal,bigendian]
+ ncacn_np:myserver[/pipe/samr,seal,validate]
+ ncacn_np:
+ ncacn_np:[/pipe/samr]
+ ncacn_ip_tcp:myserver
+ ncacn_ip_tcp:myserver[1024]
+ ncacn_ip_tcp:myserver[sign,seal]
+ ncacn_ip_tcp:myserver[spnego,seal]
+
+
+IDEA: Maybe extend UNC names like this?
+
+ smbclient //server/share
+ smbclient //server/share[sign,seal,spnego]
diff --git a/prog_guide4.txt b/prog_guide4.txt
deleted file mode 100644
index 0a33284..0000000
--- a/prog_guide4.txt
+++ /dev/null
@@ -1,777 +0,0 @@
-
-
-THIS IS INCOMPLETE! I'M ONLY COMMITING IT IN ORDER TO SOLICIT COMMENTS
-FROM A FEW PEOPLE. DON'T TAKE THIS AS THE FINAL VERSION YET.
-
-
-Samba4 Programming Guide
-========================
-
-.. contents::
-
-The internals of Samba4 are quite different from previous versions of
-Samba, so even if you are an experienced Samba developer please take
-the time to read through this document.
-
-This document will explain both the broad structure of Samba4, and
-some of the common coding elements such as memory management and
-dealing with macros.
-
-
-Coding Style
-------------
-
-In past versions of Samba we have basically let each programmer choose
-their own programming style. Unfortunately the result has often been
-that code that other members of the team find difficult to read. For
-Samba version 4 I would like to standardise on a common coding style
-to make the whole tree more readable. For those of you who are
-horrified at the idea of having to learn a new style, I can assure you
-that it isn't as painful as you might think. I was forced to adopt a
-new style when I started working on the Linux kernel, and after some
-initial pain found it quite easy.
-
-That said, I don't want to invent a new style, instead I would like to
-adopt the style used by the Linux kernel. It is a widely used style
-with plenty of support tools available. See Documentation/CodingStyle
-in the Linux source tree. This is the style that I have used to write
-all of the core infrastructure for Samba4 and I think that we should
-continue with that style.
-
-I also think that we should most definately *not* adopt an automatic
-reformatting system in cvs (or whatever other source code system we
-end up using in the future). Such automatic formatters are, in my
-experience, incredibly error prone and don't understand the necessary
-exceptions. I don't mind if people use automated tools to reformat
-their own code before they commit it, but please do not run such
-automated tools on large slabs of existing code without being willing
-to spend a *lot* of time hand checking the results.
-
-Finally, I think that for code that is parsing or formatting protocol
-packets the code layout should strongly reflect the packet
-format. That means ordring the code so that it parses in the same
-order as the packet is stored on the wire (where possible) and using
-white space to align packet offsets so that a reader can immediately
-map any line of the code to the corresponding place in the packet.
-
-
-Static and Global Data
-----------------------
-
-The basic rule is "avoid static and global data like the plague". What
-do I mean by static data? The way to tell if you have static data in a
-file is to use the "size" utility in Linux. For example if we run::
-
- size libcli/raw/*.o
-
-in Samba4 then you get the following::
-
- text data bss dec hex filename
- 2015 0 0 2015 7df libcli/raw/clikrb5.o
- 202 0 0 202 ca libcli/raw/clioplock.o
- 35 0 0 35 23 libcli/raw/clirewrite.o
- 3891 0 0 3891 f33 libcli/raw/clisession.o
- 869 0 0 869 365 libcli/raw/clisocket.o
- 4962 0 0 4962 1362 libcli/raw/clispnego.o
- 1223 0 0 1223 4c7 libcli/raw/clitransport.o
- 2294 0 0 2294 8f6 libcli/raw/clitree.o
- 1081 0 0 1081 439 libcli/raw/raweas.o
- 6765 0 0 6765 1a6d libcli/raw/rawfile.o
- 6824 0 0 6824 1aa8 libcli/raw/rawfileinfo.o
- 2944 0 0 2944 b80 libcli/raw/rawfsinfo.o
- 541 0 0 541 21d libcli/raw/rawioctl.o
- 1728 0 0 1728 6c0 libcli/raw/rawnegotiate.o
- 723 0 0 723 2d3 libcli/raw/rawnotify.o
- 3779 0 0 3779 ec3 libcli/raw/rawreadwrite.o
- 6597 0 0 6597 19c5 libcli/raw/rawrequest.o
- 5580 0 0 5580 15cc libcli/raw/rawsearch.o
- 3034 0 0 3034 bda libcli/raw/rawsetfileinfo.o
- 5187 0 0 5187 1443 libcli/raw/rawtrans.o
- 2033 0 0 2033 7f1 libcli/raw/smb_signing.o
-
-notice that the "data" and "bss" columns are all zero? That is
-good. If there are any non-zero values in data or bss then that
-indicates static data and is bad (as a rule of thumb).
-
-Lets compare that result to the equivalent in Samba3::
-
- text data bss dec hex filename
- 3978 0 0 3978 f8a libsmb/asn1.o
- 18963 0 288 19251 4b33 libsmb/cliconnect.o
- 2815 0 1024 3839 eff libsmb/clidgram.o
- 4038 0 0 4038 fc6 libsmb/clientgen.o
- 3337 664 256 4257 10a1 libsmb/clierror.o
- 10043 0 0 10043 273b libsmb/clifile.o
- 332 0 0 332 14c libsmb/clifsinfo.o
- 166 0 0 166 a6 libsmb/clikrb5.o
- 5212 0 0 5212 145c libsmb/clilist.o
- 1367 0 0 1367 557 libsmb/climessage.o
- 259 0 0 259 103 libsmb/clioplock.o
- 1584 0 0 1584 630 libsmb/cliprint.o
- 7565 0 256 7821 1e8d libsmb/cliquota.o
- 7694 0 0 7694 1e0e libsmb/clirap.o
- 27440 0 0 27440 6b30 libsmb/clirap2.o
- 2905 0 0 2905 b59 libsmb/clireadwrite.o
- 1698 0 0 1698 6a2 libsmb/clisecdesc.o
- 5517 0 0 5517 158d libsmb/clispnego.o
- 485 0 0 485 1e5 libsmb/clistr.o
- 8449 0 0 8449 2101 libsmb/clitrans.o
- 2053 0 4 2057 809 libsmb/conncache.o
- 3041 0 256 3297 ce1 libsmb/credentials.o
- 1261 0 1024 2285 8ed libsmb/doserr.o
- 14560 0 0 14560 38e0 libsmb/errormap.o
- 3645 0 0 3645 e3d libsmb/namecache.o
- 16815 0 8 16823 41b7 libsmb/namequery.o
- 1626 0 0 1626 65a libsmb/namequery_dc.o
- 14301 0 1076 15377 3c11 libsmb/nmblib.o
- 24516 0 2048 26564 67c4 libsmb/nterr.o
- 8661 0 8 8669 21dd libsmb/ntlmssp.o
- 3188 0 0 3188 c74 libsmb/ntlmssp_parse.o
- 4945 0 0 4945 1351 libsmb/ntlmssp_sign.o
- 1303 0 0 1303 517 libsmb/passchange.o
- 1221 0 0 1221 4c5 libsmb/pwd_cache.o
- 2475 0 4 2479 9af libsmb/samlogon_cache.o
- 10768 32 0 10800 2a30 libsmb/smb_signing.o
- 4524 0 16 4540 11bc libsmb/smbdes.o
- 5708 0 0 5708 164c libsmb/smbencrypt.o
- 7049 0 3072 10121 2789 libsmb/smberr.o
- 2995 0 0 2995 bb3 libsmb/spnego.o
- 3186 0 0 3186 c72 libsmb/trustdom_cache.o
- 1742 0 0 1742 6ce libsmb/trusts_util.o
- 918 0 28 946 3b2 libsmb/unexpected.o
-
-notice all of the non-zero data and bss elements? Every bit of that
-data is a bug waiting to happen.
-
-Static data is evil as it has the following consequences:
-- it makes code much less likely to be thread-safe
-- it makes code much less likely to be recursion-safe
-- it leads to subtle side effects when the same code is called from multiple places
-- doesn't play well with shared libraries or plugins
-
-Static data is particularly evil in library code (such as our internal
-smb and rpc libraries). If you can get rid of all static data in
-libraries then you can make some fairly strong guarantees about the
-behaviour of functions in that library, which really helps.
-
-Of course, it is possible to write code that uses static data and is
-safe, it's just much harder to do that than just avoid static data in
-the first place. We have been tripped up countless times by subtle
-bugs in Samba due to the use of static data, so I think it is time to
-start avoiding it in new code. Much of the core infrastructure of
-Samba4 was specifically written to avoid static data, so I'm going to
-be really annoyed if everyone starts adding lots of static data back
-in.
-
-So, how do we avoid static data? The basic method is to use context
-pointers. When reading the Samba4 code you will notice that just about
-every function takes a pointer to a context structure as its first
-argument. Any data that the function needs that isn't an explicit
-argument to the function can be found by traversing that context.
-
-Note that this includes all of the little caches that we have lying
-all over the code in Samba3. I'm referring to the ones that generally
-have a "static int initialised" and then some static string or integer
-that remembers the last return value of the function. Get rid of them!
-If you are *REALLY* absolutely completely certain that your personal
-favourite mini-cache is needed then you should do it properly by
-putting it into the appropriate context rather than doing it the lazy
-way by putting it inside the target function. I would suggest however
-that the vast majority of those little caches are useless - don't
-stick it in unless you have really firm benchmarking results that show
-that it is needed and helps by a significant amount.
-
-Note that Samba4 is not yet completely clean of static data like
-this. I've gotten the smbd/ directory down to 24 bytes of static data,
-and libcli/raw/ down to zero. I've also gotten the ntvfs layer and all
-backends down to just 8 bytes in ntvfs_base.c. The rest still needs
-some more work.
-
-Also note that truly constant data is OK, and will not in fact show up
-in the data and bss columns in "size" anyway (it will be included in
-"text"). So you can have constant tables of protocol data.
-
-
-How to use talloc
------------------
-
-Please see the separate document, lib/talloc/talloc_guide.txt
-You _must_ read this if you want to program in Samba4.
-
-
-Interface Structures
---------------------
-
-One of the biggest changes in Samba4 is the universal use of interface
-structures. Go take a look through libcli/raw/interfaces.h now to get
-an idea of what I am talking about.
-
-In Samba3 many of the core wire structures in the SMB protocol were
-never explicitly defined in Samba. Instead, our parse and generation
-functions just worked directly with wire buffers. The biggest problem
-with this is that is tied our parse code with our "business logic"
-much too closely, which meant the code got extremely confusing to
-read.
-
-In Samba4 we have explicitly defined interface structures for
-everything in the protocol. When we receive a buffer we always parse
-it completely into one of these structures, then we pass a pointer to
-that structure to a backend handler. What we must *not* do is make any
-decisions about the data inside the parse functions. That is critical
-as different backends will need different portions of the data. This
-leads to a golden rule for Samba4:
-
- "don't design interfaces that lose information"
-
-In Samba3 our backends often received "condensed" versions of the
-information sent from clients, but this inevitably meant that some
-backends could not get at the data they needed to do what they wanted,
-so from now on we should expose the backends to all of the available
-information and let them choose which bits they want.
-
-Ok, so now some of you will be thinking "this sounds just like our
-msrpc code from Samba3", and while to some extent this is true there
-are extremely important differences in the approach that are worth
-pointing out.
-
-In the Samba3 msrpc code we used explicit parse structures for all
-msrpc functions. The problem is that we didn't just put all of the
-real variables in these structures, we also put in all the artifacts
-as well. A good example is the security descriptor strucrure that
-looks like this in Samba3::
-
- typedef struct security_descriptor_info
- {
- uint16 revision;
- uint16 type;
-
- uint32 off_owner_sid;
- uint32 off_grp_sid;
- uint32 off_sacl;
- uint32 off_dacl;
-
- SEC_ACL *dacl;
- SEC_ACL *sacl;
- DOM_SID *owner_sid;
- DOM_SID *grp_sid;
- } SEC_DESC;
-
-The problem with this structure is all the off_* variables. Those are
-not part of the interface, and do not appear in any real descriptions
-of Microsoft security descriptors. They are parsing artifacts
-generated by the IDL compiler that Microsoft use. That doesn't mean
-they aren't needed on the wire - indeed they are as they tell the
-parser where to find the following four variables, but they should
-*NOT* be in the interface structure.
-
-In Samba3 there were unwritten rules about which variables in a
-structure a high level caller has to fill in and which ones are filled
-in by the marshalling code. In Samba4 those rules are gone, because
-the redundant artifact variables are gone. The high level caller just
-sets up the real variables and the marshalling code worries about
-generating the right offsets.
-
-The same rule applies to strings. In many places in the SMB and MSRPC
-protocols complex strings are used on the wire, with complex rules
-about padding, format, alighment, termination etc. None of that
-information is useful to a high level calling routine or to a backend - its
-all just so much wire fluff. So, in Samba4 these strings are
-just "char \*" and are always in our internal multi-byte format (which
-is usually UTF8). It is up to the parse functions to worry about
-translating the format and getting the padding right.
-
-The one exception to this is the use of the WIRE_STRING type, but that
-has a very good justification in terms of regression testing. Go and
-read the comment in smb_interfaces.h about that now.
-
-So, here is another rule to code by. When writing an interface
-structure think carefully about what variables in the structure can be
-left out as they are redundant. If some length is effectively defined
-twice on the wire then only put it once in the packet. If a length can
-be inferred from a null termination then do that and leave the length
-out of the structure completely. Don't put redundant stuff in
-structures!
-
-
-Async Design
-------------
-
-Samba4 has an asynchronous design. That affects *lots* of the code,
-and the implications of the asynchronous design needs to be considered
-just about everywhere.
-
-The first aspect of the async design to look at is the SMB client
-library. Lets take a look at the following three functions in
-libcli/raw/rawfile.c::
-
- struct cli_request *smb_raw_seek_send(struct cli_tree *tree, struct smb_seek *parms);
- NTSTATUS smb_raw_seek_recv(struct cli_request *req, struct smb_seek *parms);
- NTSTATUS smb_raw_seek(struct cli_tree *tree, struct smb_seek *parms);
-
-Go and read them now then come back.
-
-Ok, first notice there there are 3 separate functions, whereas the
-equivalent code in Samba3 had just one. Also note that the 3rd
-function is extremely simple - its just a wrapper around calling the
-first two in order.
-
-The three separate functions are needed because we need to be able to
-generate SMB calls asynchronously. The first call, which for smb calls
-is always called smb_raw_XXXX_send(), constructs and sends a SMB
-request and returns a "struct cli_request" which acts as a handle for
-the request. The caller is then free to do lots of other calls if it
-wants to, then when it is ready it can call the smb_raw_XXX_recv()
-function to receive the reply.
-
-If all you want is a synchronous call then call the 3rd interface, the
-one called smb_raw_XXXX(). That just calls the first two in order, and
-blocks waiting for the reply.
-
-But what if you want to be called when the reply comes in? Yes, thats
-possible. You can do things like this::
-
- struct cli_request *req;
-
- req = smb_raw_XXX_send(tree, params);
-
- req->async.fn = my_callback;
- req->async.private = my_private_data;
-
-then in your callback function you can call the smb_raw_XXXX_recv()
-function to receive the reply. Your callback will receive the "req"
-pointer, which you can use to retrieve your private data from
-req->async.private.
-
-Then all you need to do is ensure that the main loop in the client
-library gets called. You can either do that by polling the connection
-using cli_transport_pending() and cli_request_receive_next() or you
-can use transport->idle.func to setup an idle function handler to call
-back to your main code. Either way, you can build a fully async
-application.
-
-In order to support all of this we have to make sure that when we
-write a piece of library code (SMB, MSRPC etc) that we build the
-separate _send() and _recv() functions. It really is worth the effort.
-
-Now about async in smbd, a much more complex topic.
-
-The SMB protocol is inherently async. Some functions (such as change
-notify) often don't return for hours, while hundreds of other
-functions pass through the socket. Take a look at the RAW-MUX test in
-the Samba4 smbtorture to see some really extreme examples of the sort
-of async operations that Windows supports. I particularly like the
-open/open/close sequence where the 2nd open (which conflicts with the
-first) succeeds because the subsequent close is answered out of order.
-
-In Samba3 we handled this stuff very badly. We had awful "pending
-request" queues that allocated full 128k packet buffers, and even with
-all that crap we got the semantics wrong. In Samba4 I intend to make
-sure we get this stuff right.
-
-So, how do we do this? We now have an async interface between smbd and
-the NTVFS backends. Whenever smbd calls into a backend the backend has
-an option of answer the request in a synchronous fashion if it wants
-to just like in Samba3, but it also has the option of answering the
-request asynchronously. The only backend that currently does this is
-the CIFS backend, but I hope the other backends will soon do this to.
-
-To make this work you need to do things like this in the backend::
-
- req->control_flags |= REQ_CONTROL_ASYNC;
-
-that tells smbd that the backend has elected to reply later rather
-than replying immediately. The backend must *only* do this if
-req->async.send_fn is not NULL. If send_fn is NULL then it means that
-the smbd front end cannot handle this function being replied to in an
-async fashion.
-
-If the backend does this then it is up to the backend to call
-req->async.send_fn() when it is ready to reply. It the meantime smbd
-puts the call on hold and goes back to answering other requests on the
-socket.
-
-Inside smbd you will find that there is code to support this. The most
-obvious change is that smbd splits each SMB reply function into two
-parts - just like the client library has a _send() and _recv()
-function, so smbd has a _send() function and the parse function for
-each SMB.
-
-As an example go and have a look at reply_getatr_send() and
-reply_getatr() in smb_server/smb/reply.c. Read them? Good.
-
-Notice that reply_getatr() sets up the req->async structure to contain
-the send function. Thats how the backend gets to do an async reply, it
-calls this function when it is ready. Also notice that reply_getatr()
-only does the parsing of the request, and does not do the reply
-generation. That is done by the _send() function.
-
-
-NTVFS
------
-
-One of the most noticeable changes in Samba4 is the introduction of
-the NTVFS layer. This provided the initial motivation for the design
-of Samba4 and in many ways lies at the heart of the design.
-
-In Samba3 the main file serving process (smbd) combined the handling
-of the SMB protocol with the mapping to POSIX semantics in the same
-code. If you look in smbd/reply.c in Samba3 you see numerous places
-where POSIX assumptions are mixed tightly with SMB parsing code. We
-did have a VFS layer in Samba3, but it was a POSIX-like VFS layer, so
-no matter how you wrote a plugin you could not bypass the POSIX
-mapping decisions that had already been made before the VFS layer was
-called.
-
-In Samba4 things are quite different. All SMB parsing is performed in
-the smbd front end, then fully parsed requests are passed to the NTVFS
-backend. That backend makes any semantic mapping decisions and fills
-in the 'out' portion of the request. The front end is then responsible
-for putting those results into wire format and sending them to the
-client.
-
-Lets have a look at one of those request structures. Go and read the
-definition of "union smb_write" and "enum write_level" in
--
Samba Shared Repository
More information about the samba-cvs
mailing list