[jcifs] SMB URL Resource Resolution

[Allen, Michael B (RSCH)]

Hi,

It was recently discovered that jCIFS name services could easily fail when
doing mixed NetBIOS/DNS resource resolution. I have decided to rewrite the
code that resolves and connects to a server component of an SMB URL using
IP, NetBIOS, and DNS. Resource resolution is my term for resolving and
connecting to a server targeted by an SMB URL. Basically if the URL is:

  smb://foo

what is foo and how to we get a session with it? This is really the
bulk of an underlying SMB URL implementation. Everything else is
almost trivial. What follows is an outline with pseudocode for the
implementation. Note that this does not suggest any changes the SMB URL
draft at all. This is just jCIFS implementation details.

What's involved:

Unfortunately some compromises will be necessary because jCIFS does not
fully support talking to all types of resources and the focus of jCIFS
at the moment is a highly stable release that cannot be considered
broken in any way. The immediate questions that must be faced are,
for example, is complete mixed DNS/NetBIOS/IP w/w/o scope reasonable or
even possible? I will leave out scope and refactor it in later by simply
applying it globally via client parameter (that's what we do now). Also,
can a Win2K domain name on a network that serves it's browse list from
AD be distinguished from another that uses LMBs instead? Well, I'm not
about to add LDAP hooks to jCIFS when I do not even have an environment
within which to test. Hopefully the design is general enough that LDAP
and the other stuff that jCIFS doesn't support can be added later. I
have at least thought abo
ut this stuff though. In the context of the
SMB URL a complete implementation might touch on the following:

                                      key
                                        o supported by jCIFS
                                        x not supported
                                        w will be supported by 0.6


o NetBIOS over TCP/IP (NBT) Transport - Longstanding transport layer of
CIFS/SMB requires Session Request, Negative/Positive Session Response,
Session Keep Alive, etc.

x Plain TCP/IP Transport - Newer MS CIFS/SMB servers have dropped the
NBT layer. The above NBT messages are no longer required, offsets remain
the same, 4 byte header is ignored. From what little I know about this,
the difference sounds trivial.

o LANMAN Authentication - Widely used NT LM authentication is combination
of DES and MD5. Requires domain\user and password encrypted with session
key to be relayed by target host to third party (PDC) via DCE/RPC call
for comparison to user in central database.

x Kerberos Authentication - Newer MS environments may use PKI ticket
database server variant of MIT Kerberos (implementation constrained by
MS IP notable?).

w smb:// vs. cifs:// - No difference. Clients MAY use context as a hint
but should the prefix on any resource be alternated it should still be
resolved (albeit possibly slower).

x LDAP/ActiveDirectory - I believe this is also called ADSI? MS more
commonly uses COM to talk to a directory but I believe LDAP calls will
also work. The examples in their documentation use an LDAP:// prefixed
URL-ish syntax. As far as resources that might be referenced by an SMB
URL; the list of hosts associated with a domain (the browse list) and
user authentication credentials are of interest. Immediate questions
are what are the LDAP Distinguished Names to access these resources,
what type of binding/security mech. is used, and how can resources that
require querying the directory service be identified (i.e. If the browse
list for domain FOO is in AD somewhere how do we get a list of hosts from
smb://FOO/?). Don't even know enough about this to factor into the impl
and I'm not about to add LDAP lookups to jCIFS during a stable series
(don't even see a demand yet; actually one person asked about this once).

o DNS, NBNS/WINS, NetBIOS Broadcast or Unicast Name Queries, LMHOSTS,
IP, NT Domain/Workgroup, scope - All services or service types that may
be queried, addressed, or in some way affect SMB URL server component
resolution.

  More specifically about how each is used:

  DNS - DNS would only be used in one case; resolving a servername. I
  suppose it is possible that a reverse lookup could be useful because
  you have the first label to try as a CalledName but this is probably
  not cost effective.

  NBNS/WINS - to resolve server component as file server NAME<20>,
  Domain Master Browsers DOMAIN<1B>, or Local Master Browsers
  WORKGROUP<1D>. Note: After finding a DMB with a 0x1b lookup, another
  step is polite to select an LMB from that subnet for retrieving a
  browse list. This processes is outlined in a MSDN document. I have
  lost the link.

  NetBIOS Broadcast queries - to resolve empty URL smb:// by broadcasting
  for __MSBROWSE__ name, resolve server component as NetBIOS name if no
  WINS, or find Local Master Browsers WORKGROUP<1D>

Resource Resolution Methods:

The methods used to resolve CIFS/SMB and NetBIOS resources are outlined
below. Again, the descriptions here are specific to the context of
resolving resources addressed by an SMB URL.

WINS - More formerly known as NetBIOS Name Server (NBNS) this service
will resolve NetBIOS names. This is analogous to the Domain Name System
(DNS) however names are dynamically registered with WINS. Each NetBIOS
name has a hex code associated with it indicating what service a server
is offering. The following is a list of codes as well as descriptions
of how each is important to SMB URL resource resolution. Note that WINS
is being replaced by DNS and in many environments already has.

  00 - The Workstation code is advertised by all NetBIOS hosts however
  will likely not be used by a resolver because all services of interest
  can be found by the more specific codes that follow.

  20 - The File Server code indicates that the target is serving a file
  share, IPC service, Named Pipes, etc. It is the hex code that would
  most commonly be queried for because without the File Server service
  no shares, browse lists, pipes, or otherwise can be accessed.

  1B - Domain Master Browsers (DMB) register this hex code. To retrieve
  a list of hosts of a domain for which the Local Master Browser (LMB)
  is on another subnet a special procedure is necessary. The procedure
  is as follows:
    o Query WINS for the Domain Master Browser (DMB) with a Name Query
      Request for the DOMAIN<1B> name.
    o Retrieve a list of LMBs from the DMB
    o Select an LMB from the returned list and query that LMB for a
      browse list per usual.

  1C - Domain Controllers register the 1C code but do not serve or
  facilitate the resolution of any services that might be addressed by
  an SMB URL. Not to be confused with the 1B or 1D codes.

  1D - Local Master Browsers (LMB) serve the list of hosts that appear
  in Network Neighborhood for a particular subnet. They can be found by
  simply broadcasting a Name Query Request for the DOMAIN<1D> name or
  by enumerating the list of hosts from a browse list.

Broadcasting - Name Query Requests may be unicast to a WINS server
(or perhaps directly to a host) or broadcast on the local subnet. If a
WINS server is not available broadcasting may be necessary. Broadcasting
it less efficient and should therefore be avoided. Also, regardless of
whether or not a WINS address is provided, to find an LMB on the local
subnet, a broadcast Name Query Request for the __MSBROWSE__ name is used.

DNS - The Domain Name System is the most common name service used on the
Internet, internal WANs and elsewhere. The SMB URL specifically provides
for the server component to be resolved by DNS if necessary. Syntactically
the DNS name cannot be distinguished from a NetBIOS name. Both queries
must be performed to determine this information although in practice
one method would simply be favored over the other and therefore the
queries would not be performed in parallel. Note however if the favored
query fails the timeout period(s) will add the over resolution time
significantly.

Pseudocode:

This is Java-like pseudocode that is not intended to be explicit and
leaves much up to the imagination. The idea is to simply tease apart the
logic and therefore the behavior of SMB URL resource resolution. Context
will be necessary but knowledge of the Java language is not.

To start with something simple, this InetAddress type definition
encapsulates a internet hostname and it's IP address. It is intended that
it handle DNS resolution in the same way that the following NbtAddress
type handle NetBIOS related resolution. It's largely empty because the
details of DNS resolution are invariably handled by the host environment
(java.net.InetAddress.getByName() in Java).

InetAddress( name, ip )
  name
  ip

The NbtAddress definition has some initialization code followed by one
method that determines the next suitable calledName for NetBIOS session
establishment in the event that current calledName fails.

NbtAddress( name, type, ip )
  name = name
  type = type
  ip = ip
  calledName = name
  failures = 0
  maxFailures = 3
  maxPrefixFailures = 2
  if name is dot-quad ip
    maxFailures = 2
    calledName = *SMBSERVER
  else if i = name.indexOf('.') > 1 and i < 15
    maxFailures = 4
    calledName = name.substring( 0, i )
  else if name.length > 15
    calledName = name.substring( 0, 15 )

  bool nextCalledName()
    if failures++ >= maxFailures or calledName == null
      return false
    else if calledName == *SMBSERVER
      if( nodeStatus( ip ) does not have name of type ) {
        calledName = null
        return false
      }
      calledName = name of type
    else if calledName.length + 2 < name.length
      if failures < maxPrefixFailures and
          i = name.indexOf('.', calledName.length + 2) > 1 and i < 15
        calledName = name.substring( 0, i )
      else if name.length <= 15
        calledName = name
      else
        calledName = *SMBSERVER
    else
      calledName = *SMBSERVER
    return true

To use the above code, when trying to connect to a host the calledName is
tried and if rejected by the target (e.g. Not Listening on Called Name)
the nextCalledName function sets a new calledName to try if any):

  do {
    if sessionRequest( addr.calledName )
      return success
  } while( addr.nextCalledName() );
  return failure

Let nano.us.foo.net be an example target name. During initialization
nano is chosen as the initial calledName. The following are tried in
order assuming each fail:

  nano                 A dot suggests a DNS name
  nano.us              Try another prefix?
  nano.us.foo.net      Forget prefixs and just try the whole name
  *SMBSERVER           Fall back to *SMBSERVER
  <node status name>   Try name returned by a Node Status

Admittedly, I would rather the order be nano, *SMBSERVER, ... but it's
not clear how to do this and still maintain consistent support for all
cases such as:

  nano
  name.over15charlong.net
  a.b.c.d.e.f.g.h.i.foo.net
  .screwynbt
  192.168.1.15

To abstract these two host to IP address types (InetAddress and
NbtAddress) a new type that can represent both is used. Additional logic
to handle a resolveOrder seems to fit well here.

UniAddress( addr )
  addr = addr

  oneTimeInitialization()
    if resolveOrder not specified
      if wins specified
        resolveOrder = LMHOSTS,WINS,BCAST,DNS
      else
        resolveOrder = LMHOSTS,BCAST,DNS

  UniAddress resolveName( name, possibleNTDomainWorkgroup )
                      throws UnknownHostException
    if name is a dot-quad IP address
      return UniAddress( InetAddress( name ))
    for method in resolveOrder
      switch( method )
        case LMHOSTS:
          if lmhosts file has ip for name 0x20
            return UniAddress( NbtAddress( name, 0x20, ip ))
          break
        case WINS:
          if wins server has ip for name 0x20
            return UniAddress( NbtAddress( name, 0x20, ip ))
          if possibleNTDomainWorkgroup and
                  wins server has dmb list for name 0x1B
            dmb = randomly selected from list
            return UniAddress( NbtAddress( dmb, 0x1D, ip ))
          break
        case BCAST:
          if bcast area has ip for name 0x20
            return UniAddress( NbtAddress( name, 0x20, ip ))
          if possibleNTDomainWorkgroup and
                      bcast area has ip for name 0x1D
            return UniAddress( NbtAddress( name, 0x1D, ip ))
        case DNS:
          return UniAddress( InetAddress( name ))
    throw UnknownHostException

Resolve Order:

Because many resource resolution methods may be used a resolveOrder
property can be employed to specify which methods should be used and in
what order. This property might be a simple list of identifiers such as
LMHOSTS, WINS, BCAST, and DNS. For example, if the default order of this
property were:

  resolveOrder=LMHOSTS,WINS,BCAST,DNS

The lmhosts file would be queried. If that fails, WINS would be
queried. If that fails, a broadcast Name Query Request would be sent,
and if still no response has been received, the client could attempt to
resolve the name using DNS. If the resolveOrder property were specified
as:

  resolveOrder=DNS,LMHOSTS

then the client will not query WINS or broadcast name queries. Only DNS
would be queried first followed by the lmhosts file. Notice that in this
later configuration, no browse information can be retrieved. This is
because it is necessary to broadcast a name query for the __MSBROWSE__
name to find an LMB. Similarly, an NT Domain/Workgroup name will not be
resolved without WINS or at a bare minimum the BCAST method. It would
be difficult keep relatively static lmhosts or DNS zone files up to date
with NetBIOS group name to IP mappings.

Client Properties:

I think it would be difficult to implement the SMB URL without some kind
of settable property mechanism. Some critical properties that might be
associated with an SMB Client are:

  wins
  username
  password
  domain
  baddr
  laddr
  lport
  scope
  lmhosts
  resolveOrder
  hostname

The use of query string like parameters at the end of an SMB URL might
prove usefull and could use the same identifiers as the above mentioned
client properties. For example:

  smb://foo?scope=bar.com

Although I think the potential for using this mechanism as a security
exploit would suggest only select properties be settable through the
interface.

Mike