[jcifs] Master Browsing Caching

[Christopher R. Hertel]

On Fri, Oct 25, 2002 at 12:46:17AM -0400, Matthew Tippett wrote:
> Looks like I triggered a meaty thread :)

It happens that I've put aside the SMB section of my book since I was
having writer's block and I've started work on Browsing.  :)

> >>     That is pretty interesting considering you have to connect to IPC$
> >>     to retrieve the browse list.
> >
> >Some systems require a valid username/password in order to access any 
> >share, including IPC$.  I'm not sure that this is the problem, though.
> >I imagine that it would be hard to read the browse list from a system
> >that required encrypted auth...if you don't have the credentials.
> 
> Which is the problem, at the protocol level (ethereal :), it is 
> connecting with ?\GUEST and no password, and in one case the PDC is 
> saying 'bugger off, that user has expired - hence no credentials', and 
> the other one is detecting that the SMB server uses plain text passwords 
> and forcing the user to turn on plaintext pasword support.
> 
> I agree that these are not technically jCIFS problem, but my whole 
> problem lies in that if *ONE* domain is poorly configured (I was 
> *fortunate* to have 2/5 that are not configured properly), then if the 
> dice fall such that that domain responds first with the __MSBROWSE__ 
> resolution, then the SmbFile('smb://') will fail completely.

Ah.  I see.

> A happier medium might be to grab leave the NameServiceClient thread 
> continue listening for responses *JUST* in case there are more than one 
> DMB's on the network and the one that responds first will not allow us 
> to connect?  My preferred approach would be to interogate all DMBs and 
> collate their individual world views.

So you're doing the smb:// first to get the list of known workgroups, and 
then going through that list talking to the DMBs, yes?

Note that if you have a workgroup that isn't also an NTDomain then there 
won't be a DMB for that name.

> >I agree with the "Why?".  What does caching do that prevents you from 
> >getting all of the local __MSBROWSE__ IPs?
> 
> I am attempting to operate purely at the SmbFile("smb://") level as per 
> the documentation.  I understand I can go down to the network and 
> resolve all of them seperately, but from a jCIFS consumers point of 
> view, the SmbFile("smb://"); interface appears on the surface to do what 
> is needed... but doesn't in my fringe condition.

Right...

> The previous email about how the lifecycle of the names are different is 
> also important.
> 
> One final point would be that is it right to cache *only one* of the 
> addresses of a name where there are known to be multiple?  IMHO that 
> could quite easily leave a jCIFS session getting a one-sided view of the 
> network, considering the whole NetBIOS naming approach hinges on systems 
> chatting over the wire and listening to other people (as mentioned below).

If the purpose of resolving the name is to contact *a* system, then yes.  
If you want to contact multiple systems, send a broadcast.  Despite the
RFCs, the CIFS suite does not provide any mechanism for contacting all
members of a group if the IP network is routed.

> >If it sounds as if I'm waffling, I am.  Mike is right from the practical 
> >point of view.  I would say, though, that it would be worth-while to have 
> >a mechanism for doing a group name query and retrieving from that a list 
> >of IPs.  That is lower-level functionality that can be very useful for 
> >folks doing special-purpose work like this.
> 
> That is probably the crux of the issue, the code as is handles the 95% 
> of cases where the network is configured 95% correctly.  Even if I can't 
> convince the current jCIFS developers that there is a better way, I will 
> somehow work out how to graft the functionality in.

I understand.

> As a technologist and a pragmatist, I understand what is happening from 
> a technology point of view, but from a pragmatic view, the 
> SmbFile("smb://") interface is a jCIFS only implementation (I understand 
> about the ietf draft, but how man implementations use the URL format, 
> and what is their approach), and currently doesn't fully follow the 
> intent behind the documentation.

Well, I wrote that draft...  Apple implements it, Thursby, Samba's 
libsmbclient also does and that's used in KDE...  It's not just us.  The 
problem is that we don't have all of the bugs worked out yet and we 
probably never will (such is the nature of SMB) so it's a question of 
choosing the compromises.

> Quoting the SmbFile documentation :
> 
> the smb://  URL lists all available workgroups on a netbios LAN. Again, 
> in this context many methods are not valid and return default 
> values(e.g. isHidden and renameTo will always return false).

There are a lot of ways to go about returning the available workgroups.  
Since the word "LAN" is used, we could query for __MSBROWSE__ and then, 
using the IPs that are returned, do an adapter status query to find the 
<1D> name of each node.  That would be slow and ugly.  ;)

Querying the first LMB that replies is much faster and cleaner, but it
also has problems.  As you point out, if the LMB is not being cooperative 
you won't be able to get the list from it.  Asking the next LMB in the 
group list is a good solution...

> I don't see any provisos on that such as
> 
> ' only if the jCIFS client has baseline authentication suitable for the 
> master browser that responds; if there is more than one DMB on the 
> network, the DOMAIN list will be retrieved from that server only (if 
> possible).'

LMB, not DMB.  The LMB that's being queried may also have DMB status, but 
it is the LMB service that registers the __MSBROWSE__ name.  If a node is 
*the* DMB for a workgroup it will also be a LMB for that workgroup on the 
local LAN.

Anyway, yes, I understand the problem.

> Which of course would *VERY* quickly shut me up and force me to 
> implement the functionally that I wanted by using direct NetBIOS calls :).

Yep.  :)

> I tend to liken NetBIOS naming as to a whole pile of egotistic children 
> at school shouting at each other trying to work out who is going to put 
> together a list of people attending class that day for the teacher.  As 
> you can imagine, it is a miracle if it works at all :).

That's B mode.  In P mode they write on the bathroom wall.  In M and H 
mode they do both (but they do it more efficiently in H mode).

> Looking forwards to the replies :)

Poor fool.  ;)  ;)


>From your other message:

> My personal preference is that jcifs works to the documentation.

Now, now...

> Creating a SmbFile("smb://") *should* do it's absolute best to return a
> complete list of Domains if they are available.  The following Snippet
> simply doesn't work reliably in a poorly configured network.

Note the phrase "poorly configured network".  Sigh.  Implementing SMB is 
all about outsmarting a pooly designed protocol used to build poorly 
configured networks.  Grumble.

Anyway, if you *really* wanted to do your best to find all available 
information:

Query the local LAN for __MSBROWSE__
for each IP in the returned list
  {
  Send an Adapter Status Query to get the workgroup name
    Add that name to the workgroup list
  Send a query to get the workgroup list from the browse service
    Add those names to the workgroup list, removing duplicates
  }
for each name in the workgroup list
  {
  Query the DMB for that workgroup to get its list of known workgroups
    to see if you missed any the first time 'round
  }
Present the list.


See, it can get *way* out of hand.  It makes no sense to do the "absolute" 
best to return a "complete" list of workgroups.  The browse system is not 
robust enough to provide such a list without a lot of work, and even then 
there might be a workgroup on some subnet somewhere that doesn't have a 
DMB and there's no way to discover that workgroup from the outside becuase 
the WINS servers don't handle the names per-RFC so you can't find an LMB 
on a remote subnet unless you can do directed broadcast which any sane 
network admin disables.

:)  Geez this stuff is fun.

> Don't get me wrong however, jcifs is a great piece of work, but where a
> feature isn't part of the protocol, but an artefact of the development
> ('smb://'),

That's really not a development artifact.  The prescribed way to list all 
workgroups is to query an LMB for the list.  The fact is, the protocol is 
flakey and the suggestions in this discussion are ways to work around the 
lite-crusted flakiness.

> I don't feel that we should fall back to a poor (potentially
> broken) approach when *ALL* Master Browsers do respond, we could
> trivially pull together a complete list.  Contrasting what I have seen
> of other application CIFS browsers, jCIFS is a lot better and does
> actually discover most of the network.  I just want to push the
> reliability of the scan beyond what is now to be able to be more robust
> even in the face of poorly configured corporate and (more likely) home
> machines.

Let's dig into this a bit further.

The base problem is that the browsing system itself is not particularly
robust.  Depending upon who calculates what, the convergence time when a
node appears or disappears can be on the order of an hour.  There is also
the subtle distinction between a workgroup and an NTDomain (which I have
not bothered with in my discussion up 'till now).  Simply put, a workgroup
doesn't have a DMB.  If a workgroup doesn't have a DMB, I am not sure
how/if its server list is transferred outside of the subnet.  That's 
something I need to check.

Anyway, the solution you suggest is that, in response to 'smb://', jCIFS 
will collect a list of IP addresses of local LMBs and then query the LMBs 
in turn until one of them responds.

That makes sense to me.  Note the following, however:

- If there are no LMBs on the local LAN, then there is no way for jCIFS to 
  find any workgroup information.  You can't query the WINS server for a 
  DMB unless you have a workgroup name to give it.

- If you do find an LMB, its list may not be up-to-date.

- If none of the LMBs will provide the browse list, you can still do an 
  adapter status on each IP and look for the <1D> names.  That gives you a 
  list (though probably incomplete) of workgroups.  You could then query 
  the DMBs to see if any of those exist and will respond with a better
  list.  Could, but we're doing a whole lot of work at this point.


> Thanks for the lively response, hopefully this response has made my
> rationale for the first posting a bit clearer.

I tend to ramble in my replies.  That's cause I'm thinking about this 
stuff when I write it.  Have fun with it.

Chris -)-----


-- 
Samba Team -- http://www.samba.org/     -)-----   Christopher R. Hertel
jCIFS Team -- http://jcifs.samba.org/   -)-----   ubiqx development, uninq.
ubiqx Team -- http://www.ubiqx.org/     -)-----   crh at ubiqx.mn.org
OnLineBook -- http://ubiqx.org/cifs/    -)-----   crh at ubiqx.org