SV: [jcifs] BFS vs DFS

[Allen, Michael B (RSCH)]

Please note the public jCIFS API is far from ideal for an SmbCrawler. There
are a few advanced things that could be done to improve performance.

1) Do not use the public API but instead issue send( ServerMessageBlock
    req, ServerMessageBlock resp ) commands directly.
2) Cache information retrived from other operations (e.g. use Info data from
    findfirst/next operations instead of doing query info)
3) Use threads but only one per host.

These could improve performance by several orders of maganatude. It would
be faster than NT client by a lot.

Mike

> -----Original Message-----
> From:	Christopher R. Hertel [SMTP:crh at ubiqx.mn.org]
> Sent:	Thursday, July 26, 2001 12:09 PM
> To:	Torgny Johansson
> Cc:	jcifs at samba.org
> Subject:	Re: SV: [jcifs] BFS vs DFS
> 
> Hmmm...
> 
> I wonder what is taking so long.  It sounds as though some particular
> operation is just sitting there doing nothing while it waits for a
> timeout.  Multithreading the crawler might help.
> 
> Chris -)-----
> 
> On Thu, Jul 26, 2001 at 10:52:43AM +0200, Torgny Johansson wrote:
> > What is the best thing to do then?
> > I've written a crawler that just lists all the computers in the workgroups
> > from top to bottom (currently not threaded) and it takes very long time to
> > do a full "crawl". About 11 hours for 430 pcs (far from every pc have
> > shares) and that seems all too long. My code probably (read most
> > certainly...) not optimized, so briefly; which is the way to go to create an
> > efficient crawler?
> > 
> > Thanks
> > Torgny Johansson
> > 
> > -----Ursprungligt meddelande-----
> > Från: jcifs-admin at lists.samba.org
> > [mailto:jcifs-admin at lists.samba.org]För Allen, Michael B (RSCH)
> > Skickat: den 26 juli 2001 03:29
> > Till: 'jcifs at samba.org'
> > Ämne: [jcifs] BFS vs DFS
> > 
> > 
> > I wrote:
> > 
> > > try to minimize the size of your active list of URLs to
> > > search and therefore the number of URLs that might suddenly become invalid
> > > by using a Breath First Search algorithm.
> > 
> > This is not true. BFS would be awfull for an SmbCrawler.
> > 
> > Mike
> > 
> > 
> 
> -- 
> Samba Team -- http://www.samba.org/     -)-----   Christopher R. Hertel
> jCIFS Team -- http://jcifs.samba.org/   -)-----   ubiqx development, uninq.
> ubiqx Team -- http://www.ubiqx.org/     -)-----   crh at ubiqx.mn.org
>