[jcifs] Grabbing a directory and all of its subdirectories

[Michael B Allen]

On Thu, 24 Feb 2005 16:00:33 -0500
Chris Moesel <cmoesel at mitre.org> wrote:

> If you read my previous email, you saw that I have a need for grabbing 
> the entire contents of a directory tree using the SMB protocol.  You 
> also saw that it takes me over 40 seconds to traverse 5400 files in 600 
> subdirectories of the tree (NOTE, I don't need the actual contents of 
> each file, only the information for creating a listing-- name, date, 
> size, etc).

JCIFS can travers tens of thousands of files in less time than that so I
think the bottleneck is the server and/or the network. First try changing
jcifs.smb.client.listSize to a value ~MTU - 80 (1200 is good). This can
be a little better on high latency networks. Second, note that if you
build up a big list and *then* go back and examine each file and the
jcifs.smb.client.attrExpirationPeriod has expired on those files the
client will re-query the server for fresh attributes. That will make
things MUCH slower so try setting this to 0 to see what happends. Finally
if you really must have maximum performance you could change your code
to be like examples/T2Crawler which uses multiple threads and a breadth
first search technique that keeps the stale list short.

Otherwise do a diff on the 1.1.3 and 1.1.8 source and see what changed
in the relivent files.

Mike

-- 
IRC - where men are men, women are men, and the boys are FBI agents.