[jcifs] Good Value for jcifs.smb.maxBuffers

[Michael B Allen]

On Wed, 28 Feb 2007 13:28:35 -0500
Jake Goulding <goulding at vivisimo.com> wrote:

> Hey all:
> 
> We have a crawler that uses many threads to access SMB shares. We had 
> some issues with deadlocks that seem to be alleviated by increasing the 
> number of buffers from the default to 100 (running 50 threads at a 
> time). Is there a guide to how many of these buffers provides an 
> adequate pool?

You would be in a much better position to know that I. You really need
to just expirement. I don't see any problem with using 100 buffers though.

Also, I am very skeptical when I hear someone using so many threads. More
threads doesn't equate to more work completed unless they're all blocked
and if you have 50 threads blocked then something else is wrong. From
what little experimentation I have done with crawling I have found that
2 threads per host is about optimal. Any more than that and the ROI
drops dramatically.

Also, if you can get a thread dump of the deadlock it might be good
if I have it on file. If you don't want to bother that's ok too. It's
probably the same "connect while disconnect" that has been discussed
numerous times but one day I'll need to look at the deadlock problem
again in which case I'll need to figure out how to reproduce it.

Mike

-- 
Michael B Allen
PHP Active Directory SSO
http://www.ioplex.com/