[jcifs] Blocking threads

[Michael B. Allen]

On Mon, Oct 22, 2001 at 02:24:16PM +0200, Torgny Johansson wrote:
> Just a short question:
> 
> Is it possible that jcifs can block threads?

Yes. When a Thread calls a method that results in network IO, the
Thread writes the message to the Socket and calls wait() -- meaning it
blocks. There is a Thread for every socket that watches incoming messages,
decodes them and signals the waiting caller with notify(). See send()
and sendTransaction() in:

http://auditorymodels.org/jcifs/src/src/jcifs/smb/SmbTransport.java

> If I run a thread where i initiate a SmbFile and list it, could this
> cause a thread to block?

Same exact answer as above.

> Also, is there any limit as to how many SmbFiles that can be initiated
> at the same time (in my case in separate threads) or is that just
> limited by how many threads the cpu can cope with?

Not exactly. It depends on whether jCIFS is talking to one server or many.

There is the maxMpxCount semaphore which limits the number of pending
messages with any one particular host at one time. Meaning, if maxMpxCount
is 10 (the default unless the server negotiates a lower value or the
jcifs.smb.maxMpxCount is changed) and 10 requests are sent to the server,
and 11th message will block. Then, let's say 3 responses are received
satisfying 3 requests. This will allow that 11th message to go along
with 2 others until it blocks again and so on. There is a maxMpxCount
negotiated with each server so if you, for example, created a 100 threads,
each of which tired to create a file on a particular server at the
same time only maxMpxCount requests would be pending at any given time
(creating 100 files would probably take less than a second though).

If however you have many threads talking to many different hosts, then
each will have it's own maxMpxCount and the limit is pretty much up
to what the machine can cope with (actually, in practice I have found
that either the network is the bottleneck or the VM tends to crashs if
you really push things to their maximums with a lot of Threads). This
is why the number of threads is a parameter to the ThreadedSmbCrawler
example. Remember that Threads don't do more work, they just increase the
likeyhood that CPU cycles will not be wasted on blocking IO and therefore
should only be used when there is opportunity to use those otherwise
wasted clock cycles. If you have some code that calculates fibonacci
numbers all day long, you should just use one thread. Using more will
just slow you down. If however, you send a message on a network and it
takes the server 1s to respond, the client cpu will be idle for that
1s. Creating one or more Threads to do work in these spaces is more
efficient. This is why jCIFS has a daemon Thread for each socket. It
can do asyncronous IO with one or many servers.

The current behavior of jCIFS with respect to handling concurrent callers
will actually change in 0.6. There is one very kludgey thing about
Transactions in jCIFS. A 64K buffer is allocated each time a transaction
is created. The list() method uses transactions (although I think it
reuses the one Transaction object). This is probably the #1 resource
eater especially for something like the ThreadedSmbCrawler. This will
be cleaned up by creating a small pool of 64K buffers (initially only
1 is necessary for 9/10 applications) from which each Transport object
will allocate a buffer for doing network IO. There will be a limit on
the number of buffers that can be created and therefore the number of
concurrent Transactions. This should have a noticable effect on the
performance of certain work loads. Not only will the amount of memory
consumed from these buffers have a cap but this cap will be very small
compared to the potentially 100s of bufferes created for the duration
of the client and they will not be allocated (and collected) on the fly.

Mike

-- 
May The Source be with you.