[jcifs] Memory Leak?

[Michael B Allen]

On Thu, Sep 17, 2009 at 10:34 AM, Andy Thomson <a10008051 at gmail.com> wrote:
> Mike,
>
> Seeing a memory leak in the latest jcifs 1.3.12 or it sure looks like one.
>
> This is from a long running process -- typically after about 10 hours it
> will error with an out of memory condition.
>
> System has 64GB memory
> java -Xmx1024m
> jcifs.smb.client.ssnLimit=1
>
> "Exception in thread "Transport414026" java.lang.OutOfMemoryError: Java
> heap space"
>
> I happened to grab a heap dump about 11 hours into the latest run,
> ironically it ran out of memory shortly afterwards.  What it looks like
> is that the connections are never released. More about this after some
> of the statistical information from the jhat tool.

With ssnLimit=1 I'm not surprised by this at all. Setting ssnLimit = 1
causes JCIFS to create a completely new connection for each SmbFile.
I'm surprised it ran as long as it did.

However, after the transport has been idle for soTimeout, it should
release the socket and all sessions on that transport (which will be
only 1 if ssnLimit = 1) and all trees for those sessions etc so it
should recover much of those resources.

> Listed below is a partial list containing a count of the all instances
> in the dump, this is just the top of the list. After 11 hours of running
> there are 304,000+ instances of various jcifs objects.  Notice that the
> one of the largest consumers is a LinkedList. These objects account for
> 739 MB of "retained" memory in this heap dump.
>
> Instance Counts for All Classes (including platform)
>
> 1519162 instances of class [Ljava.lang.Object;
> 1236988 instances of class java.lang.Object
> 1219633 instances of class java.util.LinkedList$Entry
>  630506 instances of class java.lang.String
>  630493 instances of class [C
>  609819 instances of class java.util.LinkedList
>  607371 instances of class [S
>  313855 instances of class [B
>  312116 instances of class java.net.Inet4Address
>  310583 instances of class java.net.Socket
>  310583 instances of class java.net.SocksSocketImpl
>  305744 instances of class java.lang.ref.Finalizer
>  305253 instances of class [Ljava.util.HashMap$Entry;
>  305246 instances of class java.util.HashMap
>  304934 instances of class java.util.Vector
>  304925 instances of class jcifs.smb.SmbComBlankResponse
>  304908 instances of class jcifs.smb.SmbTransport
>  304908 instances of class jcifs.smb.SmbTransport$ServerData
>  304907 instances of class jcifs.UniAddress
>  304907 instances of class jcifs.smb.SmbSession
>  304907 instances of class jcifs.smb.SmbTree
>  302970 instances of class java.net.ConnectException
>  302695 instances of class jcifs.util.transport.TransportException
>
> An outline of the code is here, it's not really relevant, more for
> reference. It just chains an SmbFile opening a server/share, then a
> directory, and finally a file, which is written to using
> SmbFileOutputStream.  Everything is in try {} catch {} finally {} blocks
>  to make sure nothing escapes :-).  This is most important for the
> stream to insure its closed no matter what.
>
> SmbFile targetServer = new SmbFile(machinePath, auth);
>
>  do some stuff
>
> Smbfile targetDirectory =  new SmbFile(smbTargetServer, directory);
>
>  do more stuff
>
> SmbFile smbTargetFile = new SmbFile(smbTargetDir, fileName);
>
>  do even more stuff
>
> SmbFileOutputStream o =null;
>  try {
>     o = new SmbFileOutputStream(smbTargetFile);
>     write();
>  catch {
>  } finally {
>      try {
>        o.close();
>      catch (Exception ex) {
>      }
>  }
>
> ---
>
> Going back to the heap dump, one of the biggest consumers is a
> LinkedList. Drilling into the heap, the entries are from SmbConstants
> and the list contains a reference called CONNECTIONS.  Looking at the
> source file indicates that there is indeed a LinkedList called CONNECTIONS.
>
>      jcifs.smb.SmbConstants
>
>      static final LinkedList CONNECTIONS = new LinkedList();
>
> Hmm, where is this used at?  Doing a quick search for all occurrences of
>  CONNECTIONS on the jcifs code produces only one class, SmbTransport.
>
> All of the references I found were in one method, getSmbTransport()
> [lines 43-64], and it only added to the LinkedList.  I could not find
> any place where the connections are removed from the LinkedList.
> Eventually the list grows to where it consumes enough memory to cause a
> java.lang.OutOfMemoryError condition.  The error will appear faster if
> the jcifs.smb.client.ssnLimit=1 property is set.

Entries in the CONNECTIONS list should be removed at some point but
currently they are not. This is a bug. There should be some expiration
logic. However, it is a minor bug because you really should not set
ssnLimit = 1. Creating a separate connection for each session is just
an enormous waste of resources as you have discovered. With ssnLimit =
1 the CONNECTIONS list does effectively become an object sink (aka
memory leak in Java speak).

Short Answer: Don't set ssnLimit = 1.

Mike

-- 
Michael B Allen
Java Active Directory Integration
http://www.ioplex.com/