[jcifs] Strange issue with SmbFileInputStream

René Liebmann rene_liebmann at datamatics.de
Fri Feb 5 01:43:16 MST 2010


Hi Mike,

sorry I have to rewarm this issue. I have now created a command line
application where I removed all my memory eaters. Now I come into a
situation where copying my files goes again wrong and ends up with an OOME.
During the copy process I have created a thread dump and it is more than
3000 lines long. Therefore I paste here only the last part, as the first
Threads are always the same but of course with a different tid. So it seems
to me, that jcifs.UniAddress$QueryThread.run is called many 100 times. The
small files seem to work, but the large files (14 MB and larger) are not
working.  

The same application was running in a different domain without any problems
and even 200 MB files were no issue. Any Idea?

"JCIFS-QueryThread: bruker" daemon prio=6 tid=0x3d1eb000 nid=0x4a98 in
Object.wa
it() [0x3b3bf000]
   java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.lang.Object.wait(Object.java:485)
        at jcifs.netbios.NbtAddress.checkLookupTable(NbtAddress.java:332)
        - locked <0x11990260> (a java.util.HashMap)
        at jcifs.netbios.NbtAddress.doNameQuery(NbtAddress.java:305)
        at jcifs.netbios.NbtAddress.getByName(NbtAddress.java:422)
        at jcifs.UniAddress$QueryThread.run(UniAddress.java:147)

"JCIFS-QueryThread: bruker" daemon prio=6 tid=0x3d1e9800 nid=0x3060 in
Object.wa
it() [0x3b36f000]
   java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.lang.Object.wait(Object.java:485)
        at jcifs.netbios.NbtAddress.checkLookupTable(NbtAddress.java:332)
        - locked <0x11990260> (a java.util.HashMap)
        at jcifs.netbios.NbtAddress.doNameQuery(NbtAddress.java:305)
        at jcifs.netbios.NbtAddress.getByName(NbtAddress.java:422)
        at jcifs.UniAddress$QueryThread.run(UniAddress.java:147)

"Transport1" daemon prio=6 tid=0x3af8ac00 nid=0x3298 runnable [0x3b2cf000]
   java.lang.Thread.State: RUNNABLE
        at java.net.SocketInputStream.socketRead0(Native Method)
        at java.net.SocketInputStream.read(Unknown Source)
        at jcifs.util.transport.Transport.readn(Transport.java:29)
        at jcifs.smb.SmbTransport.peekKey(SmbTransport.java:370)
        at jcifs.util.transport.Transport.loop(Transport.java:100)
        at jcifs.util.transport.Transport.run(Transport.java:265)
        at java.lang.Thread.run(Unknown Source)

"Transport2" daemon prio=6 tid=0x3af73c00 nid=0x35a0 runnable [0x3b19f000]
   java.lang.Thread.State: RUNNABLE
        at java.net.SocketInputStream.socketRead0(Native Method)
        at java.net.SocketInputStream.read(Unknown Source)
        at jcifs.util.transport.Transport.readn(Transport.java:29)
        at jcifs.smb.SmbTransport.peekKey(SmbTransport.java:370)
        at jcifs.util.transport.Transport.loop(Transport.java:100)
        at jcifs.util.transport.Transport.run(Transport.java:265)
        at java.lang.Thread.run(Unknown Source)

"JCIFS-NameServiceClient" daemon prio=6 tid=0x64732400 nid=0x1cf8 runnable
[0x3b
27f000]
   java.lang.Thread.State: RUNNABLE
        at java.net.PlainDatagramSocketImpl.receive0(Native Method)
        - locked <0x119ae390> (a java.net.PlainDatagramSocketImpl)
        at java.net.PlainDatagramSocketImpl.receive(Unknown Source)
        - locked <0x119ae390> (a java.net.PlainDatagramSocketImpl)
        at java.net.DatagramSocket.receive(Unknown Source)
        - locked <0x119ae3d0> (a java.net.DatagramPacket)
        - locked <0x119ae3f0> (a java.net.DatagramSocket)
        at jcifs.netbios.NameServiceClient.run(NameServiceClient.java:180)
        at java.lang.Thread.run(Unknown Source)

"JCIFS-QueryThread: bruker" daemon prio=6 tid=0x3af6d800 nid=0x524c in
Object.wa
it() [0x3b22f000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at jcifs.netbios.NameServiceClient.send(NameServiceClient.java:241)
        - locked <0x119ae4d8> (a jcifs.netbios.NameQueryResponse)
        at
jcifs.netbios.NameServiceClient.getByName(NameServiceClient.java:324)

        at jcifs.netbios.NbtAddress.doNameQuery(NbtAddress.java:307)
        at jcifs.netbios.NbtAddress.getByName(NbtAddress.java:422)
        at jcifs.UniAddress$QueryThread.run(UniAddress.java:147)

"Low Memory Detector" daemon prio=6 tid=0x3ab1f400 nid=0x3d84 runnable
[0x000000
00]
   java.lang.Thread.State: RUNNABLE

"CompilerThread0" daemon prio=10 tid=0x3ab19000 nid=0x1360 waiting on
condition
[0x00000000]
   java.lang.Thread.State: RUNNABLE

"Attach Listener" daemon prio=10 tid=0x3ab17800 nid=0x4938 runnable
[0x00000000]

   java.lang.Thread.State: RUNNABLE

"Signal Dispatcher" daemon prio=10 tid=0x3ab16400 nid=0x43b0 waiting on
conditio
n [0x00000000]
   java.lang.Thread.State: RUNNABLE

"Finalizer" daemon prio=8 tid=0x3ab07400 nid=0x3f10 in Object.wait()
[0x3ac7f000
]
   java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.lang.ref.ReferenceQueue.remove(Unknown Source)
        - locked <0x119ae7b8> (a java.lang.ref.ReferenceQueue$Lock)
        at java.lang.ref.ReferenceQueue.remove(Unknown Source)
        at java.lang.ref.Finalizer$FinalizerThread.run(Unknown Source)

"Reference Handler" daemon prio=10 tid=0x3ab02800 nid=0x2d58 in
Object.wait() [0
x3ac2f000]
   java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.lang.Object.wait(Object.java:485)
        at java.lang.ref.Reference$ReferenceHandler.run(Unknown Source)
        - locked <0x119901a0> (a java.lang.ref.Reference$Lock)

"main" prio=6 tid=0x002a6800 nid=0x2dc0 in Object.wait() [0x0090f000]
   java.lang.Thread.State: BLOCKED (on object monitor)
        at java.lang.Object.wait(Native Method)
        at jcifs.smb.SmbTransport.send(SmbTransport.java:610)
        - locked <0x119e8058> (a java.util.HashMap)
        at jcifs.smb.SmbSession.send(SmbSession.java:241)
        - locked <0x119e8080> (a java.lang.Object)
        at jcifs.smb.SmbTree.send(SmbTree.java:111)
        at jcifs.smb.SmbTransport.getDfsReferrals(SmbTransport.java:684)
        at jcifs.smb.Dfs.getTrustedDomains(Dfs.java:64)
        at jcifs.smb.Dfs.resolve(Dfs.java:146)
        - locked <0x119ae878> (a jcifs.smb.Dfs)
        at jcifs.smb.SmbFile.resolveDfs(SmbFile.java:667)
        at jcifs.smb.SmbFile.send(SmbFile.java:727)
        at
jcifs.smb.SmbFileInputStream.readDirect(SmbFileInputStream.java:181)
        at jcifs.smb.SmbFileInputStream.read(SmbFileInputStream.java:142)
        at jcifs.smb.SmbFileInputStream.read(SmbFileInputStream.java:132)
        at ZipExtractor.copySmbFileTo(ZipExtractor.java:183)
        at ZipExtractor.runZipExtraction(ZipExtractor.java:231)
        at ZipExtractor.<init>(ZipExtractor.java:105)
        at ZipExtractor.main(ZipExtractor.java:453)

"VM Thread" prio=10 tid=0x3aaffc00 nid=0xc94 runnable

"VM Periodic Task Thread" prio=10 tid=0x3ab21c00 nid=0x1258 waiting on
condition


JNI global references: 973

Heap
 def new generation   total 245760K, used 51192K [0x02990000, 0x13430000,
0x1343
0000)
  eden space 218496K,  22% used [0x02990000, 0x05a7c5f8, 0x0fef0000)
  from space 27264K,   4% used [0x11990000, 0x11aa1c18, 0x13430000)
  to   space 27264K,   0% used [0x0fef0000, 0x0fef0000, 0x11990000)
 tenured generation   total 546176K, used 0K [0x13430000, 0x34990000,
0x34990000
)
   the space 546176K,   0% used [0x13430000, 0x13430000, 0x13430200,
0x34990000)

 compacting perm gen  total 12288K, used 5018K [0x34990000, 0x35590000,
0x389900
00)
   the space 12288K,  40% used [0x34990000, 0x34e76a90, 0x34e76c00,
0x35590000)
No shared spaces configured.

Exception in thread "main" java.lang.OutOfMemoryErro
r: unable to create new native thread
        at java.lang.Thread.start0(Native Method)
        at java.lang.Thread.start(Unknown Source)
        at jcifs.UniAddress.lookupServerOrWorkgroup(UniAddress.java:173)
        at jcifs.UniAddress.getAllByName(UniAddress.java:290)
        at jcifs.UniAddress.getByName(UniAddress.java:245)
        at jcifs.smb.Dfs.getTrustedDomains(Dfs.java:62)
        at jcifs.smb.Dfs.resolve(Dfs.java:146)
        at jcifs.smb.SmbFile.resolveDfs(SmbFile.java:667)
        at jcifs.smb.SmbFile.send(SmbFile.java:727)
        at jcifs.smb.SmbFile.close(SmbFile.java:968)
        at jcifs.smb.SmbFile.close(SmbFile.java:974)
        at jcifs.smb.SmbFile.close(SmbFile.java:978)
        at jcifs.smb.SmbFileInputStream.close(SmbFileInputStream.java:104)
        at ZipExtractor.copySmbFileTo(ZipExtractor.java:192)
        at ZipExtractor.runZipExtraction(ZipExtractor.java:231)
        at ZipExtractor.<init>(ZipExtractor.java:105)
        at ZipExtractor.main(ZipExtractor.java:453)


René
-----Ursprüngliche Nachricht-----
Von: Michael B Allen [mailto:ioplex at gmail.com] 
Gesendet: Donnerstag, 28. Januar 2010 17:00
An: Rene Liebmann
Cc: jcifs at lists.samba.org
Betreff: Re: [jcifs] Strange issue with SmbFileInputStream

On Thu, Jan 28, 2010 at 8:52 AM, René Liebmann
<rene_liebmann at datamatics.de> wrote:
> Hi Mike,
>
> Thank you for your response. Usually I have wrapped my Java app with an
EXE.
> I have changed this now and the behavior is a bit different, but the
result
> seems to be the same. The reason is an OutOfMemoryError, which in fact
> should not be, as I use the options -Xms400m -Xmx400m and during the time
> when it hangs my app has just 100 M occupied as per TaskManager. I also
> increased the memory to 800m. Then 2 or 3 files more are getting copied
and
> it hangs again. Please find here the dump, which came up in the console:
>
> C:\tmp>java -Xms400m -Xmx400m com.rli.fsi.gui.FileSystemInventory
> Exception in thread "pool-1-thread-2" java.lang.OutOfMemoryError: unable
to
> crea
> te new native thread
>        at java.lang.Thread.start0(Native Method)
>        at java.lang.Thread.start(Unknown Source)
>        at jcifs.UniAddress.lookupServerOrWorkgroup(UniAddress.java:172)
>        at jcifs.UniAddress.getAllByName(UniAddress.java:290)
>        at jcifs.UniAddress.getByName(UniAddress.java:245)
>        at jcifs.smb.Dfs.getTrustedDomains(Dfs.java:62)
>        at jcifs.smb.Dfs.resolve(Dfs.java:167)
>        at jcifs.smb.SmbFile.resolveDfs(SmbFile.java:666)
>        at jcifs.smb.SmbFile.send(SmbFile.java:768)
>        at jcifs.smb.SmbFile.close(SmbFile.java:1009)
>        at jcifs.smb.SmbFile.close(SmbFile.java:1015)
>        at jcifs.smb.SmbFile.close(SmbFile.java:1019)
>        at jcifs.smb.SmbFileInputStream.close(SmbFileInputStream.java:104)


Hi Rene,

This is not a thread dump, it is just a stack trace. Your stack trace
shows two things - 1 the problem is that you are running out of memory
and 2 JCIFS is somewhat stupidly trying to do a DFS lookup from a call
to close().

I have made a note of 2 but I do not think it is related to 1 and, in
general, I rather doubt JCIFS has anything to do with your
OutOfMemoryError. It probably just seems like it is because JCIFS is
trying to create a new Thread to do a name service lookup and that
requires a large chunk of memory. It is more likely that the code that
you are working on that uses JCIFS is actually leaking objects or
wasting reasources in some way.

If your application needs -Xms400m -Xmx400m to run, I think you need
to back up and find a JVM profiler and look at how many objects of
each type are being created and then reason about how many there
should be of each type. I have a feeling you'll find that you have an
object-sink somewhere (aka Java memory leak).

Mike

> -----Ursprüngliche Nachricht-----
> Von: Michael B Allen [mailto:ioplex at gmail.com]
> Gesendet: Mittwoch, 27. Januar 2010 18:05
> An: Rene Liebmann
> Cc: jcifs at lists.samba.org
> Betreff: Re: [jcifs] Strange issue with SmbFileInputStream
>
> Hi Rene,
>
> Wait a while to make sure it is really hanging. If it does not respond
> after a few minutes, get a thread-dump using Ctrl-Break on Windows or
> Ctrl-\ on Unix in a console. That will show you exactly where it's
> stuck.
>
> Mike
>
> On Wed, Jan 27, 2010 at 11:35 AM, René Liebmann
> <rene_liebmann at datamatics.de> wrote:
>> Hi all,
>>
>>
>>
>> I have a small function which I use to copy a remote file to my local
>> machine. On almost all environments (Win XP Home) this works fine, but on
> a
>> Windows 2003 Server environment it does only copy a small portion of a
> file
>> e.g. if the file is 6 MB large it copies 248 kB. Then it seems that it
> never
>> leaves the while block and my program hangs. It does not throw any
>> exception. What am I doing wrong?
>>
>>
>>
>> private void copySmbFileTo(SmbFile f, String location) throws IOException
> {
>>
>>         logger.debug(name + " enter function copySmbFileTo
> "+f.getUncPath()
>> + " to "+location);
>>
>>
>>
>>         SmbFileInputStream istr = null;
>>
>>         OutputStream ostr = null;
>>
>>         try {  // 1
>>
>>             istr = new SmbFileInputStream(f);
>>
>>             ostr = new FileOutputStream(location);
>>
>>             byte[] buf1 = new byte[1024];
>>
>>             int len1;
>>
>>             while((len1 = istr.read(buf1)) > 0){
>>
>>                 ostr.write(buf1, 0, len1);
>>
>>             }
>>
>>             ostr.flush();
>>
>>         } catch(Exception ex){
>>
>>             logger.error("", ex);
>>
>>         } finally{
>>
>>           try { // 2
>>
>>             if(istr!=null)  istr.close();
>>
>>             if(ostr!=null)  ostr.close();
>>
>>           } catch(Exception ex){
>>
>>               logger.error("Major Error Releasing Streams:", ex);
>>
>>
>>
>>           } // End try catch number 2
>>
>>         } // End try catch finally number 1
>>
>>         logger.debug(name + " leave function copySmbFileTo");
>>
>>
>>
>>     }
>>
>>
>>
>> Thanks for your ideas in advance
>>
>>
>>
>> René
>>
>>
>
>
>
> --
> Michael B Allen
> Java Active Directory Integration
> http://www.ioplex.com/
>
>



-- 
Michael B Allen
Java Active Directory Integration
http://www.ioplex.com/



More information about the jCIFS mailing list