[jcifs] Strange issue with SmbFileInputStream
René Liebmann
rene_liebmann at datamatics.de
Wed Feb 10 08:15:58 MST 2010
Hi Mike,
you are absolutely right! The machine where my code is running is member of
a workgroup, but is looking into a domain. Now as I had done the suggested
jcifs settings everything is working fine. Thanks, you have helped me a lot!
Kind regards
René
-----Ursprüngliche Nachricht-----
Von: Michael B Allen [mailto:ioplex at gmail.com]
Gesendet: Samstag, 6. Februar 2010 21:54
An: Rene Liebmann
Cc: jcifs at lists.samba.org
Betreff: Re: [jcifs] Strange issue with SmbFileInputStream
Hi Rene,
If your thread dump is 3000 lines of JCIFS-QueryThread then it sounds
like you have name service problems and query threads are timing out
more slowly than they are being started. If they all have NbtAddress
in them, you could just disable WINS lookups (WINS isn't really used
anymore anyway). It also looks like it might be related to DFS.
Ultimately I suspect the default properties for things like your
domain and name service are totally wrong for the environment the code
is running in. Or you are setting properties that you should not be
(but of course that cannot the case because if you were setting any
JCIFS properties you surely would have said so in your original post).
Try disabling WINS and DFS by setting:
jcifs.resolveOrder=DNS
jcifs.smb.client.dfs.disabled=true
This will disable a lot of code that seems to be related to your problem.
Mike
On Fri, Feb 5, 2010 at 3:43 AM, René Liebmann
<rene_liebmann at datamatics.de> wrote:
> Hi Mike,
>
> sorry I have to rewarm this issue. I have now created a command line
> application where I removed all my memory eaters. Now I come into a
> situation where copying my files goes again wrong and ends up with an
OOME.
> During the copy process I have created a thread dump and it is more than
> 3000 lines long. Therefore I paste here only the last part, as the first
> Threads are always the same but of course with a different tid. So it
seems
> to me, that jcifs.UniAddress$QueryThread.run is called many 100 times. The
> small files seem to work, but the large files (14 MB and larger) are not
> working.
>
> The same application was running in a different domain without any
problems
> and even 200 MB files were no issue. Any Idea?
>
> "JCIFS-QueryThread: bruker" daemon prio=6 tid=0x3d1eb000 nid=0x4a98 in
> Object.wa
> it() [0x3b3bf000]
> java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> at java.lang.Object.wait(Object.java:485)
> at jcifs.netbios.NbtAddress.checkLookupTable(NbtAddress.java:332)
> - locked <0x11990260> (a java.util.HashMap)
> at jcifs.netbios.NbtAddress.doNameQuery(NbtAddress.java:305)
> at jcifs.netbios.NbtAddress.getByName(NbtAddress.java:422)
> at jcifs.UniAddress$QueryThread.run(UniAddress.java:147)
>
> "JCIFS-QueryThread: bruker" daemon prio=6 tid=0x3d1e9800 nid=0x3060 in
> Object.wa
> it() [0x3b36f000]
> java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> at java.lang.Object.wait(Object.java:485)
> at jcifs.netbios.NbtAddress.checkLookupTable(NbtAddress.java:332)
> - locked <0x11990260> (a java.util.HashMap)
> at jcifs.netbios.NbtAddress.doNameQuery(NbtAddress.java:305)
> at jcifs.netbios.NbtAddress.getByName(NbtAddress.java:422)
> at jcifs.UniAddress$QueryThread.run(UniAddress.java:147)
>
> "Transport1" daemon prio=6 tid=0x3af8ac00 nid=0x3298 runnable [0x3b2cf000]
> java.lang.Thread.State: RUNNABLE
> at java.net.SocketInputStream.socketRead0(Native Method)
> at java.net.SocketInputStream.read(Unknown Source)
> at jcifs.util.transport.Transport.readn(Transport.java:29)
> at jcifs.smb.SmbTransport.peekKey(SmbTransport.java:370)
> at jcifs.util.transport.Transport.loop(Transport.java:100)
> at jcifs.util.transport.Transport.run(Transport.java:265)
> at java.lang.Thread.run(Unknown Source)
>
> "Transport2" daemon prio=6 tid=0x3af73c00 nid=0x35a0 runnable [0x3b19f000]
> java.lang.Thread.State: RUNNABLE
> at java.net.SocketInputStream.socketRead0(Native Method)
> at java.net.SocketInputStream.read(Unknown Source)
> at jcifs.util.transport.Transport.readn(Transport.java:29)
> at jcifs.smb.SmbTransport.peekKey(SmbTransport.java:370)
> at jcifs.util.transport.Transport.loop(Transport.java:100)
> at jcifs.util.transport.Transport.run(Transport.java:265)
> at java.lang.Thread.run(Unknown Source)
>
> "JCIFS-NameServiceClient" daemon prio=6 tid=0x64732400 nid=0x1cf8 runnable
> [0x3b
> 27f000]
> java.lang.Thread.State: RUNNABLE
> at java.net.PlainDatagramSocketImpl.receive0(Native Method)
> - locked <0x119ae390> (a java.net.PlainDatagramSocketImpl)
> at java.net.PlainDatagramSocketImpl.receive(Unknown Source)
> - locked <0x119ae390> (a java.net.PlainDatagramSocketImpl)
> at java.net.DatagramSocket.receive(Unknown Source)
> - locked <0x119ae3d0> (a java.net.DatagramPacket)
> - locked <0x119ae3f0> (a java.net.DatagramSocket)
> at jcifs.netbios.NameServiceClient.run(NameServiceClient.java:180)
> at java.lang.Thread.run(Unknown Source)
>
> "JCIFS-QueryThread: bruker" daemon prio=6 tid=0x3af6d800 nid=0x524c in
> Object.wa
> it() [0x3b22f000]
> java.lang.Thread.State: TIMED_WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> at jcifs.netbios.NameServiceClient.send(NameServiceClient.java:241)
> - locked <0x119ae4d8> (a jcifs.netbios.NameQueryResponse)
> at
> jcifs.netbios.NameServiceClient.getByName(NameServiceClient.java:324)
>
> at jcifs.netbios.NbtAddress.doNameQuery(NbtAddress.java:307)
> at jcifs.netbios.NbtAddress.getByName(NbtAddress.java:422)
> at jcifs.UniAddress$QueryThread.run(UniAddress.java:147)
>
> "Low Memory Detector" daemon prio=6 tid=0x3ab1f400 nid=0x3d84 runnable
> [0x000000
> 00]
> java.lang.Thread.State: RUNNABLE
>
> "CompilerThread0" daemon prio=10 tid=0x3ab19000 nid=0x1360 waiting on
> condition
> [0x00000000]
> java.lang.Thread.State: RUNNABLE
>
> "Attach Listener" daemon prio=10 tid=0x3ab17800 nid=0x4938 runnable
> [0x00000000]
>
> java.lang.Thread.State: RUNNABLE
>
> "Signal Dispatcher" daemon prio=10 tid=0x3ab16400 nid=0x43b0 waiting on
> conditio
> n [0x00000000]
> java.lang.Thread.State: RUNNABLE
>
> "Finalizer" daemon prio=8 tid=0x3ab07400 nid=0x3f10 in Object.wait()
> [0x3ac7f000
> ]
> java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> at java.lang.ref.ReferenceQueue.remove(Unknown Source)
> - locked <0x119ae7b8> (a java.lang.ref.ReferenceQueue$Lock)
> at java.lang.ref.ReferenceQueue.remove(Unknown Source)
> at java.lang.ref.Finalizer$FinalizerThread.run(Unknown Source)
>
> "Reference Handler" daemon prio=10 tid=0x3ab02800 nid=0x2d58 in
> Object.wait() [0
> x3ac2f000]
> java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> at java.lang.Object.wait(Object.java:485)
> at java.lang.ref.Reference$ReferenceHandler.run(Unknown Source)
> - locked <0x119901a0> (a java.lang.ref.Reference$Lock)
>
> "main" prio=6 tid=0x002a6800 nid=0x2dc0 in Object.wait() [0x0090f000]
> java.lang.Thread.State: BLOCKED (on object monitor)
> at java.lang.Object.wait(Native Method)
> at jcifs.smb.SmbTransport.send(SmbTransport.java:610)
> - locked <0x119e8058> (a java.util.HashMap)
> at jcifs.smb.SmbSession.send(SmbSession.java:241)
> - locked <0x119e8080> (a java.lang.Object)
> at jcifs.smb.SmbTree.send(SmbTree.java:111)
> at jcifs.smb.SmbTransport.getDfsReferrals(SmbTransport.java:684)
> at jcifs.smb.Dfs.getTrustedDomains(Dfs.java:64)
> at jcifs.smb.Dfs.resolve(Dfs.java:146)
> - locked <0x119ae878> (a jcifs.smb.Dfs)
> at jcifs.smb.SmbFile.resolveDfs(SmbFile.java:667)
> at jcifs.smb.SmbFile.send(SmbFile.java:727)
> at
> jcifs.smb.SmbFileInputStream.readDirect(SmbFileInputStream.java:181)
> at jcifs.smb.SmbFileInputStream.read(SmbFileInputStream.java:142)
> at jcifs.smb.SmbFileInputStream.read(SmbFileInputStream.java:132)
> at ZipExtractor.copySmbFileTo(ZipExtractor.java:183)
> at ZipExtractor.runZipExtraction(ZipExtractor.java:231)
> at ZipExtractor.<init>(ZipExtractor.java:105)
> at ZipExtractor.main(ZipExtractor.java:453)
>
> "VM Thread" prio=10 tid=0x3aaffc00 nid=0xc94 runnable
>
> "VM Periodic Task Thread" prio=10 tid=0x3ab21c00 nid=0x1258 waiting on
> condition
>
>
> JNI global references: 973
>
> Heap
> def new generation total 245760K, used 51192K [0x02990000, 0x13430000,
> 0x1343
> 0000)
> eden space 218496K, 22% used [0x02990000, 0x05a7c5f8, 0x0fef0000)
> from space 27264K, 4% used [0x11990000, 0x11aa1c18, 0x13430000)
> to space 27264K, 0% used [0x0fef0000, 0x0fef0000, 0x11990000)
> tenured generation total 546176K, used 0K [0x13430000, 0x34990000,
> 0x34990000
> )
> the space 546176K, 0% used [0x13430000, 0x13430000, 0x13430200,
> 0x34990000)
>
> compacting perm gen total 12288K, used 5018K [0x34990000, 0x35590000,
> 0x389900
> 00)
> the space 12288K, 40% used [0x34990000, 0x34e76a90, 0x34e76c00,
> 0x35590000)
> No shared spaces configured.
>
> Exception in thread "main" java.lang.OutOfMemoryErro
> r: unable to create new native thread
> at java.lang.Thread.start0(Native Method)
> at java.lang.Thread.start(Unknown Source)
> at jcifs.UniAddress.lookupServerOrWorkgroup(UniAddress.java:173)
> at jcifs.UniAddress.getAllByName(UniAddress.java:290)
> at jcifs.UniAddress.getByName(UniAddress.java:245)
> at jcifs.smb.Dfs.getTrustedDomains(Dfs.java:62)
> at jcifs.smb.Dfs.resolve(Dfs.java:146)
> at jcifs.smb.SmbFile.resolveDfs(SmbFile.java:667)
> at jcifs.smb.SmbFile.send(SmbFile.java:727)
> at jcifs.smb.SmbFile.close(SmbFile.java:968)
> at jcifs.smb.SmbFile.close(SmbFile.java:974)
> at jcifs.smb.SmbFile.close(SmbFile.java:978)
> at jcifs.smb.SmbFileInputStream.close(SmbFileInputStream.java:104)
> at ZipExtractor.copySmbFileTo(ZipExtractor.java:192)
> at ZipExtractor.runZipExtraction(ZipExtractor.java:231)
> at ZipExtractor.<init>(ZipExtractor.java:105)
> at ZipExtractor.main(ZipExtractor.java:453)
>
>
> René
> -----Ursprüngliche Nachricht-----
> Von: Michael B Allen [mailto:ioplex at gmail.com]
> Gesendet: Donnerstag, 28. Januar 2010 17:00
> An: Rene Liebmann
> Cc: jcifs at lists.samba.org
> Betreff: Re: [jcifs] Strange issue with SmbFileInputStream
>
> On Thu, Jan 28, 2010 at 8:52 AM, René Liebmann
> <rene_liebmann at datamatics.de> wrote:
>> Hi Mike,
>>
>> Thank you for your response. Usually I have wrapped my Java app with an
> EXE.
>> I have changed this now and the behavior is a bit different, but the
> result
>> seems to be the same. The reason is an OutOfMemoryError, which in fact
>> should not be, as I use the options -Xms400m -Xmx400m and during the time
>> when it hangs my app has just 100 M occupied as per TaskManager. I also
>> increased the memory to 800m. Then 2 or 3 files more are getting copied
> and
>> it hangs again. Please find here the dump, which came up in the console:
>>
>> C:\tmp>java -Xms400m -Xmx400m com.rli.fsi.gui.FileSystemInventory
>> Exception in thread "pool-1-thread-2" java.lang.OutOfMemoryError: unable
> to
>> crea
>> te new native thread
>> at java.lang.Thread.start0(Native Method)
>> at java.lang.Thread.start(Unknown Source)
>> at jcifs.UniAddress.lookupServerOrWorkgroup(UniAddress.java:172)
>> at jcifs.UniAddress.getAllByName(UniAddress.java:290)
>> at jcifs.UniAddress.getByName(UniAddress.java:245)
>> at jcifs.smb.Dfs.getTrustedDomains(Dfs.java:62)
>> at jcifs.smb.Dfs.resolve(Dfs.java:167)
>> at jcifs.smb.SmbFile.resolveDfs(SmbFile.java:666)
>> at jcifs.smb.SmbFile.send(SmbFile.java:768)
>> at jcifs.smb.SmbFile.close(SmbFile.java:1009)
>> at jcifs.smb.SmbFile.close(SmbFile.java:1015)
>> at jcifs.smb.SmbFile.close(SmbFile.java:1019)
>> at jcifs.smb.SmbFileInputStream.close(SmbFileInputStream.java:104)
>
>
> Hi Rene,
>
> This is not a thread dump, it is just a stack trace. Your stack trace
> shows two things - 1 the problem is that you are running out of memory
> and 2 JCIFS is somewhat stupidly trying to do a DFS lookup from a call
> to close().
>
> I have made a note of 2 but I do not think it is related to 1 and, in
> general, I rather doubt JCIFS has anything to do with your
> OutOfMemoryError. It probably just seems like it is because JCIFS is
> trying to create a new Thread to do a name service lookup and that
> requires a large chunk of memory. It is more likely that the code that
> you are working on that uses JCIFS is actually leaking objects or
> wasting reasources in some way.
>
> If your application needs -Xms400m -Xmx400m to run, I think you need
> to back up and find a JVM profiler and look at how many objects of
> each type are being created and then reason about how many there
> should be of each type. I have a feeling you'll find that you have an
> object-sink somewhere (aka Java memory leak).
>
> Mike
>
>> -----Ursprüngliche Nachricht-----
>> Von: Michael B Allen [mailto:ioplex at gmail.com]
>> Gesendet: Mittwoch, 27. Januar 2010 18:05
>> An: Rene Liebmann
>> Cc: jcifs at lists.samba.org
>> Betreff: Re: [jcifs] Strange issue with SmbFileInputStream
>>
>> Hi Rene,
>>
>> Wait a while to make sure it is really hanging. If it does not respond
>> after a few minutes, get a thread-dump using Ctrl-Break on Windows or
>> Ctrl-\ on Unix in a console. That will show you exactly where it's
>> stuck.
>>
>> Mike
>>
>> On Wed, Jan 27, 2010 at 11:35 AM, René Liebmann
>> <rene_liebmann at datamatics.de> wrote:
>>> Hi all,
>>>
>>>
>>>
>>> I have a small function which I use to copy a remote file to my local
>>> machine. On almost all environments (Win XP Home) this works fine, but
on
>> a
>>> Windows 2003 Server environment it does only copy a small portion of a
>> file
>>> e.g. if the file is 6 MB large it copies 248 kB. Then it seems that it
>> never
>>> leaves the while block and my program hangs. It does not throw any
>>> exception. What am I doing wrong?
>>>
>>>
>>>
>>> private void copySmbFileTo(SmbFile f, String location) throws
IOException
>> {
>>>
>>> logger.debug(name + " enter function copySmbFileTo
>> "+f.getUncPath()
>>> + " to "+location);
>>>
>>>
>>>
>>> SmbFileInputStream istr = null;
>>>
>>> OutputStream ostr = null;
>>>
>>> try { // 1
>>>
>>> istr = new SmbFileInputStream(f);
>>>
>>> ostr = new FileOutputStream(location);
>>>
>>> byte[] buf1 = new byte[1024];
>>>
>>> int len1;
>>>
>>> while((len1 = istr.read(buf1)) > 0){
>>>
>>> ostr.write(buf1, 0, len1);
>>>
>>> }
>>>
>>> ostr.flush();
>>>
>>> } catch(Exception ex){
>>>
>>> logger.error("", ex);
>>>
>>> } finally{
>>>
>>> try { // 2
>>>
>>> if(istr!=null) istr.close();
>>>
>>> if(ostr!=null) ostr.close();
>>>
>>> } catch(Exception ex){
>>>
>>> logger.error("Major Error Releasing Streams:", ex);
>>>
>>>
>>>
>>> } // End try catch number 2
>>>
>>> } // End try catch finally number 1
>>>
>>> logger.debug(name + " leave function copySmbFileTo");
>>>
>>>
>>>
>>> }
>>>
>>>
>>>
>>> Thanks for your ideas in advance
>>>
>>>
>>>
>>> René
>>>
>>>
>>
>>
>>
>> --
>> Michael B Allen
>> Java Active Directory Integration
>> http://www.ioplex.com/
>>
>>
>
>
>
> --
> Michael B Allen
> Java Active Directory Integration
> http://www.ioplex.com/
>
>
--
Michael B Allen
Java Active Directory Integration
http://www.ioplex.com/
More information about the jCIFS
mailing list