[jcifs] Hundreds of threads while listing directory

Christopher R. Hertel crh at ubiqx.mn.org
Fri Jun 4 16:49:42 MDT 2010


Michael B Allen wrote:
> Hi Peter,
> 
> Those are name lookup threads. Once upon a time there was something
> called WINS and NetBIOS that was used for hostname and "workgroup"
> lookups. But it's pretty much non-existant unless you have a really
> old network.

I wish that were true.  Having just spent two years working with (not for!)
Microsoft, I can tell you that the NBT layer is still very widely used.  It
is particularly common in small business and home networks, where several
products require it.  (See any $60 NAS device from China... there are lots
and lots of those.)  Microsoft still supports WINS on campus (which, I
suppose, qualifies as a really old network).

That's a general statement, however, and may not be relevant to the
immediate problem at hand.

Chris -)-----

> Try setting:
> 
>   jcifs.resolveOrder = DNS
> 
> Name resolution properties are described here:
> 
>   http://jcifs.samba.org/src/docs/resolver.html
> 
> So by removing WINS and BCAST from the list, I think it should stop
> all of the WINS and NetBIOS stuff which stop the code path that is
> trying to use threads to lookup names.
> 
> Mike
> 
> On Thu, Jun 3, 2010 at 5:01 AM, Peter Kossek <pkossek at aristos-online.de> wrote:
>> Hello,
>>
>>
>>
>> I just started using jCIFS a couple of days before, but now I'm experiencing
>> some problems. My application throws an OOME while trying to list the
>> content of a directory.
>>
>>
>>
>> The exception message:
>>
>>
>>
>> Exception in thread "AWT-EventQueue-0" java.lang.OutOfMemoryError: unable to
>> create new native thread
>>
>>         at java.lang.Thread.start0(Native Method)
>>
>>         at java.lang.Thread.start(Thread.java:597)
>>
>>         at jcifs.UniAddress.lookupServerOrWorkgroup(UniAddress.java:173)
>>
>>         at jcifs.UniAddress.getAllByName(UniAddress.java:290)
>>
>>         at jcifs.UniAddress.getByName(UniAddress.java:245)
>>
>>         at jcifs.smb.Dfs.getTrustedDomains(Dfs.java:62)
>>
>>         at jcifs.smb.Dfs.resolve(Dfs.java:167)
>>
>>         at jcifs.smb.SmbFile.resolveDfs(SmbFile.java:666)
>>
>>         at jcifs.smb.SmbFile.send(SmbFile.java:768)
>>
>>         at jcifs.smb.SmbFile.doFindFirstNext(SmbFile.java:2023)
>>
>>         at jcifs.smb.SmbFile.doEnum(SmbFile.java:1731)
>>
>>         at jcifs.smb.SmbFile.listFiles(SmbFile.java:1708)
>>
>>         at jcifs.smb.SmbFile.listFiles(SmbFile.java:1641)
>>
>>         at as_file.as_smb_lister.findFile(as_smb_lister.java:140)
>>
>>         ...
>>
>>
>>
>> The code for finding the file:
>>
>>
>>
>> public static SmbFile[] findFile(SmbFile startFolder, String filename) {
>>
>>         ArrayList<SmbFile> files = new ArrayList<SmbFile>();
>>
>>         ArrayList<SmbFile> foldersToWalk = new ArrayList<SmbFile>();
>>
>>
>>
>>         try {
>>
>>             if (startFolder.isDirectory()) {
>>
>>                 foldersToWalk.add(startFolder);
>>
>>             }
>>
>>             while (foldersToWalk.size() > 0) {
>>
>>                 SmbFile lookInto = foldersToWalk.remove(0);
>>
>>                 System.out.println("walking " + lookInto.getPath());
>>
>>                 try {
>>
>>                     SmbFile[] found = lookInto.listFiles(); // line 140
>>
>>                     for (int i = 0; i < found.length; i++) {
>>
>>                         if (found[i].getName().endsWith("/")) {
>>
>>                             foldersToWalk.add(found[i]);
>>
>>                         } else {
>>
>>                             if (found[i].getName().equals(filename)) {
>>
>>                                 files.add(found[i]);
>>
>>                             }
>>
>>                         }
>>
>>                     }
>>
>>                 } catch (SmbException ex) {
>>
>>                     System.out.println(ex.getMessage());
>>
>>                 }
>>
>>             }
>>
>>         } catch (SmbException ex) {
>>
>>             System.out.println(ex.getMessage());
>>
>>         }
>>
>>         SmbFile[] result = new SmbFile[files.size()];
>>
>>         for (int i = 0; i < files.size(); i++) {
>>
>>             result[i] = files.get(i);
>>
>>         }
>>
>>
>>
>>         SmbFile[] found = new SmbFile[files.size()];
>>
>>         for (int i = 0; i < files.size(); i++) {
>>
>>             found[i] = files.get(i);
>>
>>         }
>>
>>         return found;
>>
>>     }
>>
>>
>>
>>
>>
>>
>>
>> The directory it's trying to list is smb://KOSSEK/C$/Windows/winsxs/Backup/
>> which is known to contain lots of files, 3256 files in this case.
>>
>> I noticed that there is a peak of numbers of threads, up to more than 2000
>> threads (QueryThreads) at once that seem to bring the VM to its limits.
>> Maybe I made a mistake in my code that forces this massive creation of
>> threads, but I just can't find it. What can I do to fix this?
>>
>>
>>
>> Thanks in advance for your help.
>>
>>
>>
>> Best regards - Peter
> 
> 
> 

-- 
"Implementing CIFS - the Common Internet FileSystem" ISBN: 013047116X
Samba Team -- http://www.samba.org/     -)-----   Christopher R. Hertel
jCIFS Team -- http://jcifs.samba.org/   -)-----   ubiqx development, uninq.
ubiqx Team -- http://www.ubiqx.org/     -)-----   crh at ubiqx.mn.org
OnLineBook -- http://ubiqx.org/cifs/    -)-----   crh at ubiqx.org


More information about the jCIFS mailing list