[jcifs] java.lang.OutOfMemoryError: Java heap space

Edwin Portscher eportscher at meridianemr.com
Thu Sep 6 20:16:02 GMT 2007


Thanks

I am processing one file at a time.
My function returns a SmbFile[] and that seems to be what is causing the outofmemory
Exceptions. The SmbFile[] is too big. I get the out of memory way before I start reading any of the files.

ed

-----Original Message-----
From: Jake Goulding [mailto:goulding at vivisimo.com] 
Sent: Thursday, September 06, 2007 3:48 PM
To: Edwin Portscher
Cc: 'JCIFS List'
Subject: Re: [jcifs] java.lang.OutOfMemoryError: Java heap space

Simply have your filter have an Array/List, whatever that holds 100
files. In the filter function, simply store each file in it, and when
you hit 100, do your processing. I'm not sure why you can't simply
process each as it comes back, then you only need to have that File in
memory, plus whatever you need to process it. Be sure to return false
from the filter method, so the library doesn't keep it around after you
are done processing it.

However, it sounds like you may have more issues than just jCIFS taking
up memory. In my case, I had to process a directory with 10K-100K files
before I ran out of memory, and that was with a reasonable heap size,
around 160M.



Edwin Portscher wrote:
> Looking at the api there is no method to pull say the first 100 files alphabetically
> It either all or nothing
> 
> ed
> 
> -----Original Message-----
> From: Jake Goulding [mailto:goulding at vivisimo.com] 
> Sent: Thursday, September 06, 2007 3:02 PM
> To: Edwin Portscher
> Cc: JCIFS List
> Subject: Re: [jcifs] java.lang.OutOfMemoryError: Java heap space
> 
> Not sure, according to the API [1], they are some kind of attributes.
> For my case, I wanted all the files, so I directly implemented
> SmbFileFilter, and did operations simply based on the names.
> 
> -Jake
> 
> [1] http://jcifs.samba.org/src/docs/api/jcifs/smb/DosFileFilter.html
> 
> 
> Edwin Portscher wrote:
>> Thanks,
>>
>> Quick question in largelistfiles
>>
>> public LargeListFiles() {
>>         super("*", 0xFFFF);
>>
>> what is 0xFFFF specifying?
>>
>> ed
>>
>> -----Original Message-----
>> From: Jake Goulding [mailto:goulding at vivisimo.com] 
>> Sent: Thursday, September 06, 2007 2:28 PM
>> To: Edwin Portscher
>> Cc: jcifs at lists.samba.org
>> Subject: Re: [jcifs] java.lang.OutOfMemoryError: Java heap space
>>
>> Edwin:
>>
>> I had the same problem. Mike suggested implementing a SmbFileFilter that
>> you can use while listing the directory. It will be passed each file as
>> the server reads it. returning false will cause it to not be added to
>> the result list that listFiles() returns. The only trick is you have to
>> be quick while processing each file, or else you can cause network timeouts.
>>
>> Edwin Portscher wrote:
>>> Hello all,
>>>
>>>  
>>>
>>> Question when doing a SmbFile[] = smb.listfiles();
>>>
>>>  
>>>
>>> I am getting java.lang.OutOfMemoryError: Java heap space when the
>>> directory contains a lot of files
>>>
>>>  
>>>
>>> Any ideas?
>>>
>>>  
>>>
>>> Thanks
>>>
>>> ed
>>>
>>
> 
> 



More information about the jcifs mailing list