[jcifs] listFiles produce multiple entrys for some files..
Clapham, Paul
pclapham at core-mark.com
Mon Oct 26 09:29:25 MDT 2009
I have access to AS400 and I'm willing to work on this problem. I asked michael.studer (accidentally off-list) to send me a short compilable program which I could modify and test. So far I just got a large lot of code which was way too much to deal with. Seems to me that a program to test this should be less than 50 lines of code.
PC2
-----Original Message-----
From: jcifs-bounces at lists.samba.org [mailto:jcifs-bounces at lists.samba.org] On Behalf Of Michael B Allen
Sent: October 26, 2009 08:25
To: michael.studer at sisa.ch
Cc: jcifs at lists.samba.org
Subject: Re: [jcifs] listFiles produce multiple entrys for some files..
You could send me a packet capture. But I don't have access to AS400 and I don't really have "Free" time so unless it's an obvious fix this could sit for a while. But I know JCIFS has worked with mainframes before so I have to wonder why this is showing up now. Is AS400 really old or really new or unmaintained?
Mike
On Mon, Oct 12, 2009 at 3:17 AM, <michael.studer at sisa.ch> wrote:
>
> Hi again
>
> I figured out, the problem seems to occur only on AS400 shares. I did
> not face it on windows servers.
>
> Again, i try to explain the problem. When i use the "smbFile.listFiles()"
> method on a file, which is located inside a AS400 share, the amount of
> returned files is multiple times the actual amount it should return.
> For example: 2 Files in the share returns me 3 Files, where one File
> is twice in the collection. This would not be such a big deal, its
> just getting nasty when you have folders with a lot of Files inside.
> While i tested, i faced some folders with arround 5000+ Files. This
> results in arround 250'000 returned SMBFile Objects, with a total used
> amount of like 500+MB of memory and a runtime out of memory.
>
> Heres the full code for my directory reading method(s):
>
> public HashSet<String> getFilesList(String relativePath,
> boolean recursive, boolean includeFolders) throws
> MalformedURLException, SmbException, ExsException {
>
> HashSet<String> collection = new HashSet<String>();
> HashSet<String> unreadable = new HashSet<String>();
>
>
> getFilesList(relativePath,recursive,includeFolders,false,collection,
> unreadable);
>
> return collection;
>
> }
>
> private void getFilesList(String path, boolean recursive,
> boolean includeFolders, boolean isFullPath, HashSet<String>
> collection, HashSet<String> unreadable) throws MalformedURLException,
> SmbException, ExsException {
>
> if(path == null) {
> path = "";
> }
>
> if(path.trim().length() > 0 &&
> (!path.trim().endsWith("/") && !path.trim().endsWith("\\"))) {
> path = path + "/";
> }
>
> String url = "";
>
> if(isFullPath) {
> url = path;
> } else {
> url = generateURLString(path);
> }
>
> SmbFile smbFolder = new SmbFile(url);
>
> if(!smbFolder.exists() || !smbFolder.isDirectory()) {
> throw new ExsException("Error: Folder cannot
> be read from smb share: " + smbFolder.getPath());
> }
>
> if(!smbFolder.canRead()) {
> unreadable.add(smbFolder.getPath());
> return;
> }
>
> // this helps preventing heap overflows but it wont
> prevent it when to many files at once are opend
> HashSet<SmbFile> uniqueList = new HashSet<SmbFile>();
> for(SmbFile currentSmbFile : smbFolder.listFiles()) {
> uniqueList.add(currentSmbFile);
> }
>
> for(SmbFile currentSmbFile : uniqueList) {
>
> String currentPath = currentSmbFile.getPath();
>
> if(currentSmbFile.isFile()) {
> currentSmbFile = null;
> collection.add(currentPath);
>
> } else if(currentSmbFile.isDirectory()) {
> int size = 0;
> try {
> String[] list =
> currentSmbFile.list();
> size = list.length;
> list = null;
> } catch(Throwable t) {
> continue;
> }
>
> currentSmbFile = null;
> if(includeFolders) {
> collection.add(currentPath);
> }
> if(recursive) {
> if(size > 0) {
>
> getFilesList(currentPath, recursive, includeFolders, true, collection,
> unreadable);
> }
> }
> }
> }
>
> uniqueList.clear();
> uniqueList = null;
> }
>
>
>
--
Michael B Allen
Java Active Directory Integration
http://www.ioplex.com/
More information about the jCIFS
mailing list