[jcifs] listFiles produce multiple entrys for some files..

[michael.studer at sisa.ch]

Hi again

I figured out, the problem seems to occur only on AS400 shares. I did not 
face it on windows servers.

Again, i try to explain the problem. When i use the "smbFile.listFiles()" 
method on a file, which is located inside a AS400 share, the amount of 
returned files is multiple times the actual amount it should return.
For example: 2 Files in the share returns me 3 Files, where one File is 
twice in the collection. This would not be such a big deal, its just 
getting nasty when you have folders with a lot of Files inside. 
While i tested, i faced some folders with arround 5000+ Files. This 
results in arround 250'000 returned SMBFile Objects, with a total used 
amount of like 500+MB of memory and a runtime out of memory.

Heres the full code for my directory reading method(s):

        public HashSet<String> getFilesList(String relativePath, boolean 
recursive, boolean includeFolders) throws MalformedURLException, 
SmbException, ExsException {
 
                HashSet<String> collection = new HashSet<String>();
                HashSet<String> unreadable = new HashSet<String>();
 
 getFilesList(relativePath,recursive,includeFolders,false,collection, 
unreadable); 
 
                return collection;
 
        }
 
        private void getFilesList(String path, boolean recursive, boolean 
includeFolders, boolean isFullPath, HashSet<String> collection, 
HashSet<String> unreadable) throws MalformedURLException, SmbException, 
ExsException {

                if(path == null) {
                        path = "";
                } 
 
                if(path.trim().length() > 0 && (!path.trim().endsWith("/") 
&& !path.trim().endsWith("\\"))) {
                        path = path + "/";
                }
 
                String url = "";
 
                if(isFullPath) {
                        url = path;
                } else {
                        url = generateURLString(path); 
                }

                SmbFile smbFolder = new SmbFile(url);

                if(!smbFolder.exists() || !smbFolder.isDirectory()) {
                        throw new ExsException("Error: Folder cannot be 
read from smb share: " + smbFolder.getPath());
                }
 
                if(!smbFolder.canRead()) {
                        unreadable.add(smbFolder.getPath());
                        return;
                }

                // this helps preventing heap overflows but it wont 
prevent it when to many files at once are opend
                HashSet<SmbFile> uniqueList = new HashSet<SmbFile>();
                for(SmbFile currentSmbFile : smbFolder.listFiles()) {
                        uniqueList.add(currentSmbFile);
                }
 
                for(SmbFile currentSmbFile : uniqueList) {
 
                        String currentPath = currentSmbFile.getPath();
 
                        if(currentSmbFile.isFile()) {
                                currentSmbFile = null;
                                collection.add(currentPath);
 
                        } else if(currentSmbFile.isDirectory()) {
                                int size = 0;
                                try {
                                        String[] list = 
currentSmbFile.list();
                                        size = list.length;
                                        list = null;
                                } catch(Throwable t) {
                                        continue;
                                }
 
                                currentSmbFile = null;
                                if(includeFolders) {
                                        collection.add(currentPath);
                                }
                                if(recursive) {
                                        if(size > 0) {
                                                getFilesList(currentPath, 
recursive, includeFolders, true, collection, unreadable);
                                        }
                                } 
                        } 
                }
 
                uniqueList.clear();
                uniqueList = null;
        }


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.samba.org/pipermail/jcifs/attachments/20091012/609328ca/attachment.html>