[jcifs] listFiles produce multiple entrys for some files..

[Michael B Allen]

You could send me a packet capture. But I don't have access to AS400
and I don't really have "Free" time so unless it's an obvious fix this
could sit for a while. But I know JCIFS has worked with mainframes
before so I have to wonder why this is showing up now. Is AS400 really
old or really new or unmaintained?

Mike

On Mon, Oct 12, 2009 at 3:17 AM,  <michael.studer at sisa.ch> wrote:
>
> Hi again
>
> I figured out, the problem seems to occur only on AS400 shares. I did not
> face it on windows servers.
>
> Again, i try to explain the problem. When i use the "smbFile.listFiles()"
> method on a file, which is located inside a AS400 share, the amount of
> returned files is multiple times the actual amount it should return.
> For example: 2 Files in the share returns me 3 Files, where one File is
> twice in the collection. This would not be such a big deal, its just getting
> nasty when you have folders with a lot of Files inside.
> While i tested, i faced some folders with arround 5000+ Files. This results
> in arround 250'000 returned SMBFile Objects, with a total used amount of
> like 500+MB of memory and a runtime out of memory.
>
> Heres the full code for my directory reading method(s):
>
>         public HashSet<String> getFilesList(String relativePath, boolean
> recursive, boolean includeFolders) throws MalformedURLException,
> SmbException, ExsException {
>
>                 HashSet<String> collection = new HashSet<String>();
>                 HashSet<String> unreadable = new HashSet<String>();
>
>
> getFilesList(relativePath,recursive,includeFolders,false,collection,
> unreadable);
>
>                 return collection;
>
>         }
>
>         private void getFilesList(String path, boolean recursive, boolean
> includeFolders, boolean isFullPath, HashSet<String> collection,
> HashSet<String> unreadable) throws MalformedURLException, SmbException,
> ExsException {
>
>                 if(path == null) {
>                         path = "";
>                 }
>
>                 if(path.trim().length() > 0 && (!path.trim().endsWith("/")
> && !path.trim().endsWith("\\"))) {
>                         path = path + "/";
>                 }
>
>                 String url = "";
>
>                 if(isFullPath) {
>                         url = path;
>                 } else {
>                         url = generateURLString(path);
>                 }
>
>                 SmbFile smbFolder = new SmbFile(url);
>
>                 if(!smbFolder.exists() || !smbFolder.isDirectory()) {
>                         throw new ExsException("Error: Folder cannot be read
> from smb share: " + smbFolder.getPath());
>                 }
>
>                 if(!smbFolder.canRead()) {
>                         unreadable.add(smbFolder.getPath());
>                         return;
>                 }
>
>                 // this helps preventing heap overflows but it wont prevent
> it when to many files at once are opend
>                 HashSet<SmbFile> uniqueList = new HashSet<SmbFile>();
>                 for(SmbFile currentSmbFile : smbFolder.listFiles()) {
>                         uniqueList.add(currentSmbFile);
>                 }
>
>                 for(SmbFile currentSmbFile : uniqueList) {
>
>                         String currentPath = currentSmbFile.getPath();
>
>                         if(currentSmbFile.isFile()) {
>                                 currentSmbFile = null;
>                                 collection.add(currentPath);
>
>                         } else if(currentSmbFile.isDirectory()) {
>                                 int size = 0;
>                                 try {
>                                         String[] list =
> currentSmbFile.list();
>                                         size = list.length;
>                                         list = null;
>                                 } catch(Throwable t) {
>                                         continue;
>                                 }
>
>                                 currentSmbFile = null;
>                                 if(includeFolders) {
>                                         collection.add(currentPath);
>                                 }
>                                 if(recursive) {
>                                         if(size > 0) {
>                                                 getFilesList(currentPath,
> recursive, includeFolders, true, collection, unreadable);
>                                         }
>                                 }
>                         }
>                 }
>
>                 uniqueList.clear();
>                 uniqueList = null;
>         }
>
>
>



-- 
Michael B Allen
Java Active Directory Integration
http://www.ioplex.com/