[jcifs] SmbFile transfer issues.

[Paul Caswell]

Hi All,

 

I have come across an issue with jCIFS.

 

I use it to copy files from many remote PCs to a central server. Some of
the PCs may be off so I drop the timeouts down a bit and run the process
on a number of threads to maintain throughput. Unfortunately, on a
regular basis, the whole tool falls apart leaving half-copied files all
over the place.

 

jcifs.smb.SmbException: Transport1 timedout waiting for response to
SmbComNTCreateAndX[command=SMB_COM_NT_CREATE_ANDX,received=false,errorCo
de=0,flags=0x0018,flags2=0xC803,signSeq=0,tid=26631,pid=37999,uid=16385,
mid=26766,wordCount=24,byteCount=75,andxCommand=0xFF,andxOffset=0,flags=
0x00,rootDirectoryFid=0,desiredAccess=0x0089,allocationSize=0,extFileAtt
ributes=0x0080,shareAccess=0x0007,createDisposition=0x0001,createOptions
=0x00000040,impersonationLevel=0x0002,securityFlags=0x03,name=...]

 

I have narrowed the issue down to a combination of two factors, the
number of threads I use and the time between transfer requests.

 

The following test class demonstrates the issue. Test1 succeeds while
test2 collapses in a heap on about the 20th file. Can you help? Is this
a bug or am I abusing the system? If this is abuse, what "reasonable"
limits should I stick to?

 

Testing against jCIFS 1.3.15. Server=W7 Client=XP.

 

Paul

 

[code]

package jcifs.smb;

 

import java.io.File;

import java.io.FileOutputStream;

import java.io.IOException;

import java.io.InputStream;

import java.io.OutputStream;

import java.net.MalformedURLException;

import java.util.concurrent.ArrayBlockingQueue;

import java.util.concurrent.BlockingQueue;

 

/**

*

* @author 

 */

public class TestSMBPull {

 

   // Tests SMB pulling.

 

   // Unit to pull from. Can be an i/p or resolvable name.

   static final String PULL_FROM = "PaulsPC";

   // Name of file to pull.

   static final String FILE_NAME_TO_PULL = "TestFile.xml";

   // Where it is.

   static final String FOLDER_TO_PULL_FROM = "C$/Temp";

   // The full thing.

   static final String URL = "smb://" + PULL_FROM + "/" +
FOLDER_TO_PULL_FROM + "/" + FILE_NAME_TO_PULL;

 

   private static class Puller implements Runnable {

 

      // My number.

      private final int n;

      // My queue.

      private final BlockingQueue<Puller> queue;

 

      public Puller(int n, BlockingQueue<Puller> queue) {

         this.n = n;

         this.queue = queue;

      }

 

      public void go() {

         // Use the queue to limit the number of pullers running at any
one time.

         while (!queue.offer(this)) {

            try {

               Thread.sleep(10);

            } catch (InterruptedException ex) {

            }

         }

         // Start my thread running.

         Thread t = new Thread(this, "Puller " + n);

         // Run me in a thread.

         t.start();

      }

 

      public void run() {

         String result = "Failed";

         try {

            SmbFile f = new SmbFile(URL);

            if (f.exists()) {

               // Build the name of the target file.

               String pullName = FILE_NAME_TO_PULL + "_" + n;

               File pullTo = new File(pullName);

               // Pull it.

               transfer(f.getInputStream(), new
FileOutputStream(pullTo));

               result = "Success!";

            } else {

               result = "Doesn't exist.";

            }

         } catch (MalformedURLException ex) {

            result = "Failed (MalformedURLException): " + ex.toString()
+ " URL: " + URL;

         } catch (SmbException ex) {

            result = "Failed (SmbException): " + ex.toString() + " URL:
" + URL;

         } catch (IOException ex) {

            result = "Failed (IOException): " + ex.toString() + " URL: "
+ URL;

         } catch (Exception ex) {

            result = "Failed (Exception): " + ex.toString() + " URL: " +
URL;

         } finally {

            // Remove me from the queue.

            queue.remove(this);

         }

         System.out.println("Result: " + n + " was \"" + result +"\"");

      }

 

      public static void transfer(final InputStream from, final
OutputStream to) throws IOException {

         final byte[] buffer = new byte[1024];

         int len;

 

         while ((len = from.read(buffer)) >= 0) {

            to.write(buffer, 0, len);

         }

      }

   }

 

   private static void drainQueue(BlockingQueue<Puller> queue) {

      // Wait for the queue to empty.

      while ( !queue.isEmpty() ) {

         try {

            Thread.sleep(100);

         } catch (InterruptedException ex) {

         }

      }

   }

 

   private static void test(BlockingQueue<Puller> queue, int howMany) {

      // Delay between each creatrion of a puller. Starts slowly but
accelerates.

      // ## Things go bang when delay hits zero and there are lots of
threads.

      int delay = 100;

      // Test by pulling 1,000 files using a fixed maximum number of
threads.

      for (int i = 0; i < howMany; i++) {

         Puller p = new Puller(i, queue);

         p.go();

         // Start a bit slowly.

         if (delay > 0) {

            try {

               Thread.sleep(delay);

            } catch (InterruptedException ex) {

            }

            delay -= 5;

         }

      }

      drainQueue(queue);

   }

 

   private static void test1() {

      System.out.println("Test1:\n------");

      // Allow for not too many of threads. Works fine so not many
files.

      test (new ArrayBlockingQueue<Puller>(20), 1000);

   }

 

   private static void test2() {

      System.out.println("Test2:\n------");

      // Allow for lots of threads. Fails so lots of files for max
chance.

      test (new ArrayBlockingQueue<Puller>(100), 10000);

   }

 

   public static void main(String[] args) {

      jcifs.Config.setProperty("jcifs.smb.client.responseTimeout",
"5000");

      jcifs.Config.setProperty("jcifs.smb.client.soTimeout", "30000");

      jcifs.Config.setProperty("jcifs.smb.client.username",
"Administrator");

      jcifs.Config.setProperty("jcifs.smb.client.password", "***");

 

      test1 ();

      test2 ();

   }

}

[/code]

This email is from Sanderson Group plc or a subsidiary of Sanderson Group plc ('the company') and is for the personal use of the intended recipient(s). The email and any files transmitted with it are confidential to the intended recipient(s) and may be legally privileged or contain proprietary and private information. If you are not an intended recipient, you may not review, copy or distribute this message. If received in error, please notify the sender and delete the message from your system
Any views or opinions expressed in this email and any files transmitted with it are those of the author only and may not necessarily reflect the views of the company. Unless otherwise pre-agreed by exchange of hard copy documents signed by duly authorised representatives, contracts may not be concluded on behalf of the company by email.

Please note that neither the company nor the sender accepts any responsibility for any viruses and it is your responsibility to scan the email and any attachments. All email received and sent by the company may be monitored to protect its business interests. 

Members of Sanderson Group plc:
Sanderson Group plc: Registered in England, Registered No. 04968444
Sanderson Limited: Registered in England, Registered No. 03743507
Sanderson Multi-Channel Solutions Limited: Registered in England, Registered No. 05684599
Sanderson Australia Pty Ltd  ACN: 001 723 039 
Registered Office for all aforementioned companies: Sanderson House, Manor Road, Coventry, CV1 2GF
Sanderson RBS Limited: Registered in England, Registered No. 02306403
Registered Office: 24-26 Vincent Avenue, Crownhill, Milton Keynes, MK8 0AB
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.samba.org/pipermail/jcifs/attachments/20101130/b2525cd4/attachment-0001.html>