[jcifs] SmbFile transfer issues.
Michael B Allen
ioplex at gmail.com
Wed Dec 1 08:54:02 MST 2010
Hi Paul,
The problem is that you are setting responseTimeout to 5 seconds. That
means that if a server does not respond to a command within 5 seconds,
it will give up. That value is far far too low.
The property that will free resources quickly is soTimeout. But you
set that to 30 seconds even though the default is 35 which doesn't
make much sense.
So you might as well just remove those two properties and use the
default values for responseTimeout and soTimeout which is 30 and 35
seconds respectively.
If you do play with those values, responseTimeout should be slightly
less than soTimeout. But I would not go below 20 seconds for
responseTimeout.
Actually for maximum reliability, these values should probably be
higher than the defaults. I bet Windows waits longer than 30 seconds
before giving up on a command. For this type of application I would
probably use responseTimeout and soTimeout of more like 100 and 120
seconds respectively. Optimizing the algorithm used to process files
would yield much better performance improvements than messing around
potentially dangerous low-level timing properties.
Mike
--
Michael B Allen
Java Active Directory Integration
http://www.ioplex.com/
On Tue, Nov 30, 2010 at 5:52 AM, Paul Caswell
<Paul.Caswell at rbsretail.com> wrote:
> Hi All,
>
>
>
> I have come across an issue with jCIFS.
>
>
>
> I use it to copy files from many remote PCs to a central server. Some of the
> PCs may be off so I drop the timeouts down a bit and run the process on a
> number of threads to maintain throughput. Unfortunately, on a regular basis,
> the whole tool falls apart leaving half-copied files all over the place.
>
>
>
> jcifs.smb.SmbException: Transport1 timedout waiting for response to
> SmbComNTCreateAndX[command=SMB_COM_NT_CREATE_ANDX,received=false,errorCode=0,flags=0x0018,flags2=0xC803,signSeq=0,tid=26631,pid=37999,uid=16385,mid=26766,wordCount=24,byteCount=75,andxCommand=0xFF,andxOffset=0,flags=0x00,rootDirectoryFid=0,desiredAccess=0x0089,allocationSize=0,extFileAttributes=0x0080,shareAccess=0x0007,createDisposition=0x0001,createOptions=0x00000040,impersonationLevel=0x0002,securityFlags=0x03,name=...]
>
>
>
> I have narrowed the issue down to a combination of two factors, the number
> of threads I use and the time between transfer requests.
>
>
>
> The following test class demonstrates the issue. Test1 succeeds while test2
> collapses in a heap on about the 20th file. Can you help? Is this a bug or
> am I abusing the system? If this is abuse, what “reasonable” limits should I
> stick to?
>
>
>
> Testing against jCIFS 1.3.15. Server=W7 Client=XP.
>
>
>
> Paul
>
>
>
> [code]
>
> package jcifs.smb;
>
>
>
> import java.io.File;
>
> import java.io.FileOutputStream;
>
> import java.io.IOException;
>
> import java.io.InputStream;
>
> import java.io.OutputStream;
>
> import java.net.MalformedURLException;
>
> import java.util.concurrent.ArrayBlockingQueue;
>
> import java.util.concurrent.BlockingQueue;
>
>
>
> /**
>
> *
>
> * @author
>
> */
>
> public class TestSMBPull {
>
>
>
> // Tests SMB pulling.
>
>
>
> // Unit to pull from. Can be an i/p or resolvable name.
>
> static final String PULL_FROM = "PaulsPC";
>
> // Name of file to pull.
>
> static final String FILE_NAME_TO_PULL = "TestFile.xml";
>
> // Where it is.
>
> static final String FOLDER_TO_PULL_FROM = "C$/Temp";
>
> // The full thing.
>
> static final String URL = "smb://" + PULL_FROM + "/" +
> FOLDER_TO_PULL_FROM + "/" + FILE_NAME_TO_PULL;
>
>
>
> private static class Puller implements Runnable {
>
>
>
> // My number.
>
> private final int n;
>
> // My queue.
>
> private final BlockingQueue<Puller> queue;
>
>
>
> public Puller(int n, BlockingQueue<Puller> queue) {
>
> this.n = n;
>
> this.queue = queue;
>
> }
>
>
>
> public void go() {
>
> // Use the queue to limit the number of pullers running at any one
> time.
>
> while (!queue.offer(this)) {
>
> try {
>
> Thread.sleep(10);
>
> } catch (InterruptedException ex) {
>
> }
>
> }
>
> // Start my thread running.
>
> Thread t = new Thread(this, "Puller " + n);
>
> // Run me in a thread.
>
> t.start();
>
> }
>
>
>
> public void run() {
>
> String result = "Failed";
>
> try {
>
> SmbFile f = new SmbFile(URL);
>
> if (f.exists()) {
>
> // Build the name of the target file.
>
> String pullName = FILE_NAME_TO_PULL + "_" + n;
>
> File pullTo = new File(pullName);
>
> // Pull it.
>
> transfer(f.getInputStream(), new FileOutputStream(pullTo));
>
> result = "Success!";
>
> } else {
>
> result = "Doesn't exist.";
>
> }
>
> } catch (MalformedURLException ex) {
>
> result = "Failed (MalformedURLException): " + ex.toString() + "
> URL: " + URL;
>
> } catch (SmbException ex) {
>
> result = "Failed (SmbException): " + ex.toString() + " URL: " +
> URL;
>
> } catch (IOException ex) {
>
> result = "Failed (IOException): " + ex.toString() + " URL: " +
> URL;
>
> } catch (Exception ex) {
>
> result = "Failed (Exception): " + ex.toString() + " URL: " +
> URL;
>
> } finally {
>
> // Remove me from the queue.
>
> queue.remove(this);
>
> }
>
> System.out.println("Result: " + n + " was \"" + result +"\"");
>
> }
>
>
>
> public static void transfer(final InputStream from, final OutputStream
> to) throws IOException {
>
> final byte[] buffer = new byte[1024];
>
> int len;
>
>
>
> while ((len = from.read(buffer)) >= 0) {
>
> to.write(buffer, 0, len);
>
> }
>
> }
>
> }
>
>
>
> private static void drainQueue(BlockingQueue<Puller> queue) {
>
> // Wait for the queue to empty.
>
> while ( !queue.isEmpty() ) {
>
> try {
>
> Thread.sleep(100);
>
> } catch (InterruptedException ex) {
>
> }
>
> }
>
> }
>
>
>
> private static void test(BlockingQueue<Puller> queue, int howMany) {
>
> // Delay between each creatrion of a puller. Starts slowly but
> accelerates.
>
> // ## Things go bang when delay hits zero and there are lots of
> threads.
>
> int delay = 100;
>
> // Test by pulling 1,000 files using a fixed maximum number of
> threads.
>
> for (int i = 0; i < howMany; i++) {
>
> Puller p = new Puller(i, queue);
>
> p.go();
>
> // Start a bit slowly.
>
> if (delay > 0) {
>
> try {
>
> Thread.sleep(delay);
>
> } catch (InterruptedException ex) {
>
> }
>
> delay -= 5;
>
> }
>
> }
>
> drainQueue(queue);
>
> }
>
>
>
> private static void test1() {
>
> System.out.println("Test1:\n------");
>
> // Allow for not too many of threads. Works fine so not many files.
>
> test (new ArrayBlockingQueue<Puller>(20), 1000);
>
> }
>
>
>
> private static void test2() {
>
> System.out.println("Test2:\n------");
>
> // Allow for lots of threads. Fails so lots of files for max chance.
>
> test (new ArrayBlockingQueue<Puller>(100), 10000);
>
> }
>
>
>
> public static void main(String[] args) {
>
> jcifs.Config.setProperty("jcifs.smb.client.responseTimeout", "5000");
>
> jcifs.Config.setProperty("jcifs.smb.client.soTimeout", "30000");
>
> jcifs.Config.setProperty("jcifs.smb.client.username",
> "Administrator");
>
> jcifs.Config.setProperty("jcifs.smb.client.password", "***");
>
>
>
> test1 ();
>
> test2 ();
>
> }
>
> }
>
> [/code]
>
> This email is from Sanderson Group plc or a subsidiary of Sanderson Group
> plc ('the company') and is for the personal use of the intended
> recipient(s). The email and any files transmitted with it are confidential
> to the intended recipient(s) and may be legally privileged or contain
> proprietary and private information. If you are not an intended recipient,
> you may not review, copy or distribute this message. If received in error,
> please notify the sender and delete the message from your system Any views
> or opinions expressed in this email and any files transmitted with it are
> those of the author only and may not necessarily reflect the views of the
> company. Unless otherwise pre-agreed by exchange of hard copy documents
> signed by duly authorised representatives, contracts may not be concluded on
> behalf of the company by email. Please note that neither the company nor the
> sender accepts any responsibility for any viruses and it is your
> responsibility to scan the email and any attachments. All email received and
> sent by the company may be monitored to protect its business interests.
> Members of Sanderson Group plc: Sanderson Group plc: Registered in England,
> Registered No. 04968444 Sanderson Limited: Registered in England, Registered
> No. 03743507 Sanderson Multi-Channel Solutions Limited: Registered in
> England, Registered No. 05684599 Sanderson Australia Pty Ltd ACN: 001 723
> 039 Registered Office for all aforementioned companies: Sanderson House,
> Manor Road, Coventry, CV1 2GF Sanderson RBS Limited: Registered in England,
> Registered No. 02306403 Registered Office: 24-26 Vincent Avenue, Crownhill,
> Milton Keynes, MK8 0AB
More information about the jCIFS
mailing list