[jcifs] Re: Large File upload performance
navedk at cybage.com
Fri Sep 19 22:36:32 EST 2003
Mostly we will be needing to transfer files from local disk to remote
I tried increasing jcifs.netbios.client.writeSize to 65535 AND the
to ~60000. It did not help much.
Are there any planned improvements to JCIFS (wrt to speed) or is there any
other way by
which we can tweak it to increase the speed?
----- Original Message -----
From: "Michael B Allen" <mba2000 at ioplex.com>
To: "Naved Khan" <navedk at cybage.com>
Cc: <jcifs at lists.samba.org>
Sent: Wednesday, September 17, 2003 1:21 AM
Subject: Re: Large File upload performance
> > Hi,
> > This may be an elementary question. Sorry if it is.
> > I tried copying large files (~ 75 MB) to remote machine using the JCIFS
> > package and the "Put" example given with it. The source host is a Win2k
> > machine and target remote machine is a Win2k server with a shared
> > In order to compare the time taken with C++, I wrote a small test
> > application which uses CopyFileEx Win32 function to copy the file to the
> > remote machine.
> > I observed that the C++ app was 3 times faster. Maybe the Put.java in
> > examples is not the most efficient way to transfer large files. I tried
> > varies buffer sizes from 8KB to 10MB. But the best timing was about 3
> > times
> > slower. Maybe i'm missing something here.
> JCIFS write performance is poor because of a buffer copy in NbtSocket. The
> NbtXxx stuff is scheduled for removal because of this. Also, jCIFS uses
> more CPU and memory than C clients. But speedwise all other operations are
> usually just as fast. And in a multithreaded environments where you have
> many IO operations happening at the same time it is my experience that
> jCIFS can be considerably faster.
> There are a few things to consider however; first, you are likely going to
> copy a file from one remote machine to another rather than from the local
> disk. For this you should use SmbFile.copyTo(). It uses an additional
> thread to read from the source file *while* it writes to destination file.
> I found this was 2x faster than NT (using Windows Explorer) when copying
> bushy directories of small files but it doesn't copy extended attributes
> and when the destination file is NT (and I suppose Win2K too) it doesn't
> set times.
> Also try increasing jcifs.netbios.client.writeSize to 65535 AND the
> undocumented jcifs.smb.client.snd_buf_size to ~60000. Ideally, when you
> look at the writes on the write (using Ethereal) you should see a slew of
> 5K messages and maybe one little one. If you see alternating 5K messages
> and one little one it will be VERY slow. This may have been why
> snd_buf_size is undocumented. I think setting it above the MTU had a BAD
> effect on high latency networks or it caused weird problems (i.e. what is
> the MTU negotiated between the src and dest?).
> Ultimately, if you're just reading large files from the local disk and
> writing them to a server jCIFS is going to be slower because of the buffer
> copy happening in NbtSocket.
> A program should be written to model the concepts of the task it
> performs rather than the physical world or a process because this
> maximizes the potential for it to be applied to tasks that are
> conceptually similar and, more important, to tasks that have not
> yet been conceived.
More information about the jcifs