[jcifs] Re: Large File upload performance

[Christopher R. Hertel]

It would be worth-while seeing partial captures of both methods, just to
determine whether it is a difference in the wire transfer.  I would be
surprised if the additional processing overhead was really enough to cause 
a three-fold increase in transfer time.

I don't have a configuration that can test this properly.  Is there any 
chance you can grab a couple of NetMon traces?

Chris -)-----

On Fri, Sep 19, 2003 at 06:06:32PM +0530, Naved Khan wrote:
> Hi,
> 
> Mostly we will be needing to transfer files from local disk to remote
> machine.
> I tried increasing jcifs.netbios.client.writeSize to 65535 AND the
> jcifs.smb.client.snd_buf_size
> to ~60000. It did not help much.
> Are there any planned improvements to JCIFS (wrt to speed) or is there any
> other way by
> which we can tweak it to increase the speed?
> 
> Thanks,
> Naved
> 
> ----- Original Message -----
> From: "Michael B Allen" <mba2000 at ioplex.com>
> To: "Naved Khan" <navedk at cybage.com>
> Cc: <jcifs at lists.samba.org>
> Sent: Wednesday, September 17, 2003 1:21 AM
> Subject: Re: Large File upload performance
> 
> 
> >
> > > Hi,
> > >
> > > This may be an elementary question. Sorry if it is.
> > > I tried copying large files (~ 75 MB) to remote machine using the JCIFS
> > > package and the "Put" example given with it. The source host is a Win2k
> > > machine and target remote machine is a Win2k server with a shared
> folder.
> > >
> > > In order to compare the time taken with C++, I wrote a small test
> > > application which uses CopyFileEx Win32 function to copy the file to the
> > > remote machine.
> > >
> > > I observed that the C++ app was 3 times faster. Maybe the Put.java in
> > > examples is not the most efficient way to transfer large files. I tried
> > > varies buffer sizes from 8KB to 10MB. But the best timing was about 3
> > > times
> > > slower. Maybe i'm missing something here.
> >
> > JCIFS write performance is poor because of a buffer copy in NbtSocket. The
> > NbtXxx stuff is scheduled for removal because of this. Also, jCIFS uses
> > more CPU and memory than C clients. But speedwise all other operations are
> > usually just as fast. And in a multithreaded environments where you have
> > many IO operations happening at the same time it is my experience that
> > jCIFS can be considerably faster.
> >
> > There are a few things to consider however; first, you are likely going to
> > copy a file from one remote machine to another rather than from the local
> > disk. For this you should use SmbFile.copyTo(). It uses an additional
> > thread to read from the source file *while* it writes to destination file.
> > I found this was 2x faster than NT (using Windows Explorer) when copying
> > bushy directories of small files but it doesn't copy extended attributes
> > and when the destination file is NT (and I suppose Win2K too) it doesn't
> > set times.
> >
> > Also try increasing jcifs.netbios.client.writeSize to 65535 AND the
> > undocumented jcifs.smb.client.snd_buf_size to ~60000. Ideally, when you
> > look at the writes on the write (using Ethereal) you should see a slew of
> > 5K messages and maybe one little one. If you see alternating 5K messages
> > and one little one it will be VERY slow. This may have been why
> > snd_buf_size is undocumented. I think setting it above the MTU had a BAD
> > effect on high latency networks or it caused weird problems (i.e. what is
> > the MTU negotiated between the src and dest?).
> >
> > Ultimately, if you're just reading large files from the local disk and
> > writing them to a server jCIFS is going to be slower because of the buffer
> > copy happening in NbtSocket.
> >
> > Mike
> >
> > --
> > A program should be written to  model the concepts of the task it
> > performs rather than the physical world or a process because this
> > maximizes the  potential for it  to be applied  to tasks that are
> > conceptually similar and, more  important, to tasks that have not
> > yet been conceived.
> 

-- 
"Implementing CIFS - the Common Internet FileSystem" ISBN: 013047116X
Samba Team -- http://www.samba.org/     -)-----   Christopher R. Hertel
jCIFS Team -- http://jcifs.samba.org/   -)-----   ubiqx development, uninq.
ubiqx Team -- http://www.ubiqx.org/     -)-----   crh at ubiqx.mn.org
OnLineBook -- http://ubiqx.org/cifs/    -)-----   crh at ubiqx.org