[jcifs] Re: Large File upload performance

[Naved Khan]

Hi,

Mostly we will be needing to transfer files from local disk to remote
machine.
I tried increasing jcifs.netbios.client.writeSize to 65535 AND the
jcifs.smb.client.snd_buf_size
to ~60000. It did not help much.
Are there any planned improvements to JCIFS (wrt to speed) or is there any
other way by
which we can tweak it to increase the speed?

Thanks,
Naved

----- Original Message -----
From: "Michael B Allen" <mba2000 at ioplex.com>
To: "Naved Khan" <navedk at cybage.com>
Cc: <jcifs at lists.samba.org>
Sent: Wednesday, September 17, 2003 1:21 AM
Subject: Re: Large File upload performance


>
> > Hi,
> >
> > This may be an elementary question. Sorry if it is.
> > I tried copying large files (~ 75 MB) to remote machine using the JCIFS
> > package and the "Put" example given with it. The source host is a Win2k
> > machine and target remote machine is a Win2k server with a shared
folder.
> >
> > In order to compare the time taken with C++, I wrote a small test
> > application which uses CopyFileEx Win32 function to copy the file to the
> > remote machine.
> >
> > I observed that the C++ app was 3 times faster. Maybe the Put.java in
> > examples is not the most efficient way to transfer large files. I tried
> > varies buffer sizes from 8KB to 10MB. But the best timing was about 3
> > times
> > slower. Maybe i'm missing something here.
>
> JCIFS write performance is poor because of a buffer copy in NbtSocket. The
> NbtXxx stuff is scheduled for removal because of this. Also, jCIFS uses
> more CPU and memory than C clients. But speedwise all other operations are
> usually just as fast. And in a multithreaded environments where you have
> many IO operations happening at the same time it is my experience that
> jCIFS can be considerably faster.
>
> There are a few things to consider however; first, you are likely going to
> copy a file from one remote machine to another rather than from the local
> disk. For this you should use SmbFile.copyTo(). It uses an additional
> thread to read from the source file *while* it writes to destination file.
> I found this was 2x faster than NT (using Windows Explorer) when copying
> bushy directories of small files but it doesn't copy extended attributes
> and when the destination file is NT (and I suppose Win2K too) it doesn't
> set times.
>
> Also try increasing jcifs.netbios.client.writeSize to 65535 AND the
> undocumented jcifs.smb.client.snd_buf_size to ~60000. Ideally, when you
> look at the writes on the write (using Ethereal) you should see a slew of
> 5K messages and maybe one little one. If you see alternating 5K messages
> and one little one it will be VERY slow. This may have been why
> snd_buf_size is undocumented. I think setting it above the MTU had a BAD
> effect on high latency networks or it caused weird problems (i.e. what is
> the MTU negotiated between the src and dest?).
>
> Ultimately, if you're just reading large files from the local disk and
> writing them to a server jCIFS is going to be slower because of the buffer
> copy happening in NbtSocket.
>
> Mike
>
> --
> A program should be written to  model the concepts of the task it
> performs rather than the physical world or a process because this
> maximizes the  potential for it  to be applied  to tasks that are
> conceptually similar and, more  important, to tasks that have not
> yet been conceived.