[jcifs] Re: Large File upload performance

[Michael B Allen]

> Hi,
>
> This may be an elementary question. Sorry if it is.
> I tried copying large files (~ 75 MB) to remote machine using the JCIFS
> package and the "Put" example given with it. The source host is a Win2k
> machine and target remote machine is a Win2k server with a shared folder.
>
> In order to compare the time taken with C++, I wrote a small test
> application which uses CopyFileEx Win32 function to copy the file to the
> remote machine.
>
> I observed that the C++ app was 3 times faster. Maybe the Put.java in
> examples is not the most efficient way to transfer large files. I tried
> varies buffer sizes from 8KB to 10MB. But the best timing was about 3
> times
> slower. Maybe i'm missing something here.

JCIFS write performance is poor because of a buffer copy in NbtSocket. The
NbtXxx stuff is scheduled for removal because of this. Also, jCIFS uses
more CPU and memory than C clients. But speedwise all other operations are
usually just as fast. And in a multithreaded environments where you have
many IO operations happening at the same time it is my experience that
jCIFS can be considerably faster.

There are a few things to consider however; first, you are likely going to
copy a file from one remote machine to another rather than from the local
disk. For this you should use SmbFile.copyTo(). It uses an additional
thread to read from the source file *while* it writes to destination file.
I found this was 2x faster than NT (using Windows Explorer) when copying
bushy directories of small files but it doesn't copy extended attributes
and when the destination file is NT (and I suppose Win2K too) it doesn't
set times.

Also try increasing jcifs.netbios.client.writeSize to 65535 AND the
undocumented jcifs.smb.client.snd_buf_size to ~60000. Ideally, when you
look at the writes on the write (using Ethereal) you should see a slew of
5K messages and maybe one little one. If you see alternating 5K messages
and one little one it will be VERY slow. This may have been why
snd_buf_size is undocumented. I think setting it above the MTU had a BAD
effect on high latency networks or it caused weird problems (i.e. what is
the MTU negotiated between the src and dest?).

Ultimately, if you're just reading large files from the local disk and
writing them to a server jCIFS is going to be slower because of the buffer
copy happening in NbtSocket.

Mike

--
A program should be written to  model the concepts of the task it
performs rather than the physical world or a process because this
maximizes the  potential for it  to be applied  to tasks that are
conceptually similar and, more  important, to tasks that have not
yet been conceived.