[jcifs] Large File upload performance

Naved Khan navedk at cybage.com
Tue Sep 16 22:42:55 EST 2003


Hi,

This may be an elementary question. Sorry if it is.
I tried copying large files (~ 75 MB) to remote machine using the JCIFS
package and the "Put" example given with it. The source host is a Win2k
machine and target remote machine is a Win2k server with a shared folder.

In order to compare the time taken with C++, I wrote a small test
application which uses CopyFileEx Win32 function to copy the file to the
remote machine.

I observed that the C++ app was 3 times faster. Maybe the Put.java in
examples is not the most efficient way to transfer large files. I tried
varies buffer sizes from 8KB to 10MB. But the best timing was about 3 times
slower. Maybe i'm missing something here.

I know that JCIFS provides much more features than I need. I wanted to know
that for my requirement, would JCIFS be an overkill.

Is there are efficient way of transfering large files to remote machine?
Should the time taken by JCIFS for transfering files be more than a C++
application ?

In short my requirement can be summarized as "able to selectively transfer
huge binary files from host machine to different target machines with
transfer resume functionality. The target machines can either be windows or
Linux".

Please reply.

Thanks in advance,
Naved

----- Original Message -----
From: "Michael B Allen" <mba2000 at ioplex.com>
To: "Naved Khan" <navedk at cybage.com>
Cc: <jcifs at lists.samba.org>
Sent: Saturday, September 06, 2003 12:45 AM
Subject: Re: [jcifs] Binary File Upload to Multiple Servers?


> > Hi,
> >
> > I want upload binary files to multiple Win32 or Linux machines
(servers).
> > The file transfer should support resume in case of breaks.
> > I want to use Java on Linux  (client). Would JCIFS be helpful in this
> > case?
>
> JCIFS probably does this sort of thing better than any other CIFS client.
>
> > I briefly skimmed through the docs and I think its possible. If any body
> > has
> > used jcifs in a similar scenario pls let me know.
> > A few features I want is:
> >   a.. File Transfer Resumption - To continue copying from a particular
> > point
>
> The off parameter SmbFileOutputStream.write() will skip to that offset.
> Same for SmbFileInputStream.read().
>
> > in the file, when failures occur.
> >   b.. Source File Completion Checking - To ensure that files do not get
> > copied (in the source location) that are still being written.
>
> Use SmbFile.FILE_NO_SHARE to lock the file. Also, jCIFS operates very well
> in a threaded environment. I recommend using 1 VM to do all of the work so
> that you can better coordinate each transfer (and possibly eliminate the
> need for FILE_NO_SHARE shareAccess altogether).
>
> >   c.. Source File Compression
>
> Wrap SmbFileOutputStream with java.util.zip.ZipOutputStream maybe?
>
> >   d.. Multiple Destination Directories - To transfer the same source
> > file(s)
>
> You'll have transfer each separately.
>
> > to multiple destinations.
> >   e.. Destination File Completion Renaming - To ensure files do not get
> > finalized until the copy has completed
>
> SmbFile.renameTo().
>
> >   f.. Destination File Existence Checking - To check whether the file
> > already exists (in the destination location), and accordingly either do
no
> > copy, and overwrite it (default)
>
> SmbFile.exists(). One thing to watch out for however is that file
> attributes (e.g. length()) are cached briefly. So if you create two
> SmbFiles that point to the same physical file and you change the
> attributes of one and then quickly view that attribute using the other,
> the second will not have updated yet. So take care to coordinate your
> threads properly so that you never have two SmbFile instances to the same
> file. Or if you are using separate VMs/machines you may want to turn off
> caching (see the API documentation) but you will take a significant
> perfomance hit for certain operations.
>
> > I strictly want to use Java on Linux (client requirement).
>
> No problem.




More information about the jcifs mailing list