only 1.3 of 12 GB will transfer from OSX to Linux

[Wallace Matthews]

I have done a lot of testing with single files as large as 85Gig and havent seen
any significant issues. My testbed(s) are Debian Linux and Redhat Linux. The tests
run as root. I have had performance issues with large files but nothing "broke" because
of the size of the files. That is assuming that the basic OS will handle huge files 
correctly. You do have to do some work with Debian and Redhat but our business is huge files
so all of our testbeds are correctly configured for them. I tested with rsync 2.6.2 and with 2.5.7.

I can't testify to any other os or operating environment. But rsync works correctly (if somewhat 
slower) with huge files. 2.5.7 is slower than 2.6.2 but I didnt see a huge difference in their performance. What I did see was that the rolling checksum isnt much use with huge files and you
spend a lot of time computing strong checksums. 

I am not being critical of rsync. It is designed for large directories of reasonable sized files and using it for a single huge file is at the extreme limits of what it is intended to be used for. 

We have a special case with large proprietary tarballs (ie. Veritas) that we can't trust to untar correctly so we occasionally have to handle them as a single large file. Unfortunately, we loose many of the advantages that rsync has in this environment and are now working on a proprietary solution to these
exceptions.

wally 

-----Original Message-----
From: Wayne Davison [mailto:wayned at samba.org]
Sent: Wednesday, July 28, 2004 6:21 PM
To: Anthony DiSante
Cc: rsync at lists.samba.org
Subject: Re: only 1.3 of 12 GB will transfer from OSX to Linux


On Wed, Jul 28, 2004 at 02:44:34PM -0400, Anthony DiSante wrote:
> I'm doing everything as root on both ends to make sure it's 
> not a permissions issue, but still can't get all the files to transfer.

First off, look for error messages in the log file that the daemon is
logging into (it will be one of your syslog files since you did not
override it).  Also, your config file didn't specify a uid, so it is
defaulting to running the daemon as user "nobody", so it may be a
permissions problem after all.  See the uid and gid settings in the
rsyncd.conf manpage.

..wayne..
-- 
To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html