crash on 2 gig file

Brian K. White brian at
Mon Jun 16 14:10:03 EST 2003

Tomasz Ciolek wrote:
>  I assume you mean you have a single 2gb file, not 2gb worth worth of
>  files .
>  The 2gb barrier is probably your problem. Rsync will try to create a file lists, and with a 2gb files in store you will make large RAM usage a norm.
>  Now when a file is being updated, you might find that it temporarily
>  swells beyond 2gb and then goes back to the right size. If yuor OS
>  barfs at that time ... then you get a problem.
>  Tomasz
> On Sun, Jun 15, 2003 at 09:51:13PM -0400, Brian K. White wrote:
>>>On Wed, Jun 11, 2003 at 04:16:38PM -0400, Brian K. White wrote:
>>>>2.5.6 (protocol 26) from cvs on SCO Open Server 5.0.6
>>>>both machines are the same version of OS and the same copy of rsync.
>>>>file on LIVE is 0 bytes
>>>>file on BACKUP is 2 gigs
>>>>on BACKUP you run:
>>>>  rsync -az --delete LIVE:/u/appl /u
>>>>causes all files in the list before the 2gig file to update, nothing
>>>Does SCO support files over 2GB in size?
>>>Tomasz M. Ciolek
>>Brian K. White  --  brian at  --
>>filePro BBx  Linux SCO  Prosper/FACTS AutoCAD  #callahans Satriani
>>To unsubscribe or change options:
>>Before posting, read:

I can imagine that dealing with a file that reaches the OS's maximum file 
size limit is problematic. No surprize there. The point of the report was 
that, don't you think rsync should deal with this situation in a better 
way than crashing and thus failing to do the rest of the job it was in the 
middle of?

Call me ignorant, but it seems to me rsync should be able to deal with any 
file that the OS can deal with, including one that is exactly as large as 
it is possible for a file to get. I would not put the mere existance of 2 
gig files in the same "don't do that" category as say, not having any free 
space for temp, not having any swap, trying to deal with large file lists 
and large files without lots of ram etc...

Brian K. White  --  brian at  --
filePro BBx  Linux SCO  Prosper/FACTS AutoCAD  #callahans Satriani

More information about the rsync mailing list