Is it possible to transfer a large, dynamic file in a piecemeal fashion? Yes it is.

Kevin Korb kmk at
Tue Sep 5 00:43:37 UTC 2017

You probably want --inplace for this rather than partial.  The upside of
--partial is that the existing file isn't replaced until there is a
complete file to replace it with.  The downside is building up that
entire new file.

On 09/05/2017 12:01 AM, Don Kuenz via rsync wrote:
> Subject: Re: Is it possible to transfer a large, dynamic file in a piecemeal fashion? Yes it is.
> Don Kuenz via rsync <rsync at> wrote:
>> Greetings,
>> Is it possible to use rsync to transmit a large, dynamic 2TB file in a
>> piecemeal fashion during daylight hours over the course of a dozen days?
>> On a good day, about 200GB of data can be transferred before rsync times
>> out to enable a nightly local backup to complete. The local backup
>> changes a few 512 byte blocks in the 2TB file. That's why the 2TB file
>> is dynamic.
>> As an experiment, rsync was started on the client to sync the 2TB file.
>> At that point, rsyncd on the backup server created a temporarily file
>> with the 2TB file name appended with a dot followed by six characters.
>> After about 100MB was transferred the rsync client was interrupted and
>> stopped in order to simulate the nightly shutdown. When rsync was
>> restarted it erased the original 100MB temporary file and started over
>> with a new, empty temporarily file.
>> In summary:
>> * rsync apparently erases temporarily files upon restart
>> * it takes about a dozen days to transfer the 2TB file
>> Unless there's an option to have rsync pick up where it left off it
>> seems impossible to transmit a large file in a piecemeal fashion.
>> Because the (nearly) same 200GB at the beginning of the 2TB file gets
>> retransmitted over and over with each restart.
> The rsync --partial argument retains the partial file between an 
> interruption and a restart. It works for --timelimit interruptions as
> well as ctrl-c interruptions on the client side.    
> Thank you,

	Kevin Korb			Phone:    (407) 252-6853
	Systems Administrator		Internet:
	FutureQuest, Inc.		Kevin at  (work)
	Orlando, Florida		kmk at (personal)
	Web page:
	PGP public key available on web site.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 224 bytes
Desc: OpenPGP digital signature
URL: <>

More information about the rsync mailing list