rsync limit to file size/file count
echen at ateonix.com
Thu Jun 12 05:28:47 EST 2003
I left the rsync running since last night, and it is still running
receiving file list ...
expand file_list to 8000 bytes, did move
expand file_list to 16000 bytes, did move
expand file_list to 32000 bytes, did move
expand file_list to 64000 bytes, did move
expand file_list to 128000 bytes, did move
expand file_list to 256000 bytes, did move
expand file_list to 512000 bytes, did move
expand file_list to 1024000 bytes, did move
expand file_list to 2048000 bytes, did move
expand file_list to 4096000 bytes, did move
expand file_list to 8192000 bytes, did move
will this file-list ever be processed? what does the "expand file_list to X
bytes, did move" supposed to mean?
From: rsync-bounces+echen=ateonix.com at lists.samba.org
[mailto:rsync-bounces+echen=ateonix.com at lists.samba.org]On Behalf Of jw
Sent: Tuesday, June 10, 2003 5:30 PM
To: Rsync List
Subject: Re: rsync limit to file size/file count
On Tue, Jun 10, 2003 at 04:46:25PM -0700, Eric Chen wrote:
> What are the limits to file size and file count when doing a rsync
> using 2.5.6?
File sizes somewhere between 300MB and 1GB 2.5.6 begins
having noticeable issues. This is not a limit but merely
pushing the capacity of truncated digests. This problem
impacts performance only. It can be mitigated somewhat by
using a larger block size. These issues have been fixed in
CVS so 2.5.7 and later will be fine with huge files.
> I was trying to rsync about 500 GB of data with many files and many
> directories, but it has been stuck building the file list for several
> First of all, is it possible to transfer 500 GB of data? Secondly, what
> would the limit for file count be when doing a rsync transfer?
There is no hard limit on file count you could hit.
Each file requires approximately 70 + path-length bytes so it
is a question of how much process addressable memory you have.
J.W. Schultz Pegasystems Technologies
email address: jw at pegasys.ws
Remember Cernan and Schmitt
To unsubscribe or change options:
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
More information about the rsync