rsync limit to file size/file count
jw at pegasys.ws
Wed Jun 11 10:29:46 EST 2003
On Tue, Jun 10, 2003 at 04:46:25PM -0700, Eric Chen wrote:
> What are the limits to file size and file count when doing a rsync transfer
> using 2.5.6?
File sizes somewhere between 300MB and 1GB 2.5.6 begins
having noticeable issues. This is not a limit but merely
pushing the capacity of truncated digests. This problem
impacts performance only. It can be mitigated somewhat by
using a larger block size. These issues have been fixed in
CVS so 2.5.7 and later will be fine with huge files.
> I was trying to rsync about 500 GB of data with many files and many
> directories, but it has been stuck building the file list for several hours.
> First of all, is it possible to transfer 500 GB of data? Secondly, what
> would the limit for file count be when doing a rsync transfer?
There is no hard limit on file count you could hit.
Each file requires approximately 70 + path-length bytes so it
is a question of how much process addressable memory you have.
J.W. Schultz Pegasystems Technologies
email address: jw at pegasys.ws
Remember Cernan and Schmitt
More information about the rsync