Max filesize for rsync?

Tony at ServaCorp.com Tony at ServaCorp.com
Wed Apr 13 21:00:38 GMT 2005


I would be very happy to test any patches. (Assorted RedHat/Fedora i386)
(Assume I am a total newbie, much safer that way)

A few very large files regularly rsync'd in production.
Seems like it sometimes gets somewhat stuck in the middle of something
large.
(The rsync is mostly staging area to staging area.
Plenty of redundancy, so I'm unlikely to get hurt if I'm aware of problems.
The way the targets are used, I will know about problems before any real
damage is done.)
The more important of the transfers are over occasionally very bad internet
connections, so I'm pretty much in the situation of something to gain,
nothing to lose.
-rw-rw----    1 27       27       1187270120 Apr 13 03:24
/home/rsync-sjs/mysql/sjs/dwf.MYD
-rw-rw----    1 27       27       1098515060 Apr  8 07:34
/home/rsync-sjs/mysql/srvs/dwf.MYD
-rw-rw----    1 mysql    mysql    840374964 Apr 12 12:59
/home/rsync-2bb/mysql/srv/dwf.MYD
-rw-rw----    1 mysql    mysql    520216980 Apr 12 20:54
/home/rsync-2bb/mysql/map/dwf.MYD
-rw-rw----    1 mysql    mysql    221876208 Apr 11 14:21
/home/rsync-2bb/mysql/ecp/dwf.MYD

Scripts are along the lines of:
#!/bin/sh
# staged backup
time rsync -a /home/rsync-sjs/* /home/rsync-sjs-1bk/
time
sync -avPz --timeout=7777  --password-file=/etc/rsync.secrets/rsync-sjs \
        rsync-sjs at sjs::rsync-sjs/*      /home/rsync-sjs/
date; echo rsync-sjs

-----Original Message-----
From: rsync-bounces+tony=servacorp.com at lists.samba.org
[mailto:rsync-bounces+tony=servacorp.com at lists.samba.org]On Behalf Of
Shachar Shemesh
Sent: Wednesday, April 13, 2005 2:53 PM
To: Jeff Schoby
Cc: rsync at lists.samba.org
Subject: Re: Max filesize for rsync?


Jeff Schoby wrote:

>What the maximum filesize rsync can transfer?
>
>I'm trying to rsync one of my servers to another but the rsync is
>croaking on a file that's barely 1GB.
>
>Tips, hints, suggestions?
>
>rsync server is AIX 4.3.3 ML11 - rsync 2.6.3
>
>rsync client is AIX 5.3 ML1 - rsync 2.6.4
>
>Thanks
>-Jeff
>
>
Please note that, all file size OS limitations aside, rsync has
suboptimal performance for too big files. When I get around to it I'll
try to create a patch, but in the mean while too big files will have too
many non-real hash table collisions, and may become extremely slow.

If you run across this problem, please post on list, as we need someone
to experience this problem in order to try and fix it.

          Shachar

--
Shachar Shemesh
Lingnu Open Source Consulting ltd.
Have you backed up today's work? http://www.lingnu.com/backup.html

--
To unsubscribe or change options:
https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html



More information about the rsync mailing list