Rsync Problems, Possible Addressed Bug?
Chris Shoemaker
c.shoemaker at cox.net
Wed Jul 14 18:49:50 GMT 2004
On Wed, Jul 14, 2004 at 04:55:31PM -0400, Robert Caskey wrote:
> I got this mail from a cronjob and can't figure out what is causing
> rsync to crap out on me. I received the message at 5:03, when the cron
> job is scheduled to run at 4:00, so total runtime is approximately an
> hour.
>
> Machine that is fetching the files is a low-end G3 running Yellowdog
> 3.01, rsync --version 2.5.5. It has only 128 megs of ram, but isn't
> doing anything else. Transfers are CPU bound.
>
> It syncs approximately 120 gigs of files over ssh from various
> machines, but has difficulty with a Mac OS X.3 server machine running
> rsync 2.5.7, which is also the largest host with over 100 gigs of files
> to be backed up. Is there a known issue with these versions of r-sync?
>
> Anyone here have any ideas? I really don't sleep well at night when my
> nightlys don't go well.
>
> Thanks,
> --Rob
>
>
> Subject: Cron <root at cornerstone> run-parts /etc/cron.daily
>
>
> /etc/cron.daily/backup_ribs:
>
> -= Beginning backups for calendar
>
> receiving file list ... done
> Public Event Server/
> Public Event Server/Public Event Server.db
> rsync[6790] (receiver) heap statistics:
> arena: 118120 (bytes from sbrk)
> ordblks: 5 (chunks not in use)
> smblks: 2
> hblks: 0 (chunks from mmap)
> hblkhd: 0 (bytes from mmap)
> usmblks: 0
> fsmblks: 80
> uordblks: 101704 (bytes used)
> fordblks: 16416 (bytes free)
> keepcost: 3352 (bytes in releasable chunk)
>
> Number of files: 2
> Number of files transferred: 1
> Total file size: 6883446 bytes
> Total transferred file size: 6883446 bytes
> Literal data: 18546 bytes
> Matched data: 6864900 bytes
> File list size: 120
> Total bytes written: 59158
> Total bytes read: 8517
>
> wrote 59158 bytes read 8517 bytes 3301.22 bytes/sec
> total size is 6883446 speedup is 101.71
>
> Additional information:
> total 48
> lrwxrwxrwx 1 root root 37 Jun 2 04:02 current ->
> /mnt/storage/backups/calendar/daily.0
> drwxr-xr-x 3 root root 4096 Jul 13 04:02 daily.0
> drwxr-xr-x 3 root root 4096 Jul 12 17:04 daily.1
> drwxr-xr-x 3 root root 4096 Jul 12 07:55 daily.2
> drwxr-xr-x 3 root root 4096 Jul 12 05:13 daily.3
> drwxr-xr-x 3 root root 4096 Jul 12 04:02 daily.4
> drwxr-xr-x 3 root root 4096 Jul 11 04:02 daily.5
> drwxr-xr-x 3 root root 4096 Jul 10 04:02 daily.6
> drwxr-xr-x 3 root root 4096 Jul 1 04:42 monthly.0
> drwxr-xr-x 3 root root 4096 Jul 11 04:22 weekly.0
> drwxr-xr-x 3 root root 4096 Jul 4 04:22 weekly.1
> drwxr-xr-x 3 root root 4096 Jun 27 04:22 weekly.2
> drwxr-xr-x 3 root root 4096 Jun 20 04:22 weekly.3
> Filesystem Size Used Avail Use% Mounted on
> /dev/hda3 5.7G 547M 4.8G 10% /
> none 30M 0 30M 0% /dev/shm
> /dev/hdb2 126G 40G 79G 34% /mnt/storage
>
> -= Beginning backups for db
>
> ssh: connect to host db.music.uga.edu port 22: No route to host
> rsync: connection unexpectedly closed (0 bytes read so far)
> rsync error: error in rsync protocol data stream (code 12) at io.c(150)
This error is pretty clear. I don't think you're asking about this one,
but I notice that your script is showing its own error report, probably
by checking the return status of rsync, but...
>
> WARNING: there seems to have been an rsync error.
> Check the logs for more information.
>
> -= Beginning backups for figaro
>
> receiving file list ... done
> mailman/archives/private/faculty-announce/
> mailman/locks/
> www/html/tools/labs/
> etc/mail/statistics
> mailman/archives/private/faculty-announce/2004-July.txt.gz
> www/html/syllabus/temp/MUSI_1810_zerkeld_200108.pdf
> www/html/tools/labs/current_index.php
> rsync[6805] (receiver) heap statistics:
> arena: 6544744 (bytes from sbrk)
> ordblks: 2673 (chunks not in use)
> smblks: 1
> hblks: 1 (chunks from mmap)
> hblkhd: 258048 (bytes from mmap)
> usmblks: 0
> fsmblks: 48
> uordblks: 4048432 (bytes used)
> fordblks: 2496312 (bytes free)
> keepcost: 2832 (bytes in releasable chunk)
>
> Number of files: 33769
> Number of files transferred: 4
> Total file size: 362384239 bytes
> Total transferred file size: 18814 bytes
> Literal data: 4064 bytes
> Matched data: 14750 bytes
> File list size: 557394
> Total bytes written: 236
> Total bytes read: 560149
>
> wrote 236 bytes read 560149 bytes 38647.24 bytes/sec
> total size is 362384239 speedup is 646.67
>
> WARNING: there seems to have been an rsync error.
> Check the logs for more information.
>
This run seems to have completed successfully, but your script still
complains. Maybe your script fails to reset the exit status check?
> -= Beginning backups for files
>
> receiving file list ... done
> rsync error: timeout in data send/receive (code 30) at
This is the run you're asking about, right? Have you tried something
like "--time-out=7200 -vvv"? Maybe the client is entering swap? How
many files are in this run?
-chris
> /SourceCache/rsync/rsync-12/rsync/io.c(103)
> rsync: connection unexpectedly closed (7844194 bytes read so far)
> rsync error: error in rsync protocol data stream (code 12) at io.c(150)
> Groups/Journals/SJME Omni/
> Groups/Journals/SJME Omni/Volume 8/
> rsync: connection unexpectedly closed (7844194 bytes read so far)
> rsync error: error in rsync protocol data stream (code 12) at io.c(150)
>
> WARNING: there seems to have been an rsync error.
> Check the logs for more information.
>
> -= Beginning backups for rectitude
>
> receiving file list ... done
> etc/cups/certs/
> backups/local.nidump
> backups/mkpassdb.dump
> backups/slapcat.dump
> etc/cups/certs/0
> rsync[6821] (receiver) heap statistics:
> arena: 200040 (bytes from sbrk)
> ordblks: 65 (chunks not in use)
> smblks: 3
> hblks: 0 (chunks from mmap)
> hblkhd: 0 (bytes from mmap)
> usmblks: 0
> fsmblks: 120
> uordblks: 145664 (bytes used)
> fordblks: 54376 (bytes free)
> keepcost: 18936 (bytes in releasable chunk)
>
> Number of files: 380
> Number of files transferred: 4
> Total file size: 8112964 bytes
> Total transferred file size: 2828484 bytes
> Literal data: 16617 bytes
> Matched data: 2811867 bytes
> File list size: 8325
> Total bytes written: 24331
> Total bytes read: 13736
>
> wrote 24331 bytes read 13736 bytes 8459.33 bytes/sec
> total size is 8112964 speedup is 213.12
>
> WARNING: there seems to have been an rsync error.
> Check the logs for more information.
>
> -= Beginning backups for example
>
> example is disabled. Skipping backup.
> --
> To unsubscribe or change options:
> http://lists.samba.org/mailman/listinfo/rsync
> Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
More information about the rsync
mailing list