Rsync help

[Craig Barratt]

> SUN box, 2gig ram, hard drive space to spare. Rsync 2.5.5, solaris 5.7
> version 7.
> Half moon, I think it only seems to work on full moon nights.
> 
> Here's the command I run as well  .....
> /usr/local/bin/rsync --delete --partial -P -p -z -e /usr/local/bin/ssh /dir1
> systemname:/storage
>
> [snip]
> 
> > I get the following transering a large file use rsync over ssh.
> > 
> > root at pbodb bin$ ./ausbk.sh
> > building file list ... 
> > 10 files to consider 
> > ERROR: out of memory in generate_sums 
> > rsync: connection unexpectedly closed (8 bytes read so far) 
> > rsync error: error in rsync protocol data stream (code 12) at io.c(150) 

How big are the files you are trying to rsync?  It is probably failing
here:

       if (verbose > 3)
               rprintf(FINFO,"count=%d rem=%d n=%d flength=%.0f\n",
                       s->count,s->remainder,s->n,(double)s->flength);

       s->sums = (struct sum_buf *)malloc(sizeof(s->sums[0])*s->count);
       if (!s->sums) out_of_memory("generate_sums");

sizeof(s->sums[0]) is at least 32, and s->count is ceil() of file size
divided by the block size (default is 700).  So this malloc should be
around 5% of the largest file size (eg: approx 500MB for a 10GB file).
If VM is tight on your machine (you say it is intermittent) then this
might fail.

You could try -vvvv and see what the previous rprintf() shows --
unfortunately inside the loop below it also prints every checksum
when verbose > 3 so you will get a huge amount of output; just
tailing the output should be enough.

A solution is to increase the block size (eg --block-size=4096), which
reduces the malloc() needs proportionally.

Craig