rsync 2.5.6, No buffer space available error...

[Matt Miller]

I have seen this error in many postings, but the solutions seem not to 
apply in this case.  This error happens during a small file transfer 
(gif image) and after 75% of a 165G rsync job (the building file list 
portion is complete.)  Most solutions I saw were related to not having 
enough RAM or running out of disk space.  Check out the machine specs 
below the error.

rsync: writefd_unbuffered failed to write 4830 bytes: phase "unknown": 
No buffer space available
rsync error: error in rsync protocol data stream (code 12) at io.c(515)
rsync: writefd_unbuffered failed to write 69 bytes: phase "unknown": 
Broken pipe
rsync error: error in rsync protocol data stream (code 12) at io.c(515)

uptime reports---- 11:17pm  up  8:43,  2 users,  load average: 1.65, 
1.40, 1.33

IBM x330 server
2x1.4GHz PIII
4G RAM
RedHat 7.2 (2.4.7-10smp -- required for veritas)
rsync 2.5.6
local vxfs source and destination (different filesystems) and 
destination has 800G of free space
(ideally the source will be on another machine, but I made it local to 
simplify troubleshooting... same problem with an NFS source)


rsync command used: 
    rsync -Wav --stats /sourcedir /destdir >> /opt/rsync.log 2>&1

and syslog reports:
    Aug 31 04:07:57 moss kernel: ENOMEM in do_get_write_access, retrying.
    Aug 31 04:16:05 moss kernel: journal_write_metadata_buffer: ENOMEM 
at get_unused_buffer_head, trying again.

anyone seen this?

--
Matt Miller
IT Infrastructure Team
Duke University - Fuqua School of Business