<html><body><div style="color:#000; background-color:#fff; font-family:HelveticaNeue, Helvetica Neue, Helvetica, Arial, Lucida Grande, sans-serif;font-size:10pt"><div>Hello,<br><br> First, I have to compliment the crew on the documentation. It's solid.<br><br>
Running on Ubuntu 12.04.3, I was using rsync to synchronize a 5T
filesystem across two servers. The default Ubuntu configuration is to
run rsync over ssh, conveniently the way I prefer it.<br><br>
Initially, it would run for hours and copy 4T or so data. Then as it
began to progress further, perhaps using more RAM, the rsync job would
run and copy some amount of data, then the client side (doing the pull)
would throw this type of error:<br><br>rsync: connection unexpectedly closed (21313238038 bytes received so far) [receiver]<br>rsync error: error in rsync protocol data stream (code 12) at io.c(605) [receiver=3.0.9]<br>[receiver]
_exit_cleanup(code=12, file=io.c, line=605): about to call exit(12)<br>rsync: connection unexpectedly closed (183 bytes received so far) [generator]<br>rsync error: unexplained error (code 255) at io.c(605) [generator=3.0.9]<br>[generator] _exit_cleanup(code=12, file=io.c, line=605): about to call exit(255)<br><br><br>It didn't take much searching to find the rsync troubleshooting page <a rel="nofollow" target="_blank" href="http://rsync.samba.org/issues.htmlhttp://">http://rsync.samba.org/issues.htmlhttp://</a>. After reading that and spending a few minutes looking around, I found this on the remote server during pulls:<br><br><br>Jan 9 19:54:45 ubuntu kernel: [371329.296889] Out of memory: Kill process 11810 (rsync) score 2 or sacrifice child<br>Jan 9 19:54:45 ubuntu kernel: [371329.296946] Killed process 11810 (rsync) total-vm:66120kB, anon-rss:0kB, file-rss:4kB<br>Jan 9 20:30:58 ubuntu kernel: [373501.815163] sshd invoked
oom-killer: gfp_mask=0x201da,
order=0, oom_adj=0, oom_score_adj=0<br><br></div><div style="color:rgb(0, 0, 0);font-size:13.3333px;font-family:HelveticaNeue, Helvetica Neue, Helvetica, Arial, Lucida Grande, sans-serif;background-color:transparent;font-style:normal;"><br><span></span></div><div style="color:rgb(0, 0, 0);font-size:13.3333px;font-family:HelveticaNeue, Helvetica Neue, Helvetica, Arial, Lucida Grande, sans-serif;background-color:transparent;font-style:normal;"><span>Root
cause: sshd was killing it's child rsync because the OS was out of
RAM. This may be an uncommon occurrence, but I thought I'd toss it into
the mix and let you decide if you want to include a tip covering this
cause of failure. I only allocated 1G RAM to it because it's a VM. I
increased the RAM to 2G and - problem solved. I would have seen it
sooner if I was watching the console.<br></span></div><div style="color:rgb(0, 0, 0);font-size:13.3333px;
font-family:HelveticaNeue, Helvetica Neue, Helvetica, Arial, Lucida Grande, sans-serif;background-color:transparent;font-style:normal;"><br><span></span></div><div class="yiv4541356992yui_3_13_0_ym1_7_1389375070411_9" style="color:rgb(0, 0, 0);font-size:13.3333px;font-family:HelveticaNeue, Helvetica Neue, Helvetica, Arial, Lucida Grande, sans-serif;background-color:transparent;font-style:normal;"><span>thanks for the awesome software.</span></div><div> </div><div>-<br>The human mind is like Van Halen. If you just pull out one piece and keep replacing it, it just degenerates.</div></div></body></html>