FW: rsync hanging after working reliably for years on one of my shares

Tue Jul 17 11:36:37 MDT 2012

Thanks for the suggestions - I have done my best to follow them.

I inserted the debug shim script and changed the rsync executable in rsync so that it called that instead, and got the same results. Here is the tail from that:

21381      0.000108 open("www.athenshousing.org/xmlrpc.php", O_RDONLY|O_LARGEFILE) = 3
21381      0.000122 fstat64(3, {st_mode=S_IFREG|0644, st_size=294, ...}) = 0
21381      0.000327 read(3, "<?php\n\n/**\n * @file\n * PHP page for handling incoming XML-RPC requests from clients.\n */\n\ninclude_once './includes/boot
strap.inc';\ndrupal_bootstrap(DRUPAL_BOOTSTRAP_FULL);\ninclude_once './includes/xmlrpc.inc';\ninclude_once './includes/xmlrpcs.inc';\n\nxmlrpc_server(modul
e_invoke_all('xmlrpc'));\n", 294) = 294
21381      0.045446 close(3)            = 0
21381      0.000090 select(2, NULL, [1], [1], {60, 0}) = 1 (out [1], left {59, 999996})
21381      0.000101 write(1, "\261\5\0\7", 4) = 4
21381      0.000094 select(2, NULL, [1], [1], {60, 0}) = 1 (out [1], left {59, 999997})
21381      0.000103 write(1, "e_results']);\n  unset($_SESSION['update_success']);\n\n  return $output;\n}\n\nfunction update_info_page() {\n  // Change qu
ery-strings on css/js files to enforce reload for all users.\n  _drupal_flush_css_js();\n  // Flush the cache of all data for the update status module.\n
if (db_table_exists('cache_update')) {\n    cache_clear_all('*', 'cache_update', TRUE);\n  }\n\n  update_task_list('info');\n  drupal_set_title('Drupal dat
abase update');\n  $token = drupal_get_token('update');\n  $output = '<p>Use this utility to update your database whenever a new release of Drupal or a mod
ule is installed.</p><p>For more detailed information, see the <a href=\"http://drupal.org/upgrade\">upgrading handbook</a>. If you are unsure what these t
erms mean you should probably contact your hosting provider.</p>';\n  $output .= \"<ol>\\n\";\n  $output .= \"<li><strong>Back up your database</strong>. T
his process will change your database values and in case of emergency you may need to revert to a backup.</li>\\n\";\n  $output .= \"<li><strong>Back up yo
ur code</strong>. Hint: when backing up module code, do not leave that backup in the 'modules' or 'sites/*/modules' directories as this may confuse Drupal'
s auto-discovery mechanism.</li>\\n\";\n  $output .= '<li>Put your site into <a href=\"'. base_path() .'?q=admin/settings/site-maintenance\">maintenance mo
de</a>.</li>'.\"\\n\";\n  $output .= \"<li>\370\377\377\377\367\377\377\377\366\377\377\377\365\377\377\377\364\377\377\377\363\377\377\377\0\0\0\0\273\37\
304\335\274T\376\334\207\312!\362\t\246\333[\243b\0\0\1\0\0\0\0\10\0\0\2\0\0\0&\1\0\0\377\377\377\377\0\0\0\0\353\245\267C\35\323\353!\274\274\245\201\211\
306\3140", 1457) = 1457
21381      0.000617 select(2, NULL, [1], [1], {60, 0}) = 1 (out [1], left {59, 999997})
21381      0.000097 write(1, "\4\0\0\7\377\377\377\377", 8) = 8
21381      0.000095 select(1, [0], [], NULL, {60, 0}) = 0 (Timeout)
21381     60.060243 select(1, [0], [], NULL, {60, 0}) = 0 (Timeout)
21381     60.008189 select(1, [0], [], NULL, {60, 0}) = 0 (Timeout)
21381     60.060284 select(1, [0], [], NULL, {60, 0}) = 0 (Timeout)

Also, I did do a manual rsync of that last file using similar options, and it worked fine.

The share I am backing up contains 3 folders, an index.html and an htaccess file. No packages changed on the backup or target server in the prior 3 months. No files have changed on the target server of note in the last month (just some xml site layout files spit out by wordpress and the nightly mysql dump). No drastic changes in file size. Stumped...

Adding the /var/www share back in...hangs again...

Ideas?

--Rob

From: Rob J. Caskey
Sent: Friday, July 13, 2012 9:21 AM
To: 'rsync at lists.samba.org'
Subject: rsync hanging after working reliably for years on one of my shares

Howdy all,

I'm using rsync in conjunction with backuppc and have been backing up this share without incident for almost 3 years and it has decided to go and hang on me. Other shares on the same machine sync fine with identical settings, and nothing has changed on the backup end other than some routine dist-upgrades to keep update on security patches. I checked the apt cache and no rsync related dpkgs seemed to be hanging around in there so...

Checked files with recent ctimes and mtimes, nothing funny looking there like recursive links.

Log excerpt attached beneath. I think alphabetically xmlrpc.php may be the last file in the share, is it coincidence it hangs right after? Thoughts? Ideas? Unfounded speculation? Blatant slander?  Anything? Just about out of ideas on my end. I've got a dozen or so other hosts that are happily syncing away and passing all their unit tests but this one has greeted me the last 5 or 6 days with a sad face and I don't seem to be making any progress. Any ideas at all? Again, log snippet underneath. Let me know if this is a backuppc issue and I am barking up the wrong tree, but there is definitely an rsync process sitting out there indefinitely long.

--Rob

<snippet>
pollChild(0)
/var/lib/backuppc/pc/luna.aha/900/f%2fvar%2fwww/fwww.athenshousing.org/fxmlrpc.php cache = , invalid = , phase = 0
Sending csums for www.athenshousing.org/xmlrpc.php<http://www.athenshousing.org/xmlrpc.php> (size=294)
pollChild(0)
Sending: acf531d26f1c50f1366757a80100d5d75332c5e7e75a6d<snip, about 1k of data>
Sending csums, cnt = 0, phase = 1
pollChild()

Parent read EOF from child: fatal error!
Done: 0 files, 0 bytes
exiting after signal INT
dump failed: aborted by user (signal=INT)
link luna.aha
</snippet>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.samba.org/pipermail/rsync/attachments/20120717/ad9d5d4b/attachment.html>