rsync over ssh hang on error

[Volker Kuhlmann]

On Sat 17 Aug 2013 13:20:12 NZST +1200, Justin Pryzby wrote:

> I saw something similar on a solaris machine.  Do you know what kind of
> redirection of file descriptors is going on ?  I worked around my
> problem by using tempfiles rather than sophisticated redirections [0].

The script snippet is (on local), with all redirections:

  (
  set -x
  rsync </dev/null "${rsync_args[@]}" ...
  )

The /dev/null was my first attempt at a workaround.

The local process tree is

  +-cron,2344 -n
  |   +-cron,7757 -n
  |       +-rsnapshot,7762 -w /usr/bin/rsnapshot -x -v daily
  |       |   +-rsnapshot-to-re,7995 /root/bin/rsnapshot-to-remote --backup
  |       |       +-rsnapshot-to-re,8494 /root/bin/rsnapshot-to-remote --backup
  |       |           +-rsync,8495 -aAH --numeric-ids --no-inc-recursive --delete /local/stuff/snapshots remote:/local/backup --exclude /snapshots/_delete.*
  |       |               +-ssh,8496 kereru rsync --server -lHogDtpAre.Lsf --delete --numeric-ids . /local/backup
  |       +-sendmail,7797,mail -FCronDaemon -i -odi -oem -oi -t -f root

And the FDs are (local)

rsync /proc/8495/fd
lr-x------ 1 root root 64 2013-08-17 01:35 0 -> /dev/null
l-wx------ 1 root root 64 2013-08-17 01:35 1 -> pipe:[5947237]
l-wx------ 1 root root 64 2013-08-17 01:35 2 -> pipe:[5947237]
lr-x------ 1 root root 64 2013-08-17 01:35 3 -> /local/stuff/snapshots/...
lrwx------ 1 root root 64 2013-08-17 01:35 4 -> socket:[5952163]
lrwx------ 1 root root 64 2013-08-17 01:35 5 -> socket:[5952164]

ssh /proc/8496/fd
lrwx------ 1 root root 64 2013-08-17 14:16 0 -> socket:[5952162]
lrwx------ 1 root root 64 2013-08-17 14:16 1 -> socket:[5952165]
l-wx------ 1 root root 64 2013-08-17 14:16 2 -> pipe:[5947237]
lrwx------ 1 root root 64 2013-08-17 14:16 3 -> socket:[5949104]
lrwx------ 1 root root 64 2013-08-17 01:35 5 -> socket:[5952165]
l-wx------ 1 root root 64 2013-08-17 01:35 6 -> pipe:[5947237]

remote:

sshd (daemon) /proc/25280/fd/
lr-x------ 1 root root 64 2013-08-14 21:54 0 -> /dev/null
lrwx------ 1 root root 64 2013-08-14 21:54 1 -> socket:[5977851]
lrwx------ 1 root root 64 2013-08-14 21:54 2 -> socket:[5977851]
lrwx------ 1 root root 64 2013-08-14 21:54 3 -> socket:[5977866]
lrwx------ 1 root root 64 2013-08-14 21:54 4 -> socket:[5977876]

sshd (server) /proc/32730/fd
lrwx------ 1 root root 64 2013-08-16 02:18 0 -> /dev/null
lrwx------ 1 root root 64 2013-08-16 02:18 1 -> /dev/null
lrwx------ 1 root root 64 2013-08-16 02:18 2 -> socket:[5977851]
lrwx------ 1 root root 64 2013-08-16 02:18 3 -> socket:[6828046]
lrwx------ 1 root root 64 2013-08-16 02:18 4 -> socket:[6828088]
lr-x------ 1 root root 64 2013-08-16 02:18 5 -> pipe:[6828101]
l-wx------ 1 root root 64 2013-08-17 14:00 6 -> pipe:[6828101]
l-wx------ 1 root root 64 2013-08-17 14:00 7 -> /run/systemd/sessions/935.ref|
lr-x------ 1 root root 64 2013-08-17 14:00 10 -> pipe:[6828103]


Unfortunately the symlink destinations are not that informative and I don't know how to get more info about them.

I see you tried -T with ssh, but I take it to no avail?

What do you mean by using temp files? For the rsync output? On local or remote?

I can reproduce this problem from a local interactive shell, and with
adding -vi to rsync. The output just stops, the remote rsync process
exits, the local rsync never gets to know about that and waits forever.

Redirecting rsync stdin/out/err on local to /dev/null or file does not
solve the problem. Neither does redirecting rsync's stderr on remote,
with rsync -e myssh, and myssh being a script with ssh "$@" 2\>/tmp/6.
The stderr file is empty, stdin/out of the remote ssh contain the rsync
protocol stream.

Thanks muchly,

Volker

-- 
Volker Kuhlmann
http://volker.dnsalias.net/	Please do not CC list postings to me.