Problem with rsync --inplace very slow/hung on large files

[Evan Harris]

I'm trying to rsync a very large (62gig) file from one machine to another as
part of a nightly backup.  If the file does not exist at the destination, it
takes about 2.5 hours to copy in my environment.

But, if the file does exist and --inplace is specified, and the file
contents differ, rsync either is so significantly slowed as to take more
than 30 hours (the longest I've let an instance run), or it is just hung.

Running with -vvv gives this as the last few lines of the output:

match at 205401064 last_match=205401064 j=821 len=250184 n=0
match at 205651248 last_match=205651248 j=822 len=250184 n=0
match at 205901432 last_match=205901432 j=823 len=250184 n=0
match at 206151616 last_match=206151616 j=824 len=250184 n=0

at which point it has not printed anything else since I last looked at the
current run attempt about 8 hours ago.

Doing an strace on the rsync processes on the sending and receiving machines
it appears that there is still reading and writing going on, but there isn't
any output from the -vvv and I can't tell if it's really doing anything.

Is this excessive slowness just an artifact of doing an rsync --inplace on
such a large file, and it will eventually complete if let run long enough?

I would try testing without the --inplace, but the system in question
doesn't have enough disk space for two copies of that size file, which is
why I am using --inplace.

Using 2.6.3, on Debian.  Any help appreciated.

Evan