Problems with rsync on high-loss networks

[Daniel Santos]

Hello all,

I'm using rsync over a fairly expensive satellite network that has
intermittent connectivity and a lot of packet loss.  I'm getting a lot
of corrupted file names, which is making me wonder if this is just the
one in 64k-odd packets that are corrupted with a matching TCP checksum
or if there's something else going on.

My server is using 3.1.2-2.1ubuntu1_amd64 (it's an old version of
Ubuntu, so I hack-upgraded it) run from xinetd and the clients are using
3.1.2 from Cygwin.

For example, in one case a file is expected to have a name similar to:

SH02491_20191021203744-20191021211456_1WJ1170GTKL003151.hpr

but I find:

170GTKL003151.hpr^G?~]<F3>^G<D0><F0><A4><C2>56> ^Q,6000000-20191116

In another case, this should have no subdirectories, but I find this
(with ls -alFb)

-rw-rw-rw- 1 nobody users   211346 Aug  4 18:05 3074777997.hpr
-rw-rw-rw- 1 nobody users   255851 Aug  4 19:05 3074777998.hpr
-rw-rw-rw- 1 nobody users   276949 Aug  4 20:05 3074777999.hpr
-rw-rw-rw- 1 nobody users   242214 May 30  2019 307477799.hpr
-rw-rw-rw- 1 nobody users   102301 May 23  2019 30747779.hpr
-rw-rw-rw- 1 nobody users    66197 May 22  2019 3074777.hpr
drwx------ 2 nobody users     4096 Dec 23 13:44 3.mom/
drwx------ 2 nobody users     4096 Dec 22 07:51 6\360\004[Q\030\>\ \t0.hpr/
drwx------ 2 nobody users     4096 Dec 22 07:51 6\360\004[Q\030\>\ \t1.hpr/
drwx------ 2 nobody users     4096 Dec 22 07:51 6\360\004[Q\030\>\ \t2.hpr/
drwx------ 2 nobody users     4096 Dec 22 07:51 6\360\004[Q\030\>\ \t3.hpr/
drwx------ 2 nobody users     4096 Dec 22 07:51 6\360\004[Q\030\>\ \t4.hpr/
drwx------ 2 nobody users     4096 Dec 22 07:51 6\360\004[Q\030\>\ \t5.hpr/
drwx------ 2 nobody users     4096 Dec 22 07:51 6\360\004[Q\030\>\ \t6.hpr/
drwx------ 2 nobody users     4096 Dec 22 07:51 6\360\004[Q\030\>\ \t7.hpr/
drwx------ 2 nobody users     4096 Dec 22 07:51 6\360\004[Q\030\>\ \t\0057.hpr\004\356\210]\036\2066\360h,F\ \>\ \t\0058.hpr\004#,]\260\2306\360T\=pr\002\255+]\264\3306\354\020L4\>\ \t\0058.hpr\001[\353]2.7\360\264ut%\>\ \t\0059.hpr\004ݯ]D<7\360X/
drwx------ 2 nobody users     4096 Dec 22 07:51 6\360\004[Q\030\>\ \t\0057.hpr\004\356\210]\036\2066\360h,F\ \>\ \t\0058.hpr\004#,]\260\2306\360T\=pr\002\255+]\264\3306\354\020L4\>\ \t\t\0059.hpr\003\334t]\261\3377\360\254ˈ\035\>\ \b\0057.hpr\004\305X\\/
drwx------ 2 nobody users     4096 Dec 22 07:51 6\360\004[Q\030\>\ 80.hpr/
drwx------ 2 nobody users     4096 Dec 22 07:51 6\360\004[Q\030\>\ 81.hpr/
drwx------ 2 nobody users     4096 Dec 22 07:51 6\360\004[Q\030\>\ 82.hpr/
drwx------ 2 nobody users     4096 Dec 22 07:51 6\360\004[Q\030\>\ 83.hpr/
drwx------ 2 nobody users     4096 Dec 22 07:51 6\360\004[Q\030\>\ 84.hpr/
drwx------ 2 nobody users     4096 Dec 22 07:51 6\360\004[Q\030\>\ 85.hpr/
drwx------ 2 nobody users     4096 Dec 22 07:51 6\360\004[Q\030\>\ 86.hpr/
drwx------ 2 nobody users     4096 Dec 22 07:51 6\360\004[Q\030\>\ 87.hpr/
drwx------ 2 nobody users     4096 Dec 22 07:51 6\360\004[Q\030\>\ 88.hpr/
drwx------ 2 nobody users     4096 Dec 22 07:51 6\360\004[Q\030\>\ 89.hpr/
drwx------ 2 nobody users     4096 Dec 22 07:51 6\360\004[Q\030\>\ \t8.hpr/

I haven't dug deeply into the rsync sources, but I have a few questions
or suggestions

 1. Does each logical transaction between rsync server and client employ
    any integrity guarantees beyond the TCP checksum?
 2. Can the file list be xferred as a regular /tmp file?  (I'm guessing
    the file would have to include file times, sizes, permissions, acls,
    attrs, etc.)
 3. Is there any option (that I've missed) to give rsync the hint that
    we're in the 1980s with a dial-up-like connection and we need more
    to eat more CPU (for compression and such) to save bandwidth?
 4. Is the file list compressed when using -z?  In our case, this would
    help a LOT.
 5. I don't suppose there's an option to specify the compression
    mechanism or level is there?
 6. When using -z and if files are similar enough, would it be possible
    to reuse the dictionary from a previously compressed file to send
    less data over the wire?

I have recently added "max verbosity = 3" to /etc/rsyncd.conf and
"--verbose" to the arguments when launched from xinetd and I'm getting a
dump of all the recv_file_name() calls now, but I haven't seen a
corrupted file name in the log yet.

Thanks
Daniel