rsync splits filenames, creates special characters where none are, weird permissions

[Paul Slootman]

On Wed 07 Jan 2015, Lenz Weber wrote:

> Where the local destination /data/snapshots is an NFS volume mounted with the flags
>     (rw,noatime,addr=192.168.1.XX)
> and the source is a symlink to a zfs snapshot - that looks like this:
>     /var/backups/mail -> /tank/mail/.zfs/snapshot/zfs-auto-snap_hourly-2015-01-07-1417

Why not skip the NFS part and run rsync to the destination over the
network? Rsync is written to minimize network traffic at the cost of
local IO, and if you're doing NFS then that "local IO" is really also
network traffic.  You also eliminate one potential source of problems in
that case.

> as far as I can tell, both systems work with UTF8 just fine (source is Ubuntu 14.04 and target is Debian Lenny)
> 
> Now there seems to be a problem while gathering or transferring the file list,
> as rsync tries to create files/folders that share a part with real files on the source,
> but with additional characters, sometimes cut off, without the preceding parent folder et cetera.

How often? Every file? 10% 1%? ...

> The source file names in this case look like this:
> 
> /var/backups/mail/redacted-domain/catchall/Maildir/.Sent/cur/1313508314.M654736P32713V0000000000000801I000B03CC_6.redacted-hostname,S=42352:2,S
> /var/backups/mail/redacted-domain/info/Maildir/.Sent/cur/1313508314.M654736P32713V0000000000000801I000B03CC_6.redacted-hostname,S=42352:2,S
> 
> but rsync fails on files like this, that clearly do not exist:
> 
> skipping non-regular file "2713V0000000000000801I000B03CC_6.redacted-hostname,S=423\#001\#305\#001O\#233\#240é"
> skipping non-regular file "2713V0000000000000801I000B03CC_6.redacted-hostname,S=42352:2,S"
> skipping non-regular file "83E13498714.M297793P23544V000"
> skipping non-regular file "Ø \#201"
> skipping non-regular file "redacted-domain/catchall/Maildir/.Sent/cur/1301490998.M622842P6671V0000000000000801I00280BD9_0.redacted-hostname\#004"
> skipping non-regular file "2713V0000000000000801I000B03CC_6.redacted-hostname,S=pedition/courierimapkeywords/:list"

Is this reproducible, i.e. a second run (after cleaning up the mess it
left behind) creates these same files again, or others?

My first thought is that this combination of factors is triggering some
sort of memory problems which is corrupting the filenames. It may also
be useful to do a run with --checksum to catch any data corruption (or
to see if it finds mismatches where there shouldn't).

If this can be narrowed down to a fairly small transfer which goes wrong
reproducibly, then using strace -f on rsync (with -o strace-output.txt)
then perhaps you can see whether the errors already occur when reading
the files or not.


I have not heard of rsync performing this way, so I strongly suspect
some hardware problem.


Paul