Seemingly impossible bug: -v not always listing every copied file

[raf]

Kyle Bassett via rsync wrote:

> Hi raf,
> 
> Curious issue you have.  A few things:
> 
> What distro(s) are you using?

debian-9

> Same rsync version on both ends?

Yes, rsync-3.1.2

> Hash of files look correct before and after the rsync?

I didn't check but I expect so. I don't expect rsync to
ever do the wrong thing. :-)

> Have you tried using inotify to monitor for changes at the fs level?  You
> should see a "read" on the sender and a "read" + "write" on the receiver.

No, but that's a good idea. I'll investigate that.

> On Tue, Oct 29, 2019, 11:25 PM raf via rsync <rsync at lists.samba.org> wrote:
> 
> > Thanks. I'll try that. But I agree that it'll be
> > something else. It's unlikely that whole trace files
> > are being overwritten, because there's locking in place
> > to prevent that sort of thing, but it's more likely
> > than anything else. I'll check that the locking is
> > working properly.
> >
> > Although, when I last investigated it, it really did
> > look as though, in a single run of this task, three
> > files were copied for one directory but only two were
> > listed. I need to verify that more thoroughly, perhaps
> > with directory listings on both sides before and after
> > each run.
> >
> > cheers,
> > raf
> >
> > Kevin Korb via rsync wrote:
> >
> > > It does seem impossible.  I would suggest adding --itemize-changes (-v
> > > isn't really all that useful without it anyway).  If entries are still
> > > missing then I would suspect that either log files are missing (maybe
> > > duplicate file names replacing the occasional log file?) or something
> > > other than rsync is doing things in the same dir.
> > >
> > > On 10/29/19 9:00 PM, raf via rsync wrote:
> > > > Hi,
> > > >
> > > > debian-9, rsync-3.1.2 (both ends)
> > > >
> > > > I have a task that rsyncs files from a list of
> > > > candidate files (--files-from=). It's verbose (-v) and
> > > > its stdout is captured to a file which is then sent to
> > > > the receiving host. The captured verbose output is
> > > > examined on the receiving host to know which files were
> > > > actually copied so that notification emails can be sent
> > > > to various people.
> > > >
> > > > The problem is that, sometimes, not all copied files
> > > > are listed in the verbose output, and so some
> > > > notification emails don't get sent out. At first, I
> > > > thought there was something wrong with the notification
> > > > emails not arriving, but the files in question, that
> > > > had definitely been copied, did not appear in any of
> > > > the captured verbose output files.
> > > >
> > > > This seems like an impossible bug but it really seems
> > > > to be happening. Has anyone else encountered behaviour
> > > > like this? I didn't have much luck searching the
> > > > internet for it. It would probably be hard to notice
> > > > if the verbose output wasn't being used for something
> > > > like triggering notifications whose absence might be
> > > > noticed.
> > > >
> > > > cheers,
> > > > raf
> > >
> > > --
> > > ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,
> > >       Kevin Korb                      Phone:    (407) 252-6853
> > >       Systems Administrator           Internet:
> > >       FutureQuest, Inc.               Kevin at FutureQuest.net  (work)
> > >       Orlando, Florida                kmk at sanitarium.net (personal)
> > >       Web page:                       https://sanitarium.net/
> > >       PGP public key available on web site.
> > > ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,