Why does "rsync -av" always claim (incorrectly?) that it is updating write protected directories?

[Matthew Bostrom]

> On Thu, Mar 03, 2005 at 12:32:39PM -0800, Matthew Bostrom wrote:
> > "rsync -av" always claims to be updating write protected
> > directories, even when they have not been changed.  I do not
> > understand why.

On Sun, Mar 06, 2005 at 09:46:47AM -0800, Wayne Davison wrote:
> The reason is that rsync always tries to update the time on the
> directories in the transfer to match the sender, but it ignores any
> error generated from that operation because some OSes don't allow
> the time of a directory to be set to anything other than "now" by a
> normal user.  Thus, rsync attempts to set the time on these dirs on
> every run.

	Okay.

	1) Why update directory mtimes if they already match?

	2) Assuming that dir0 and dir1 are synced and writable:

$ chmod -w dir0
$ rsync -av dir0/ dir1/

	Why is no change reported here?  I can observe that rsync is
successfully updating both the permissions and the mtime of dir1.  Yet
no change is reported.

	If an error is encountered (thereby suppressing the change
notification), what system call do you suspect of returning the error
code?

	Additionally, should not the successful permission update be
reported as a change (it did indeed happen), even if the failure of a
subsequent system call is being ignored?

	3) Assuming that dir0 and dir1 are synced and writable:

$ chmod -w dir0
$ rsync -av dir0/ dir1/
$ rsync -av dir0/ dir1/
./

	If, during the second rsync, the attempt to (redundantly)
re-sync the (already synced) mtimes is failing and being ignored due
to the write-protection of the destination directory, why is that
error reported here?  I thought you said such errors were ignored?  Is
the error being ignored (as an error) but still being reported as an
(unnecessary and redundant but attempted yet failed) change?

	4) Even when the directories (source and destination) are
already write-protected, rsync is still able sync the mtimes!  I can
verify this by "touch"ing a write-protected source directory, rsyncing
it and then "stat"ing the destination directory.  So which system call
do you suspect of returning the error code (that is being ignored but
reported anyway)?

--------

	After coming up with these observations and questions I was so
confused that I decided to paw through rsync source code myself to see
what I could find.

	So... here is what is really going on:

	In generator.c, in the recv_generator function, on line 336
and following:

                /* f_out is set to -1 when doing final directory-permission
                 * and modification-time repair. */
                if (set_perms(fname, file, statret ? NULL : &st, 0)
                    && verbose && f_out != -1)
                        rprintf(FINFO, "%s/\n", safe_fname(fname));
                return;

	This is the offending rprintf - the rprintf that is printing
out the name of write-protected directories, even though they do not
need to be changed.  So, why is it printing?

	Well, let's look at rsync.c, the set_perms function, line 197
and following:

#ifdef HAVE_CHMOD
        if (!S_ISLNK(st->st_mode)) {
                if ((st->st_mode & CHMOD_BITS) != (file->mode & CHMOD_BITS)) {
                        updated = 1;
                        if (do_chmod(fname,(file->mode & CHMOD_BITS)) != 0) {
                                rsyserr(FERROR, errno, "failed to set permissions on %s",
                                        full_fname(fname));
                                return 0;
                        }
                }
        }
#endif

	As the comment at generator.c line 336 says, we pass through
this code twice.  The first time f_out is not equal to -1.  The second
time it is.

	The first time we pass through, the permissions on the write
protected directory are set from 555 to 755.  I presume this is done
so that rsync can make changes inside the directory should it need to.
This change is reported, even though this change will be undone later
on.

	The second time we pass through, the permissions on the
(formerly) write-protected directory are switched from 755 back to
555.  This change is not reported as f_out is now equal to -1.

	So, why is this happening?

	Well, at generator.c in the function generate_files on line
617 we have:

                /* we need to ensure that any directories we create have writeable
                   permissions initially so that we can create the files within
                   them. This is then fixed after the files are transferred */
                if (!am_root && S_ISDIR(file->mode) && !(file->mode & S_IWUSR)) {
                        copy = *file;
                        /* XXX: Could this be causing a problem on SCO?  Perhaps their
                         * handling of permissions is strange? */
                        copy.mode |= S_IWUSR; /* user write */
                        file = ©
                }

                recv_generator(local_name ? local_name : f_name_to(file, fbuf),
                               file, i, f_out);

	So... rsync is making sure that all directories are writeable,
and is reporting these changes even if they are not permanent.  Later
on, rsync is undoing these transient user write permission changes,
but not reporting the undos, as f_out then is equal to -1.

	Attached is a patch with a fix that addresses this problem.
It has the downside of calling recv_generator one additional time for
each write-protected directory.  I do not know how much extra overhead
this will create, but it was the cleanest way I could see to address
the problem.

	-Matthew.
______________________________________________________________________
                                                     mb at mb.mbostrom.us
-------------- next part --------------
--- generator.c.orig	Mon Sep 20 12:47:59 2004
+++ generator.c	Tue Mar  8 15:56:48 2005
@@ -610,6 +610,10 @@
 
 		if (!file->basename)
 			continue;
+
+		recv_generator(local_name ? local_name : f_name_to(file, fbuf),
+			       file, i, f_out);
+
 		/* we need to ensure that any directories we create have writeable
 		   permissions initially so that we can create the files within
 		   them. This is then fixed after the files are transferred */
@@ -619,10 +623,9 @@
 			 * handling of permissions is strange? */
 			copy.mode |= S_IWUSR; /* user write */
 			file = ©
+			recv_generator(local_name ? local_name : f_name_to(file, fbuf),
+				       file, i, -1);
 		}
-
-		recv_generator(local_name ? local_name : f_name_to(file, fbuf),
-			       file, i, f_out);
 	}
 
 	phase++;