--itemize-changes not recursive (not printing created files inside created directories)

Kevin Korb kmk at sanitarium.net
Wed Dec 18 11:21:46 MST 2013


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

- -c checksums everything *BEFORE* transferring anything.  It would only
validate an rsync if run *AFTER* an rsync has already finished and
nothing has changed since the rsync ran.

Also, rsync's -c is rather dumb as it computes checksums for files
that have different sizes so they can't possibly be the same and it
computes checksums for files that only exist on one end and therefore
has nothing to compare them to.

On 12/18/2013 11:45 AM, Richard Michael wrote:
> Kevin,
> 
> Thanks for your reply.
> 
> On Tue, Dec 17, 2013 at 6:49 PM, Kevin Korb <kmk at sanitarium.net>
> wrote: You are running with -n (--dry-run)  It isn't logging any
> transfers because it isn't actually doing any.  Your command should
> show the dry run details on stdout but only what it actually
> transferred (nothing) in the log file.  If you drop the -n it will
> actually make the changes and log them.
> 
>> Ah yes, I should have checked this, thanks!
> 
>> I should prepare a tiny man page patch mentioning --dry-run will 
>> affect --log-file output.
> 
>> I suppose for my use-case, I should ignore --log-file and simply 
>> capture stdout, because they are nearly identical except for the 
>> misleading missing transfers:
> 
>> $ diff -B <(cut -d' ' -f 4- rsync.test1.log) rsync.test1.stdout 
>> 1c1 < building file list ---
> sending incremental file list
>> 8a9
>>>> f+++++++++
>>>> S-1-5-21-947408474-2404634453-201028329-3641/desktop.ini
>> 9a11
>>>> f+++++++++
>>>> S-1-5-21-947408474-2404634453-201028329-3642/desktop.ini
>> 10a13
>>>> f+++++++++
>>>> S-1-5-21-947408474-2404634453-201028329-500/desktop.ini
> 
> 
> Note also that you shouldn't use -c unless you have a really good
> (and unusual) reason.
> 
>> "shouldn't" ?  I'm aware it will take longer, and don't mind.
>> I'm thoroughly testing an rsync of two NTFS filesystems, and want
>> to avoid any time precision issues on at least one test run.
> 
> 
>> Regards, Richard
> 
> 
> 
> On 12/17/2013 06:19 PM, Richard Michael wrote:
>>>> Hello list,
>>>> 
>>>> Using rsync  version 3.1.0pre1  protocol version 31.PR14, I
>>>> ran this command to sync two local filesystems:
>>>> 
>>>> rsync -rvvctpgonh --itemize-changes --progress --stats 
>>>> --log-file=/root/rsync.log /mnt/ntfs/vmdk/
>>>> /mnt/ntfs/restore/
>>>> 
>>>> -r = recursive -vv = verbose -c = checksum compare -t =
>>>> transfer modification times -p = set destination perms to
>>>> match source -g = set destination group to match source -o =
>>>> set destination owner to match source -n = dry run (should
>>>> cause --progress to be meaningless, but harmless) -h = human
>>>> readable output
>>>> 
>>>> 
>>>> Output includes this summary:
>>>> 
>>>> 2013/12/11 16:35:14 [3349] Number of files: 443,967 (reg:
>>>> 419,427, dir: 24,540) 2013/12/11 16:35:14 [3349] Number of
>>>> created files: 2,581 (reg: 2,511, dir: 70) 2013/12/11
>>>> 16:35:14 [3349] Number of deleted files: 0 2013/12/11
>>>> 16:35:14 [3349] Number of regular files transferred: 2,744
>>>> 2013/12/11 16:35:14 [3349] Total file size: 560.66G bytes
>>>> 2013/12/11 16:35:14 [3349] Total transferred file size:
>>>> 196.26G bytes 2013/12/11 16:35:14 [3349] Literal data: 0 
>>>> bytes 2013/12/11 16:35:14 [3349] Matched data: 0 bytes
>>>> 2013/12/11 16:35:14 [3349] File list size: 589.75K 2013/12/11
>>>> 16:35:14 [3349] File list generation time: 0.047 seconds
>>>> 2013/12/11 16:35:14 [3349] File list transfer time: 0.000
>>>> seconds 2013/12/11 16:35:14 [3349] Total bytes sent: 20.97M
>>>> 2013/12/11 16:35:14 [3349] Total bytes received: 1.36M
>>>> 2013/12/11 16:35:14 [3349] sent 20.97M bytes received 1.36M
>>>> bytes 2.28K bytes/sec 2013/12/11 16:35:14 [3349] total size
>>>> is 560.66G  speedup is 25,111.56 (DRY RUN)
>>>> 
>>>> 
>>>> --itemize-changes has output 70 lines pertaining to created 
>>>> directories (matches the summary output above), similar to
>>>> this:
>>>> 
>>>> 2013/12/11 13:54:23 [3349] cd+++++++++ 
>>>> $RECYCLE.BIN/S-1-5-21-947408474-2404634453-201028329-3641/
>>>> 
>>>> $ awk '$4 ~ /^cd/' rsync.log | wc -l   # ==> 70
>>>> 
>>>> However, the log file contains 0 occurrences of '>', '<'  or 
>>>> 'cf++++++++' .
>>>> 
>>>> 
>>>> A bit of arithmetic indicates exactly the "Number of regular
>>>> files transferred" ( = 2,744 from stats output) are missing
>>>> from the log:
>>>> 
>>>> $ wc -l rsync.log # ==> 441279 ( - 56 lines of stats/info +
>>>> 2,744 unreported files = 443967 files/dirs, as summarized)
>>>> 
>>>> 
>>>> Questions:
>>>> 
>>>> 1/ Why doesn't --itemize-changes report "cf+++++++++" lines
>>>> for files contained inside created directories ("cd+++++++")?
>>>> For example, I expected output similar to: (faked output)
>>>> 
>>>> 2013/12/11 13:54:23 [3349] cd+++++++++ 
>>>> $RECYCLE.BIN/S-1-5-21-947408474-2404634453-201028329-3641/ 
>>>> 2013/12/11 13:54:23 [3349] cf+++++++++ 
>>>> $RECYCLE.BIN/S-1-5-21-947408474-2404634453-201028329-3641/desktop.ini
>>>>
>>>>
>>>>
>>>>
>>>> 
2/ Of the 2,744 transferred files, 2,581 are created (I expect
>>>> each, dir or file, as a "c[fd]+++++++") ; this leaves 163 
>>>> transferred files, presumably because they were changed (but
>>>> not created), also not reported.  Why were they not
>>>> reported?
>>>> 
>>>> 
>>>> Ultimately, I would like a list of all differences, both file
>>>> and directory, sent to the client.  I suspect
>>>> --itemize-changes is a bit archaic, and I should be
>>>> investigating --info= customized output.
>>>> 
>>>> Thank you!
>>>> 
>>>> Regards, Richard
>>>> 
> 
>> -- Please use reply-all for most replies to avoid omitting the
>> mailing list. To unsubscribe or change options:
>> https://lists.samba.org/mailman/listinfo/rsync Before posting,
>> read: http://www.catb.org/~esr/faqs/smart-questions.html

- -- 
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~
	Kevin Korb			Phone:    (407) 252-6853
	Systems Administrator		Internet:
	FutureQuest, Inc.		Kevin at FutureQuest.net  (work)
	Orlando, Florida		kmk at sanitarium.net (personal)
	Web page:			http://www.sanitarium.net/
	PGP public key available on web site.
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iEYEARECAAYFAlKx57kACgkQVKC1jlbQAQfgYACfZDIKobBo0vO8BsFdn7YOesvM
RcgAoMORYlmTmoZiof4IIioyGDhtw7H+
=eKNH
-----END PGP SIGNATURE-----


More information about the rsync mailing list