rsync keeps writing files over

[McDowell, Blake]

Hi Steve,

I run my first transfer always as rsync -avvPh. I now add in -i based on
what I’ve learned here.

My understanding from the rsync man page is that -a or ―archive implicitly
includes -t or ―times. Am I mistaken there? Do I need to run ―times
separately?


The output of my first transfer is:
>f++++++++++ 2012_79_1_79_1__New_Ark__Derivatives/2012_79_1_79_1_DER_01.mov
         12.09G 100%  132.21MB/s    0:01:27 (xfr#4, to-chk=13489/13497)

When I immediately re-run the same rsync command in dry-run mode I get:
>f..t....... 2012_79_1_79_1__New_Ark__Derivatives/2012_79_1_79_1_DER_01.mov

I am running about 13,000 files on this transfer and that is what is case
for every file except for a very small amount that output:
.f           2012_79_1_79_1__New_Ark__UHD_DPX/NewArk_00086558.dpx


It is a mystery to me as to why these particular files are transferring
their timestamps correctly.

Re-running the rysnc command without dry run does not make only the
timestamps transfer; the whole file retransfers and it takes the same
length of time to transfer as the first time. This is continually the
case, the timestamps never update.

I can’t imagine what would be wrong with my rsync build - I pull it
straight from homebrew/github

/Users/medialab
?:? brew install rsync
Warning: homebrew/dupes/rsync-3.1.2 already installed

Maybe the file system… Any suggested troubleshooting for that or rsync?

Thanks,
Blake
 





 

On 6/10/16, 11:43 PM, "rsync on behalf of Steven Levine"
<rsync-bounces at lists.samba.org on behalf of steve53 at earthlink.net> wrote:

>In <D37E24FA.120D%mcdowellh at si.edu>, on 06/09/16
>   at 12:17 AM, "McDowell, Blake" <McDowellH at si.edu> said:
>
>Hi Blake,
>
>Please reply to the list.
>
>>rsync -nri --modify-window=1 <src> <dest>
>
>As others mentioned, you need need to use --times.  This is needed so that
>we can see use output from --itemize-changes.
>
>>Gives me the following for most files >f..T.......
>>2015_167_1_1__Boy_What_A_Girl_R2/2015_167_1_1__Boy_What_A_Girl__UHD_DPX_R
>>2/
>>BWAG_R2_00138428.dpx
>>Although a few have  >f..T......n
>>2015_167_1_1__Boy_What_A_Girl_R2/2015_167_1_1__Boy_What_A_Girl__UHD_DPX_R
>>2/
>>BWAG_R2_00135909.dpx
>
>At a certain level this makes sense.  Without --times, the timestamps are
>not used to determine whether or not a file needs to be transferred and,
>in addition, the receiver will set the timestamp to the current time on
>the receiver.
>
>This is what the docs mean when they say
>
>"Note that if this option is not used, the optimization that excludes
>files that have not been modified  cannot  be  effective;"
>
>>(I¹m not quite sure I completely understand  -modify-window)
>
>You should not need --modify-window.  Modify window is needed when the
>source and destination file systems have differing timestamp resolutions.
>For example if transferring to a Window's filesystem that had 2-second
>timestamp resolution from a *ix system with 1-second resolution, you would
>need to use --modify-window=2 to avoid spurious transfers.
>
>>Here is a <dest> file example of timestamps as rsync interprets them:
>>-rwxrwxrwx     24,839,552 2016/06/08 13:13:19 BWAG_R2_00086400.dpx
>
>>Here is a <src> file example of timestamps as rsync interprets them:
>>-rwxrwxrwx     24,839,552 2016/05/27 13:43:32 BWAG_R2_00086400.dpx
>
>Without --times, this is the expected behavior.  The timestamps differ, so
>rsync will transfer the file.  Because the file content is the same, the
>transfer will be quick, but a tranfer will happen.
>
>If you cannot use --times, you many need to use some combination of
>--update and --ignore-times an possibly --size-only to avoid selecting
>these files for transfer.  Exactly which options will be approriate will
>depend on the content of your files and how the content changes.
>
>FWIW, if --times cannot set the timestamp correctly on the receiver, I
>would suspect an issue with the filesystem or your rsync build.  Rsync
>uses the standard platform APIs for setting the timestamps so this should
>just work.
>
>>But, for the files that have the ³n²
>
>What "n" do you mean?
>
>Another FWIW, when testing, --dry-run (i.e. -n) is useful to understand
>which files will be transferred, especially when working with complex
>filters, but you need to run without -n to see that true results of the
>transfer.
>
>Steven
>
>-- 
>----------------------------------------------------------------------
>"Steven Levine" <steve53 at earthlink.net>  Warp/DIY/BlueLion etc.
>www.scoug.com www.arcanoae.com www.warpcave.com
>----------------------------------------------------------------------
>
>
>-- 
>Please use reply-all for most replies to avoid omitting the mailing list.
>To unsubscribe or change options:
>https://lists.samba.org/mailman/listinfo/rsync
>Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html