rsync keeps writing files over

Thu Jun 2 22:42:12 UTC 2016

Cool Thanks!

Specifically, the timestamps on both <src> and <dest> match for "ls -l"
but do not match for "ls -lu" or "ls -lc”

The storage is just an regular HDD in a mac pro tower. I can’t imagine why
it wouldn’t handle timestamps. Also of note - this problem doesn’t exist
for every file, just the vast majority. So, that just makes it more
confusing. 

Yes, I imagine rsync is not the best for linear tape but give the choice
between cp (which is faster and causes less problems but offers almost
zero verbosity) and rsync, I’ll choose rsync. If people know of other
options, I’d be very happy to know of them.

On 6/2/16, 6:33 PM, "Kevin Korb" <kmk at sanitarium.net> wrote:

>The man page has a section on what all the itemize-changes flags do.
>
>There is a --ignore-times but the result is what you have now, re-copy
>everything even if the timestamp matches.
>
>The best you can really do with storage that can't handle timestamps is
>to use --update.  But if you do that you need to get rid of --partial
>(part of -P) or else rsync will never complete a file that was partially
>copied since the partial copy is newer that the source file.
>
>OTOH, are you using rsync to copy files to a tape drive?  If so then I
>would think rsync is not the right tool for dealing with linear storage.
>
>On 06/02/2016 06:25 PM, McDowell, Blake wrote:
>> OK. Thanks. Where can I find information regarding how to interpret
>> —itemize-changes?
>> 
>> The timestamps aren’t changing, so the target must not be storing them,
>> which I have no idea why. The directory I’m writing to is 777.
>> 
>> What is the flag to tell rsync to ignore the timestamps?
>> 
>> Thanks,
>> Blake
>> 
>> On 6/2/16, 6:18 PM, "rsync on behalf of Kevin Korb"
>> <rsync-bounces at lists.samba.org on behalf of kmk at sanitarium.net> wrote:
>> 
>>> It is saying the timestamp is wrong and that it is copying the file and
>>> changing the timestamp.  If it does that every time then either the
>>> timestamps are changing on the source or the target isn't storing them.
>>>
>>> On 06/02/2016 06:13 PM, McDowell, Blake wrote:
>>>> Thanks Kevin! I¹m unclear how to read the ‹itemize-changes output. Can
>>>> you
>>>> provide some insight?
>>>>
>>>> This is a local transfer from an external drive to an internal drive
>>>>all
>>>> attached to one computer.
>>>>
>>>>
>>>> rsync -aPh --itemize-changes -n
>>>> /Volumes/shuttle_05/2012_79_1_14_1__1199_Workprint
>>>> /Volumes/3TB_LTO/LT003A/
>>>> sending incremental file list
>>>>> f..t....... 
>>>>>
>>>>> 
>>>>>2012_79_1_14_1__1199_Workprint/2012_79_1_14_1__1199_Workprint__Derivat
>>>>>iv
>>>>> es
>>>>> /2012_79_1_14_1_DER_01.mov
>>>>> f..t....... 
>>>>>
>>>>> 
>>>>>2012_79_1_14_1__1199_Workprint/2012_79_1_14_1__1199_Workprint__Derivat
>>>>>iv
>>>>> es
>>>>> /2012_79_1_14_1_DER_02.mov
>>>>> f..t....... 
>>>>>
>>>>> 
>>>>>2012_79_1_14_1__1199_Workprint/2012_79_1_14_1__1199_Workprint__UHD_DPX
>>>>>/1
>>>>> 19
>>>>> 9WP_00086400.dpx
>>>>> f..t....... 
>>>>>
>>>>> 
>>>>>2012_79_1_14_1__1199_Workprint/2012_79_1_14_1__1199_Workprint__UHD_DPX
>>>>>/1
>>>>> 19
>>>>> 9WP_00086401.dpx
>>>>> f..t....... 
>>>>>
>>>>> 
>>>>>2012_79_1_14_1__1199_Workprint/2012_79_1_14_1__1199_Workprint__UHD_DPX
>>>>>/1
>>>>> 19
>>>>> 9WP_00086402.dpx
>>>>> f..t....... 
>>>>>
>>>>> 
>>>>>2012_79_1_14_1__1199_Workprint/2012_79_1_14_1__1199_Workprint__UHD_DPX
>>>>>/1
>>>>> 19
>>>>> 9WP_00086403.dpx
>>>>> f..t....... 
>>>>>
>>>>> 
>>>>>2012_79_1_14_1__1199_Workprint/2012_79_1_14_1__1199_Workprint__UHD_DPX
>>>>>/1
>>>>> 19
>>>>> 9WP_00086404.dpx
>>>>> f..t....... 
>>>>>
>>>>> 
>>>>>2012_79_1_14_1__1199_Workprint/2012_79_1_14_1__1199_Workprint__UHD_DPX
>>>>>/1
>>>>> 19
>>>>> 9WP_00086405.dpx
>>>>> f..t....... 
>>>>>
>>>>> 
>>>>>2012_79_1_14_1__1199_Workprint/2012_79_1_14_1__1199_Workprint__UHD_DPX
>>>>>/1
>>>>> 19
>>>>> 9WP_00086406.dpx
>>>>
>>>>
>>>> ~Blake
>>>>
>>>>
>>>>
>>>> On 6/2/16, 5:44 PM, "rsync on behalf of Kevin Korb"
>>>> <rsync-bounces at lists.samba.org on behalf of kmk at sanitarium.net> wrote:
>>>>
>>>>> Instead of the second -v (or even the first) add --itemize-changes.
>>>>>It
>>>>> will tell you why each file is being copied.  If the file timestamps
>>>>> are
>>>>> not correct then perhaps the underlying storage doesn't allow
>>>>>arbitrary
>>>>> mtime changes.
>>>>>
>>>>> On 06/02/2016 05:23 PM, McDowell, Blake wrote:
>>>>>> Hi,
>>>>>>
>>>>>>  
>>>>>>
>>>>>> At my work we use rsync to move files between drives and to LTO
>>>>>>among
>>>>>> other things.
>>>>>>
>>>>>>  
>>>>>>
>>>>>> I¹m having an issue using rsync to move material between and
>>>>>>external
>>>>>> drive and an internal drive.
>>>>>>
>>>>>>  
>>>>>>
>>>>>> We run ³rsync avvPh <src> <dest>² and most of the files keep
>>>>>>writing
>>>>>> every time I run this. It appears that the modification times are
>>>>>>not
>>>>>> being carried through to the destination resulting in the files
>>>>>> continually wanting to re-write.
>>>>>>
>>>>>>  
>>>>>>
>>>>>> I¹m hoping someone can help me figure this out. Any information you
>>>>>> need
>>>>>> from me (logs, etc) I¹m happy to try and provide.
>>>>>>
>>>>>>  
>>>>>>
>>>>>> Many thanks,
>>>>>>
>>>>>> Blake  
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>> -- 
>>>>>
>>>>> 
>>>>>~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._
>>>>>.,
>>>>> 	Kevin Korb			Phone:    (407) 252-6853
>>>>> 	Systems Administrator		Internet:
>>>>> 	FutureQuest, Inc.		Kevin at FutureQuest.net  (work)
>>>>> 	Orlando, Florida		kmk at sanitarium.net (personal)
>>>>> 	Web page:			http://www.sanitarium.net/
>>>>> 	PGP public key available on web site.
>>>>>
>>>>> 
>>>>>~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._
>>>>>.,
>>>>>
>>>
>>> -- 
>>> 
>>>~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,
>>> 	Kevin Korb			Phone:    (407) 252-6853
>>> 	Systems Administrator		Internet:
>>> 	FutureQuest, Inc.		Kevin at FutureQuest.net  (work)
>>> 	Orlando, Florida		kmk at sanitarium.net (personal)
>>> 	Web page:			http://www.sanitarium.net/
>>> 	PGP public key available on web site.
>>> 
>>>~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,
>>>
>> 
>
>-- 
>~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,
>	Kevin Korb			Phone:    (407) 252-6853
>	Systems Administrator		Internet:
>	FutureQuest, Inc.		Kevin at FutureQuest.net  (work)
>	Orlando, Florida		kmk at sanitarium.net (personal)
>	Web page:			http://www.sanitarium.net/
>	PGP public key available on web site.
>~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,
>