rsync keeps writing files over

McDowell, Blake McDowellH at si.edu
Thu Jun 9 00:24:52 UTC 2016 

Hi Kevin, 

Stat is pretty cool! But, I can’t quite figure out some of the stuff it is
telling me right now.

I’ll try using —update but I’m hoping to figure out why the timestamps are
not transferring on the majority of my files.

Thanks,
Blake

On 6/2/16, 6:58 PM, "rsync on behalf of Kevin Korb"
<rsync-bounces at lists.samba.org on behalf of kmk at sanitarium.net> wrote:

>Rsync only cares about the modification time.  The ls command usually
>abbreviates the timestamp so it is better to use the stat command on one
>of the problem files to see the full thing.
>
>On 06/02/2016 06:42 PM, McDowell, Blake wrote:
>> Cool Thanks!
>> 
>> Specifically, the timestamps on both <src> and <dest> match for "ls -l"
>> but do not match for "ls -lu" or "ls -lc”
>> 
>> The storage is just an regular HDD in a mac pro tower. I can’t imagine
>>why
>> it wouldn’t handle timestamps. Also of note - this problem doesn’t exist
>> for every file, just the vast majority. So, that just makes it more
>> confusing. 
>> 
>> Yes, I imagine rsync is not the best for linear tape but give the choice
>> between cp (which is faster and causes less problems but offers almost
>> zero verbosity) and rsync, I’ll choose rsync. If people know of other
>> options, I’d be very happy to know of them.
>> 
>> On 6/2/16, 6:33 PM, "Kevin Korb" <kmk at sanitarium.net> wrote:
>> 
>>> The man page has a section on what all the itemize-changes flags do.
>>>
>>> There is a --ignore-times but the result is what you have now, re-copy
>>> everything even if the timestamp matches.
>>>
>>> The best you can really do with storage that can't handle timestamps is
>>> to use --update.  But if you do that you need to get rid of --partial
>>> (part of -P) or else rsync will never complete a file that was
>>>partially
>>> copied since the partial copy is newer that the source file.
>>>
>>> OTOH, are you using rsync to copy files to a tape drive?  If so then I
>>> would think rsync is not the right tool for dealing with linear
>>>storage.
>>>
>>> On 06/02/2016 06:25 PM, McDowell, Blake wrote:
>>>> OK. Thanks. Where can I find information regarding how to interpret
>>>> —itemize-changes?
>>>>
>>>> The timestamps aren’t changing, so the target must not be storing
>>>>them,
>>>> which I have no idea why. The directory I’m writing to is 777.
>>>>
>>>> What is the flag to tell rsync to ignore the timestamps?
>>>>
>>>> Thanks,
>>>> Blake
>>>>
>>>> On 6/2/16, 6:18 PM, "rsync on behalf of Kevin Korb"
>>>> <rsync-bounces at lists.samba.org on behalf of kmk at sanitarium.net> wrote:
>>>>
>>>>> It is saying the timestamp is wrong and that it is copying the file
>>>>>and
>>>>> changing the timestamp.  If it does that every time then either the
>>>>> timestamps are changing on the source or the target isn't storing
>>>>>them.
>>>>>
>>>>> On 06/02/2016 06:13 PM, McDowell, Blake wrote:
>>>>>> Thanks Kevin! I¹m unclear how to read the ‹itemize-changes output.
>>>>>>Can
>>>>>> you
>>>>>> provide some insight?
>>>>>>
>>>>>> This is a local transfer from an external drive to an internal drive
>>>>>> all
>>>>>> attached to one computer.
>>>>>>
>>>>>>
>>>>>> rsync -aPh --itemize-changes -n
>>>>>> /Volumes/shuttle_05/2012_79_1_14_1__1199_Workprint
>>>>>> /Volumes/3TB_LTO/LT003A/
>>>>>> sending incremental file list
>>>>>>> f..t.......
>>>>>>>
>>>>>>>
>>>>>>> 
>>>>>>>2012_79_1_14_1__1199_Workprint/2012_79_1_14_1__1199_Workprint__Deriv
>>>>>>>at
>>>>>>> iv
>>>>>>> es
>>>>>>> /2012_79_1_14_1_DER_01.mov
>>>>>>> f..t.......
>>>>>>>
>>>>>>>
>>>>>>> 
>>>>>>>2012_79_1_14_1__1199_Workprint/2012_79_1_14_1__1199_Workprint__Deriv
>>>>>>>at
>>>>>>> iv
>>>>>>> es
>>>>>>> /2012_79_1_14_1_DER_02.mov
>>>>>>> f..t.......
>>>>>>>
>>>>>>>
>>>>>>> 
>>>>>>>2012_79_1_14_1__1199_Workprint/2012_79_1_14_1__1199_Workprint__UHD_D
>>>>>>>PX
>>>>>>> /1
>>>>>>> 19
>>>>>>> 9WP_00086400.dpx
>>>>>>> f..t.......
>>>>>>>
>>>>>>>
>>>>>>> 
>>>>>>>2012_79_1_14_1__1199_Workprint/2012_79_1_14_1__1199_Workprint__UHD_D
>>>>>>>PX
>>>>>>> /1
>>>>>>> 19
>>>>>>> 9WP_00086401.dpx
>>>>>>> f..t.......
>>>>>>>
>>>>>>>
>>>>>>> 
>>>>>>>2012_79_1_14_1__1199_Workprint/2012_79_1_14_1__1199_Workprint__UHD_D
>>>>>>>PX
>>>>>>> /1
>>>>>>> 19
>>>>>>> 9WP_00086402.dpx
>>>>>>> f..t.......
>>>>>>>
>>>>>>>
>>>>>>> 
>>>>>>>2012_79_1_14_1__1199_Workprint/2012_79_1_14_1__1199_Workprint__UHD_D
>>>>>>>PX
>>>>>>> /1
>>>>>>> 19
>>>>>>> 9WP_00086403.dpx
>>>>>>> f..t.......
>>>>>>>
>>>>>>>
>>>>>>> 
>>>>>>>2012_79_1_14_1__1199_Workprint/2012_79_1_14_1__1199_Workprint__UHD_D
>>>>>>>PX
>>>>>>> /1
>>>>>>> 19
>>>>>>> 9WP_00086404.dpx
>>>>>>> f..t.......
>>>>>>>
>>>>>>>
>>>>>>> 
>>>>>>>2012_79_1_14_1__1199_Workprint/2012_79_1_14_1__1199_Workprint__UHD_D
>>>>>>>PX
>>>>>>> /1
>>>>>>> 19
>>>>>>> 9WP_00086405.dpx
>>>>>>> f..t.......
>>>>>>>
>>>>>>>
>>>>>>> 
>>>>>>>2012_79_1_14_1__1199_Workprint/2012_79_1_14_1__1199_Workprint__UHD_D
>>>>>>>PX
>>>>>>> /1
>>>>>>> 19
>>>>>>> 9WP_00086406.dpx
>>>>>>
>>>>>>
>>>>>> ~Blake
>>>>>>
>>>>>>
>>>>>>
>>>>>> On 6/2/16, 5:44 PM, "rsync on behalf of Kevin Korb"
>>>>>> <rsync-bounces at lists.samba.org on behalf of kmk at sanitarium.net>
>>>>>>wrote:
>>>>>>
>>>>>>> Instead of the second -v (or even the first) add --itemize-changes.
>>>>>>> It
>>>>>>> will tell you why each file is being copied.  If the file
>>>>>>>timestamps
>>>>>>> are
>>>>>>> not correct then perhaps the underlying storage doesn't allow
>>>>>>> arbitrary
>>>>>>> mtime changes.
>>>>>>>
>>>>>>> On 06/02/2016 05:23 PM, McDowell, Blake wrote:
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>>  
>>>>>>>>
>>>>>>>> At my work we use rsync to move files between drives and to LTO
>>>>>>>> among
>>>>>>>> other things.
>>>>>>>>
>>>>>>>>  
>>>>>>>>
>>>>>>>> I¹m having an issue using rsync to move material between and
>>>>>>>> external
>>>>>>>> drive and an internal drive.
>>>>>>>>
>>>>>>>>  
>>>>>>>>
>>>>>>>> We run ³rsync ­avvPh <src> <dest>² and most of the files keep
>>>>>>>> writing
>>>>>>>> every time I run this. It appears that the modification times are
>>>>>>>> not
>>>>>>>> being carried through to the destination resulting in the files
>>>>>>>> continually wanting to re-write.
>>>>>>>>
>>>>>>>>  
>>>>>>>>
>>>>>>>> I¹m hoping someone can help me figure this out. Any information
>>>>>>>>you
>>>>>>>> need
>>>>>>>> from me (logs, etc) I¹m happy to try and provide.
>>>>>>>>
>>>>>>>>  
>>>>>>>>
>>>>>>>> Many thanks,
>>>>>>>>
>>>>>>>> Blake  
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>> -- 
>>>>>>>
>>>>>>>
>>>>>>> 
>>>>>>>~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,
>>>>>>>._
>>>>>>> .,
>>>>>>> 	Kevin Korb			Phone:    (407) 252-6853
>>>>>>> 	Systems Administrator		Internet:
>>>>>>> 	FutureQuest, Inc.		Kevin at FutureQuest.net  (work)
>>>>>>> 	Orlando, Florida		kmk at sanitarium.net (personal)
>>>>>>> 	Web page:			http://www.sanitarium.net/
>>>>>>> 	PGP public key available on web site.
>>>>>>>
>>>>>>>
>>>>>>> 
>>>>>>>~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,
>>>>>>>._
>>>>>>> .,
>>>>>>>
>>>>>
>>>>> -- 
>>>>>
>>>>> 
>>>>>~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._
>>>>>.,
>>>>> 	Kevin Korb			Phone:    (407) 252-6853
>>>>> 	Systems Administrator		Internet:
>>>>> 	FutureQuest, Inc.		Kevin at FutureQuest.net  (work)
>>>>> 	Orlando, Florida		kmk at sanitarium.net (personal)
>>>>> 	Web page:			http://www.sanitarium.net/
>>>>> 	PGP public key available on web site.
>>>>>
>>>>> 
>>>>>~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._
>>>>>.,
>>>>>
>>>>
>>>
>>> -- 
>>> 
>>>~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,
>>> 	Kevin Korb			Phone:    (407) 252-6853
>>> 	Systems Administrator		Internet:
>>> 	FutureQuest, Inc.		Kevin at FutureQuest.net  (work)
>>> 	Orlando, Florida		kmk at sanitarium.net (personal)
>>> 	Web page:			http://www.sanitarium.net/
>>> 	PGP public key available on web site.
>>> 
>>>~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,
>>>
>> 
>
>-- 
>~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,
>	Kevin Korb			Phone:    (407) 252-6853
>	Systems Administrator		Internet:
>	FutureQuest, Inc.		Kevin at FutureQuest.net  (work)
>	Orlando, Florida		kmk at sanitarium.net (personal)
>	Web page:			http://www.sanitarium.net/
>	PGP public key available on web site.
>~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,
>

More information about the rsync mailing list