Rsync of LVM Snapshots copies whole file

Peter P GMX Prometheus001 at gmx.net
Tue Aug 26 13:02:17 GMT 2008


Hello

does nobody has a clue about this?

Best regards
Peter

Peter P GMX schrieb:
> Hello Paul,
>
> here is the status of the transfer.
> If I understand it right, about 3% of the data was changed (literal 
> data against matched data). It took about 1h40min to transfer ~250MB 
> of data.
> But to be honest, I do not get a real clue out of this status report.
> E.g. Laptop_D_10GB.vdi takes around 25 minutes to be transferred with 
> 6,74MBytes/sec. If I divide 10G by 6,74MBytes/sec (for a complete file 
> transfer) I end up at 40 minutes. So almost half of the complete data 
> transfer time of this volume is needed, although this drive is rather 
> passive and there were almost no changes on the drive between the last 
> sync. (The machines are a AMD X2 3800 on one originating side and a 
> Atom 1,6GHz on the destination side, Atom pulls from AMDs side).
>
> Best regards
> Peter
>
> receiving file list ...
> 13 files to consider
> mit_backup/Debian40.vdi
>  1395660800 100%    5.66MB/s    0:03:55 (xfer#1, to-check=11/13)
> mit_backup/Laptop_D_10GB.vdi
> 10734313984 100%    6.74MB/s    0:25:19 (xfer#2, to-check=10/13)
> mit_backup/RubyServer_1_9GB.vdi
>  2039489024 100%    8.15MB/s    0:03:58 (xfer#3, to-check=8/13)
> mit_backup/XXXX_web.vdi
>  2874171904 100%    6.17MB/s    0:07:24 (xfer#4, to-check=7/13)
> mit_backup/Ubuntu_8_04_Development.vdi
>  4367340032 100%    6.80MB/s    0:10:12 (xfer#5, to-check=6/13)
> mit_backup/Ubuntu_XXXX_3GB.vdi
>  3221238272 100%    5.61MB/s    0:09:07 (xfer#6, to-check=5/13)
> mit_backup/W2000_6GB.vdi
>  6442476032 100%    7.49MB/s    0:13:39 (xfer#7, to-check=4/13)
> mit_backup/XXXXXXXX.vdi
>  4294984192 100%    8.05MB/s    0:08:28 (xfer#8, to-check=3/13)
>
> Number of files: 13
> Number of files transferred: 8
> Total file size:             47.710.416.177 bytes
> Total transferred file size: 35.369.674.240 bytes
> Literal data:  1.181.132.240 bytes
> Matched data: 34.188.542.000 bytes
> File list size: 530
> File list generation time: 0.001 seconds
> File list transfer time: 0.000 seconds
> Total bytes sent: 3.984.972
> Total bytes received: 249.776.322
>
> sent 3.984.972 bytes  received 249.776.322 bytes  41515.14 bytes/sec
> total size is 47.710.416.177  speedup is 188.01
>
>
>
> Paul Slootman schrieb:
>> On Wed 20 Aug 2008, Peter P GMX wrote:
>>
>>  
>>> Hallo Paul,
>>>     
>>
>> please keep replies to list messages on the list, others may also be
>> able to help.
>>
>>  
>>> here is the call and the protocol for the transfer of one file. For 
>>> me  it seems that the whole file is transferred, right?
>>>     
>>
>> Nothing in the output indicates that all the file data is being
>> transferred; only that the file is indeed being transferred.
>> The output of --stats will help in showing how much file data is
>> actually transferred.
>>
>>
>> Paul Slootman
>>
>>
>>  
>>> sudo -u rsyncbackup rsync -avzP --exclude-from  
>>> /home/rsyncbackup/excludefile -e ssh  
>>> "192.168.0.3:/mnt/vdi_snapshot/VDI/mit_backup"  
>>> "/mnt/crypteddevice/linuxX2/mit_backup"
>>> receiving file list ...
>>> 14 files to consider
>>> mit_backup/
>>> mit_backup/Debian40_XXXX.vdi
>>> 0 0% 0.00kB/s 0:00:00 10235877 0% 9.69MB/s 0:02:19 21075684 1% 
>>> 10.02MB/s  0:02:13 27933044 2% 8.86MB/s 0:02:30 32490259 2% 7.72MB/s 
>>> 0:02:52  39455369 2% 6.95MB/s 0:03:10 45837776 3% 5.89MB/s 0:03:43 
>>> 50797900 3%  5.44MB/s 0:04:01 53961592 3% 5.12MB/s 0:04:16 61553523 
>>> 4% 5.27MB/s  0:04:07 67043585 4% 5.05MB/s 0:04:16 75011647 5% 
>>> 5.60MB/s 0:03:50  84224736 6% 7.00MB/s 0:03:03 89720101 6% 6.51MB/s 
>>> 0:03:15 99630301 7%  7.53MB/s 0:02:47 107071219 7% 7.63MB/s 0:02:44 
>>> 113938846 8% 7.07MB/s  0:02:56 125802777 9% 8.53MB/s 0:02:25 
>>> 132129910 9% 7.57MB/s 0:02:42  139575721 10% 7.58MB/s 0:02:41 
>>> 144926021 10% 7.23MB/s 0:02:49 149718979  10% 5.61MB/s 0:03:36 
>>> 154452981 11% 5.32MB/s 0:03:47 160614584 11%  5.00MB/s 0:04:01 
>>> 165958612 11% 5.00MB/s 0:04:00 170811085 12% 5.01MB/s  0:03:58 
>>> 175120144 12% 4.82MB/s 0:04:07 183661136 13% 5.39MB/s 0:03:39  
>>> 189571426 13% 5.52MB/s 0:03:33 195568688 14% 5.78MB/s 0:03:22 
>>> 202404006  14% 6.49MB/s 0:02:59 210410776 15% 6.35MB/s 0:03:02 
>>> 216901117 15%  6.48MB/s 0:02:57 225794756 16% 6.94MB/s 0:02:44 
>>> 233657909 16% 7.19MB/s  0:02:37 239352121 17% 6.67MB/s 0:02:49 
>>> 244896643 17% 6.44MB/s 0:02:54  249979132 17% 5.75MB/s 0:03:14 
>>> 255669742 18% 5.23MB/s 0:03:32 262874393  18% 5.58MB/s 0:03:18 
>>> 275524040 19% 7.27MB/s 0:02:30 294537718 21%  10.58MB/s 0:01:41 
>>> 306682448 21% 12.06MB/s 0:01:28 316879483 22%  12.74MB/s 0:01:22 
>>> 322410223 23% 11.06MB/s 0:01:34 331382172 23% 8.38MB/s  0:02:03 
>>> 344124151 24% 8.36MB/s 0:02:02 356277012 25% 8.84MB/s 0:01:54  
>>> 368731867 26% 10.30MB/s 0:01:37 376902647 27% 10.49MB/s 0:01:34  
>>> 382055316 27% 8.96MB/s 0:01:50 385099184 27% 6.81MB/s 0:02:24 
>>> 394514192  28% 6.14MB/s 0:02:39 399166976 28% 5.31MB/s 0:03:03 
>>> 409033762 29%  6.42MB/s 0:02:29 419680637 30% 8.02MB/s 0:01:58 
>>> 429225330 30% 7.96MB/s  0:01:58 443550036 31% 10.15MB/s 0:01:31 
>>> 452116589 32% 9.86MB/s 0:01:33  457430449 32% 8.85MB/s 0:01:43 
>>> 465762610 33% 8.68MB/s 0:01:44 471632053  33% 6.66MB/s 0:02:15 
>>> 477048104 34% 5.91MB/s 0:02:31 483430930 34%  5.93MB/s 0:02:30 
>>> 490941562 35% 5.74MB/s 0:02:33 496540662 35% 5.70MB/s  0:02:34 
>>> 501713859 35% 5.64MB/s 0:02:34 508037197 36% 5.83MB/s 0:02:28  
>>> 518535763 37% 6.53MB/s 0:02:11 526524427 37% 7.08MB/s 0:01:59 
>>> 538465877  38% 8.46MB/s 0:01:38 552112154 39% 10.15MB/s 0:01:21 
>>> 563067426 40%  10.23MB/s 0:01:19 573336231 41% 10.73MB/s 0:01:14 
>>> 574650941 41% 8.50MB/s  0:01:34 585401475 41% 7.67MB/s 0:01:43 
>>> 604962094 43% 9.70MB/s 0:01:19  614156492 44% 9.44MB/s 0:01:20 
>>> 616733228 44% 8.78MB/s 0:01:26 623854469  44% 8.19MB/s 0:01:32 
>>> 634887177 45% 6.37MB/s 0:01:56 643660294 46%  6.31MB/s 0:01:56 
>>> 651766969 46% 8.34MB/s 0:01:27 652307889 46% 6.71MB/s  0:01:48 
>>> 666193629 47% 7.38MB/s 0:01:36 675491703 48% 7.49MB/s 0:01:33  
>>> 684601774 49% 7.60MB/s 0:01:31 693070083 49% 9.52MB/s 0:01:12 
>>> 698081645  50% 7.45MB/s 0:01:31 704963446 50% 6.90MB/s 0:01:37 
>>> 707735864 50%  5.29MB/s 0:02:07 724318532 51% 7.15MB/s 0:01:31 
>>> 743526578 53% 10.03MB/s  0:01:03 754724412 54% 10.71MB/s 0:00:58 
>>> 761665355 54% 12.06MB/s 0:00:51  771483521 55% 10.55MB/s 0:00:57 
>>> 778832521 55% 8.18MB/s 0:01:13 786776637  56% 7.51MB/s 0:01:19 
>>> 794628478 56% 7.73MB/s 0:01:15 800825574 57%  6.87MB/s 0:01:24 
>>> 807411368 57% 6.69MB/s 0:01:25 813464143 58% 6.34MB/s  0:01:29 
>>> 822882968 58% 6.54MB/s 0:01:25 836248436 59% 8.16MB/s 0:01:06  
>>> 842835485 60% 8.15MB/s 0:01:06 850384164 60% 8.52MB/s 0:01:02 
>>> 862409333  61% 9.33MB/s 0:00:55 872067584 62% 8.51MB/s 0:01:00 
>>> 877114368 62%  8.14MB/s 0:01:02 882553071 63% 7.58MB/s 0:01:06 
>>> 891031774 63% 6.75MB/s  0:01:12 900651352 64% 6.72MB/s 0:01:11 
>>> 912028399 65% 8.22MB/s 0:00:57  923130092 66% 9.63MB/s 0:00:47 
>>> 930375419 66% 9.34MB/s 0:00:48 945088136  67% 10.23MB/s 0:00:43 
>>> 953608062 68% 9.57MB/s 0:00:45 960536527 68%  8.51MB/s 0:00:49 
>>> 971018224 69% 9.25MB/s 0:00:44 977655293 70% 7.66MB/s  0:00:53 
>>> 982496601 70% 6.80MB/s 0:00:59 990341428 70% 7.09MB/s 0:00:55  
>>> 996521787 71% 6.06MB/s 0:01:04 1005523756 72% 6.63MB/s 0:00:57  
>>> 1011459653 72% 6.88MB/s 0:00:54 1017321634 72% 6.41MB/s 0:00:57  
>>> 1024275542 73% 6.59MB/s 0:00:55 1032295357 73% 6.36MB/s 0:00:55  
>>> 1035005098 74% 5.12MB/s 0:01:08 1051640925 75% 7.47MB/s 0:00:44  
>>> 1058476439 75% 7.45MB/s 0:00:44 1068116134 76% 7.79MB/s 0:00:41  
>>> 1072174076 76% 8.72MB/s 0:00:36 1076894984 77% 5.90MB/s 0:00:52  
>>> 1086364486 77% 6.52MB/s 0:00:46 1093933058 78% 6.04MB/s 0:00:48  
>>> 1100717102 78% 6.75MB/s 0:00:42 1104505412 79% 6.53MB/s 0:00:43  
>>> 1109563535 79% 5.38MB/s 0:00:51 1127916628 80% 7.85MB/s 0:00:33  
>>> 1134948712 81% 7.92MB/s 0:00:32 1140284922 81% 8.33MB/s 0:00:29  
>>> 1148466121 82% 8.89MB/s 0:00:27 1148596687 82% 4.18MB/s 0:00:57  
>>> 1160963510 83% 5.26MB/s 0:00:43 1168681452 83% 5.74MB/s 0:00:38  
>>> 1175945190 84% 5.74MB/s 0:00:37 1178665232 84% 6.46MB/s 0:00:32  
>>> 1189338749 85% 6.09MB/s 0:00:33 1198811725 85% 6.47MB/s 0:00:29  
>>> 1207430133 86% 6.76MB/s 0:00:27 1216592101 87% 8.89MB/s 0:00:19  
>>> 1223955783 87% 8.06MB/s 0:00:20 1229789052 88% 7.19MB/s 0:00:22  
>>> 1255105302 89% 10.81MB/s 0:00:12 1268802266 90% 12.01MB/s 0:00:10  
>>> 1279289970 91% 12.82MB/s 0:00:08 1304782634 93% 17.29MB/s 0:00:05  
>>> 1325786214 94% 16.68MB/s 0:00:04 1353009354 96% 19.89MB/s 0:00:02  
>>> 1361234818 97% 19.35MB/s 0:00:01 1369458218 98% 15.27MB/s 0:00:01  
>>> 1379158338 98% 12.09MB/s 0:00:01 1395660800 100% 7.94MB/s 0:02:47  
>>> (xfer#1, to-check=12/14)
>>>     
>>
>>   
>


More information about the rsync mailing list