Rsync of LVM Snapshots copies whole file

Peter P GMX Prometheus001 at gmx.net
Wed Aug 20 20:23:59 GMT 2008


Hello Paul,

here is the status of the transfer.
If I understand it right, about 3% of the data was changed (literal data 
against matched data). It took about 1h40min to transfer ~250MB of data.
But to be honest, I do not get a real clue out of this status report.
E.g. Laptop_D_10GB.vdi takes around 25 minutes to be transferred with 
6,74MBytes/sec. If I divide 10G by 6,74MBytes/sec (for a complete file 
transfer) I end up at 40 minutes. So almost half of the complete data 
transfer time of this volume is needed, although this drive is rather 
passive and there were almost no changes on the drive between the last 
sync. (The machines are a AMD X2 3800 on one originating side and a Atom 
1,6GHz on the destination side, Atom pulls from AMDs side).

Best regards
Peter

receiving file list ...
13 files to consider
mit_backup/Debian40.vdi
  1395660800 100%    5.66MB/s    0:03:55 (xfer#1, to-check=11/13)
mit_backup/Laptop_D_10GB.vdi
 10734313984 100%    6.74MB/s    0:25:19 (xfer#2, to-check=10/13)
mit_backup/RubyServer_1_9GB.vdi
  2039489024 100%    8.15MB/s    0:03:58 (xfer#3, to-check=8/13)
mit_backup/XXXX_web.vdi
  2874171904 100%    6.17MB/s    0:07:24 (xfer#4, to-check=7/13)
mit_backup/Ubuntu_8_04_Development.vdi
  4367340032 100%    6.80MB/s    0:10:12 (xfer#5, to-check=6/13)
mit_backup/Ubuntu_XXXX_3GB.vdi
  3221238272 100%    5.61MB/s    0:09:07 (xfer#6, to-check=5/13)
mit_backup/W2000_6GB.vdi
  6442476032 100%    7.49MB/s    0:13:39 (xfer#7, to-check=4/13)
mit_backup/XXXXXXXX.vdi
  4294984192 100%    8.05MB/s    0:08:28 (xfer#8, to-check=3/13)

Number of files: 13
Number of files transferred: 8
Total file size:             47.710.416.177 bytes
Total transferred file size: 35.369.674.240 bytes
Literal data:  1.181.132.240 bytes
Matched data: 34.188.542.000 bytes
File list size: 530
File list generation time: 0.001 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 3.984.972
Total bytes received: 249.776.322

sent 3.984.972 bytes  received 249.776.322 bytes  41515.14 bytes/sec
total size is 47.710.416.177  speedup is 188.01



Paul Slootman schrieb:
> On Wed 20 Aug 2008, Peter P GMX wrote:
>
>   
>> Hallo Paul,
>>     
>
> please keep replies to list messages on the list, others may also be
> able to help.
>
>   
>> here is the call and the protocol for the transfer of one file. For me  
>> it seems that the whole file is transferred, right?
>>     
>
> Nothing in the output indicates that all the file data is being
> transferred; only that the file is indeed being transferred.
> The output of --stats will help in showing how much file data is
> actually transferred.
>
>
> Paul Slootman
>
>
>   
>> sudo -u rsyncbackup rsync -avzP --exclude-from  
>> /home/rsyncbackup/excludefile -e ssh  
>> "192.168.0.3:/mnt/vdi_snapshot/VDI/mit_backup"  
>> "/mnt/crypteddevice/linuxX2/mit_backup"
>> receiving file list ...
>> 14 files to consider
>> mit_backup/
>> mit_backup/Debian40_XXXX.vdi
>> 0 0% 0.00kB/s 0:00:00 10235877 0% 9.69MB/s 0:02:19 21075684 1% 10.02MB/s  
>> 0:02:13 27933044 2% 8.86MB/s 0:02:30 32490259 2% 7.72MB/s 0:02:52  
>> 39455369 2% 6.95MB/s 0:03:10 45837776 3% 5.89MB/s 0:03:43 50797900 3%  
>> 5.44MB/s 0:04:01 53961592 3% 5.12MB/s 0:04:16 61553523 4% 5.27MB/s  
>> 0:04:07 67043585 4% 5.05MB/s 0:04:16 75011647 5% 5.60MB/s 0:03:50  
>> 84224736 6% 7.00MB/s 0:03:03 89720101 6% 6.51MB/s 0:03:15 99630301 7%  
>> 7.53MB/s 0:02:47 107071219 7% 7.63MB/s 0:02:44 113938846 8% 7.07MB/s  
>> 0:02:56 125802777 9% 8.53MB/s 0:02:25 132129910 9% 7.57MB/s 0:02:42  
>> 139575721 10% 7.58MB/s 0:02:41 144926021 10% 7.23MB/s 0:02:49 149718979  
>> 10% 5.61MB/s 0:03:36 154452981 11% 5.32MB/s 0:03:47 160614584 11%  
>> 5.00MB/s 0:04:01 165958612 11% 5.00MB/s 0:04:00 170811085 12% 5.01MB/s  
>> 0:03:58 175120144 12% 4.82MB/s 0:04:07 183661136 13% 5.39MB/s 0:03:39  
>> 189571426 13% 5.52MB/s 0:03:33 195568688 14% 5.78MB/s 0:03:22 202404006  
>> 14% 6.49MB/s 0:02:59 210410776 15% 6.35MB/s 0:03:02 216901117 15%  
>> 6.48MB/s 0:02:57 225794756 16% 6.94MB/s 0:02:44 233657909 16% 7.19MB/s  
>> 0:02:37 239352121 17% 6.67MB/s 0:02:49 244896643 17% 6.44MB/s 0:02:54  
>> 249979132 17% 5.75MB/s 0:03:14 255669742 18% 5.23MB/s 0:03:32 262874393  
>> 18% 5.58MB/s 0:03:18 275524040 19% 7.27MB/s 0:02:30 294537718 21%  
>> 10.58MB/s 0:01:41 306682448 21% 12.06MB/s 0:01:28 316879483 22%  
>> 12.74MB/s 0:01:22 322410223 23% 11.06MB/s 0:01:34 331382172 23% 8.38MB/s  
>> 0:02:03 344124151 24% 8.36MB/s 0:02:02 356277012 25% 8.84MB/s 0:01:54  
>> 368731867 26% 10.30MB/s 0:01:37 376902647 27% 10.49MB/s 0:01:34  
>> 382055316 27% 8.96MB/s 0:01:50 385099184 27% 6.81MB/s 0:02:24 394514192  
>> 28% 6.14MB/s 0:02:39 399166976 28% 5.31MB/s 0:03:03 409033762 29%  
>> 6.42MB/s 0:02:29 419680637 30% 8.02MB/s 0:01:58 429225330 30% 7.96MB/s  
>> 0:01:58 443550036 31% 10.15MB/s 0:01:31 452116589 32% 9.86MB/s 0:01:33  
>> 457430449 32% 8.85MB/s 0:01:43 465762610 33% 8.68MB/s 0:01:44 471632053  
>> 33% 6.66MB/s 0:02:15 477048104 34% 5.91MB/s 0:02:31 483430930 34%  
>> 5.93MB/s 0:02:30 490941562 35% 5.74MB/s 0:02:33 496540662 35% 5.70MB/s  
>> 0:02:34 501713859 35% 5.64MB/s 0:02:34 508037197 36% 5.83MB/s 0:02:28  
>> 518535763 37% 6.53MB/s 0:02:11 526524427 37% 7.08MB/s 0:01:59 538465877  
>> 38% 8.46MB/s 0:01:38 552112154 39% 10.15MB/s 0:01:21 563067426 40%  
>> 10.23MB/s 0:01:19 573336231 41% 10.73MB/s 0:01:14 574650941 41% 8.50MB/s  
>> 0:01:34 585401475 41% 7.67MB/s 0:01:43 604962094 43% 9.70MB/s 0:01:19  
>> 614156492 44% 9.44MB/s 0:01:20 616733228 44% 8.78MB/s 0:01:26 623854469  
>> 44% 8.19MB/s 0:01:32 634887177 45% 6.37MB/s 0:01:56 643660294 46%  
>> 6.31MB/s 0:01:56 651766969 46% 8.34MB/s 0:01:27 652307889 46% 6.71MB/s  
>> 0:01:48 666193629 47% 7.38MB/s 0:01:36 675491703 48% 7.49MB/s 0:01:33  
>> 684601774 49% 7.60MB/s 0:01:31 693070083 49% 9.52MB/s 0:01:12 698081645  
>> 50% 7.45MB/s 0:01:31 704963446 50% 6.90MB/s 0:01:37 707735864 50%  
>> 5.29MB/s 0:02:07 724318532 51% 7.15MB/s 0:01:31 743526578 53% 10.03MB/s  
>> 0:01:03 754724412 54% 10.71MB/s 0:00:58 761665355 54% 12.06MB/s 0:00:51  
>> 771483521 55% 10.55MB/s 0:00:57 778832521 55% 8.18MB/s 0:01:13 786776637  
>> 56% 7.51MB/s 0:01:19 794628478 56% 7.73MB/s 0:01:15 800825574 57%  
>> 6.87MB/s 0:01:24 807411368 57% 6.69MB/s 0:01:25 813464143 58% 6.34MB/s  
>> 0:01:29 822882968 58% 6.54MB/s 0:01:25 836248436 59% 8.16MB/s 0:01:06  
>> 842835485 60% 8.15MB/s 0:01:06 850384164 60% 8.52MB/s 0:01:02 862409333  
>> 61% 9.33MB/s 0:00:55 872067584 62% 8.51MB/s 0:01:00 877114368 62%  
>> 8.14MB/s 0:01:02 882553071 63% 7.58MB/s 0:01:06 891031774 63% 6.75MB/s  
>> 0:01:12 900651352 64% 6.72MB/s 0:01:11 912028399 65% 8.22MB/s 0:00:57  
>> 923130092 66% 9.63MB/s 0:00:47 930375419 66% 9.34MB/s 0:00:48 945088136  
>> 67% 10.23MB/s 0:00:43 953608062 68% 9.57MB/s 0:00:45 960536527 68%  
>> 8.51MB/s 0:00:49 971018224 69% 9.25MB/s 0:00:44 977655293 70% 7.66MB/s  
>> 0:00:53 982496601 70% 6.80MB/s 0:00:59 990341428 70% 7.09MB/s 0:00:55  
>> 996521787 71% 6.06MB/s 0:01:04 1005523756 72% 6.63MB/s 0:00:57  
>> 1011459653 72% 6.88MB/s 0:00:54 1017321634 72% 6.41MB/s 0:00:57  
>> 1024275542 73% 6.59MB/s 0:00:55 1032295357 73% 6.36MB/s 0:00:55  
>> 1035005098 74% 5.12MB/s 0:01:08 1051640925 75% 7.47MB/s 0:00:44  
>> 1058476439 75% 7.45MB/s 0:00:44 1068116134 76% 7.79MB/s 0:00:41  
>> 1072174076 76% 8.72MB/s 0:00:36 1076894984 77% 5.90MB/s 0:00:52  
>> 1086364486 77% 6.52MB/s 0:00:46 1093933058 78% 6.04MB/s 0:00:48  
>> 1100717102 78% 6.75MB/s 0:00:42 1104505412 79% 6.53MB/s 0:00:43  
>> 1109563535 79% 5.38MB/s 0:00:51 1127916628 80% 7.85MB/s 0:00:33  
>> 1134948712 81% 7.92MB/s 0:00:32 1140284922 81% 8.33MB/s 0:00:29  
>> 1148466121 82% 8.89MB/s 0:00:27 1148596687 82% 4.18MB/s 0:00:57  
>> 1160963510 83% 5.26MB/s 0:00:43 1168681452 83% 5.74MB/s 0:00:38  
>> 1175945190 84% 5.74MB/s 0:00:37 1178665232 84% 6.46MB/s 0:00:32  
>> 1189338749 85% 6.09MB/s 0:00:33 1198811725 85% 6.47MB/s 0:00:29  
>> 1207430133 86% 6.76MB/s 0:00:27 1216592101 87% 8.89MB/s 0:00:19  
>> 1223955783 87% 8.06MB/s 0:00:20 1229789052 88% 7.19MB/s 0:00:22  
>> 1255105302 89% 10.81MB/s 0:00:12 1268802266 90% 12.01MB/s 0:00:10  
>> 1279289970 91% 12.82MB/s 0:00:08 1304782634 93% 17.29MB/s 0:00:05  
>> 1325786214 94% 16.68MB/s 0:00:04 1353009354 96% 19.89MB/s 0:00:02  
>> 1361234818 97% 19.35MB/s 0:00:01 1369458218 98% 15.27MB/s 0:00:01  
>> 1379158338 98% 12.09MB/s 0:00:01 1395660800 100% 7.94MB/s 0:02:47  
>> (xfer#1, to-check=12/14)
>>     
>
>   


More information about the rsync mailing list