Rsync performance with large exchange database files

[bruce at sqls.net]

------ Original Message ------
From: bruce at sqls.net
To: rsync at lists.samba.org
Sent: 2/10/2014 8:38:06 AM
Subject: Rsync performance with large exchange database files

>I'm using a mixture of FreeBSD w/ ZFS+snapshots and rsync to backup all 
>the servers at my day job. This works pretty good overall but on one 
>server it's not working so well :)
>
>We have an Exchange 2003 server with 4 separate mail store databases. 
>One of them is roughly 900GB the others are ~200GB, ~160GB, and ~50GB. 
>Rsync seems to spend a lot of time trying to find the differences in 
>the files. On the Windows server where rsync is kicked off there's very 
>little CPU or RAM usage for the rsync client. On the server rsync 
>(rsyncd, no ssh) is using around 70-85% of a cpu (well, half a cpu due 
>to hyper threading). I'm using VSS on the windows server to take a 
>snapshot and expose it then running rsync from that to avoid locking 
>issues.
>
>Is there anything I should check to help narrow down "problems?" or any 
>settings I should try that could help speed things up any?
>
>Below is the final output of the last two rsync runs to give you an 
>idea. It's taking 30-40+ hours to finish even though it's only 
>transferring 80-160GB of change. Right now I'm testing this against a 
>local rsync server so it should get pretty fast network performance. 
>Eventually it will be moved to our off-site backup but that connection 
>is still pretty fast (20 MBbit) and the backup is only hitting 800-1000 
>Kbytes/sec.
>
>
>Number of files: 19
>Number of files transferred: 6
>Total file size: 1265.74G bytes
>Total transferred file size: 1057.06G bytes
>Literal data: 160.67G bytes
>Matched data: 896.39G bytes
>File list size: 482
>File list generation time: 0.001 seconds
>File list transfer time: 0.000 seconds
>Total bytes sent: 160.71G
>Total bytes received: 73.74M
>
>sent 160.71G bytes received 73.74M bytes 991.84K bytes/sec
>total size is 1265.74G speedup is 7.87
>[sender] _exit_cleanup(code=0, 
>file=/home/lapo/package/rsync-3.0.9-1/src/rsync-3.0.9/main.c, 
>line=1052): about to call exit(0)
>
>real 2833m1.324s
>user 2225m55.906s
>sys 45m10.015s
>
>Number of files: 11
>Number of files transferred: 6
>Total file size: 1268.78G bytes
>Total transferred file size: 1251.04G bytes
>Literal data: 83.43G bytes
>Matched data: 1167.61G bytes
>File list size: 216
>File list generation time: 1.360 seconds
>File list transfer time: 0.000 seconds
>Total bytes sent: 83.48G
>Total bytes received: 87.25M
>
>sent 83.48G bytes received 87.25M bytes 836.85K bytes/sec
>total size is 1268.78G speedup is 15.18
>
>real 1745m5.647s
>user 1129m14.000s
>sys 39m58.875s
>
>Thanks (in advance) for the help :)

rsync options I'm using on the client are : -rltihv --progress --stats 
--inplace --modify-window=1

On the windows client I'm using cygwin + rsync 3.0.9 but I'm going to 
test 3.1.0 there and see if there's a difference.

On the server it's rsync 3.1.0 running rsyncd.

Perhaps useful bit of information :).
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.samba.org/pipermail/rsync/attachments/20140210/a2546d50/attachment.html>