Multi-Threading?

[CACook at quantum-sci.com]

On Tuesday 17 May, 2011 10:43:21 you wrote:
> Wow! That's a lot of data you're transferring into one server, even with a GB pipe you're using, which appears to be using a fraction of it.

Yes, it's my home theater computer with all the movies.  I've just set up a RAID array and need to move my data there.

 
> At any rate, I was wondering can you break up that /home directory into several parts in terms of your "rsync" command line.
> 
> Example: rsync... /home/a...
> 	   rsync... /home/b...
>          rsync... /home/c...

I thought of this, but so many subdirectories are tiny, and it would be a fussy operation to transfer them individually.  If I needed this as a matter of course I might go to the trouble, but I am just waiiting for this to complete so I can watch some decent shows. (MythTV)

I've removed the -compress flag and added --whole-file, and am waiting for it to recalculate the transfer.  It has already done some small files, and one large file which transferred at 18.65MB/s, a big improvement over 4.5.  Still running CPU at 100% on one core.
 

On Tuesday 17 May, 2011 10:46:12 Carlos Carvalho wrote:
> You might want to check the read rate at the server, it should be
> in the hundreds of MB/s. If not rsync got stuck somewhere.

Carlos as I am CPU-bound on the server, I don't really take the disk read rate very seriously, but iotop says rsync is doing around 85M/s on these three WD Green 2TB drives.