[clug] rsync question
brett.worth at gmail.com
Sat Jun 22 16:55:41 MDT 2013
On 22/06/13 20:03, Eyal Lebedinsky wrote:
> I am moving data between two arrays. I use this command
> rsync -aSv --progress rsync://root@e7/root/data1/ /data1
> to pull the data from 'e7' to a local disk (both arrays mounted on /data1 and
> 'root' is a module that exports e7:/, created just for this purpose).
> Not all the data seems to arrive. The copy did not complete but one of the large
> subdirectories shows less files in the target (71k) than the source (82k).
That's a worry. If your receiving rsync didn't error then there must have been an error
on the rsyncd. It's probably logging to syslog.
If the target ran out of inodes you should have had errors.
> Also, am I correct that rsync does *not* encrypt the data, and since the
> two arrays can do ~500MB/s I should expect the wire to be the limiting
> factor? The only CPU work is the checksum calcs.
I have never used ext4 on a filesystem larger than 16TB. Under CentOS there's been some
issue with the extutils not being up to the version that can handle big filesystems even
though ext4 can.
Another thing I've used for a speedup is to run several rsyncs in parallel after carving
up the subdirs into sort of even chunks. That's when I've had to use encryption though it
may help with the checksumming.
/) _ _ _/_/ / / / _ _//
/_)/</= / / (_(_/()/< ///
More information about the linux