Speeding Up Rsync for Large File Sets

Kevin Korb kmk at sanitarium.net
Fri Nov 30 10:40:16 MST 2012

Hash: SHA1

If you are using zfs then why not use zfs send and zfs receive?

Rsync has to stat every file on both ends which can take a long time
with millions of files.  The zfs tools don't have to do any of that.

On 11/30/12 12:33, Tim Gustafson wrote:
> We have a particular file system that we're trying to keep in sync 
> between two FreeBSD/ZFS servers using Rsync.
> The file system has many millions of files, and about 4TB of data 
> total.  Rsync takes HOURS to run, even when there are no files to 
> transfer.  Just the comparison itself takes hours.
> Is there any way to speed up the transfer?  The command line I'm
> using is:
> /usr/local/bin/rsync --stats --rsh=/usr/bin/rsh --recursive
> --delete --one-file-system --xattrs --links --hard-links --times
> --perms --owner --group --exclude=.zfs /foo/bar
> root at remote:/foo/bar
> Thanks!

- -- 
	Kevin Korb			Phone:    (407) 252-6853
	Systems Administrator		Internet:
	FutureQuest, Inc.		Kevin at FutureQuest.net  (work)
	Orlando, Florida		kmk at sanitarium.net (personal)
	Web page:			http://www.sanitarium.net/
	PGP public key available on web site.
Version: GnuPG v2.0.19 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://www.enigmail.net/


More information about the rsync mailing list