Speeding Up Rsync for Large File Sets

[Kevin Korb]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

If you are using zfs then why not use zfs send and zfs receive?

Rsync has to stat every file on both ends which can take a long time
with millions of files.  The zfs tools don't have to do any of that.

On 11/30/12 12:33, Tim Gustafson wrote:
> We have a particular file system that we're trying to keep in sync 
> between two FreeBSD/ZFS servers using Rsync.
> 
> The file system has many millions of files, and about 4TB of data 
> total.  Rsync takes HOURS to run, even when there are no files to 
> transfer.  Just the comparison itself takes hours.
> 
> Is there any way to speed up the transfer?  The command line I'm
> using is:
> 
> /usr/local/bin/rsync --stats --rsh=/usr/bin/rsh --recursive
> --delete --one-file-system --xattrs --links --hard-links --times
> --perms --owner --group --exclude=.zfs /foo/bar
> root at remote:/foo/bar
> 
> Thanks!
> 

- -- 
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~
	Kevin Korb			Phone:    (407) 252-6853
	Systems Administrator		Internet:
	FutureQuest, Inc.		Kevin at FutureQuest.net  (work)
	Orlando, Florida		kmk at sanitarium.net (personal)
	Web page:			http://www.sanitarium.net/
	PGP public key available on web site.
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://www.enigmail.net/

iEYEARECAAYFAlC474AACgkQVKC1jlbQAQemwgCbBxfuOks4q6jX/1ic8IvfLavs
VVMAoMByIhWP6PVAh3SCo5YVDKCu6xx5
=pYfe
-----END PGP SIGNATURE-----