strange behavior of --inplace on ZFS
Pavel Herrmann
morpheus.ibis at gmail.com
Thu Mar 6 16:50:47 MST 2014
Hi
apologies for reordering the message parts, and for the long bit at the end
On Thursday 06 March 2014 23:45:22 Hendrik Visage wrote:
> Question: The source and destination folder host OS and are both sides ZFS?
no, the remote (source) was ext4. However, i plan to use it against at least
one NTFS machine as well
> I'd like to see some stats that rsync said it transfered, also add the
> -S flag as an extra set of tests.
--sparse had the opposite result, minimal size for zeroed file, no space saved
for random file.
combination of --sparse and --inplace is not supported
> When you are on Solaris, also see the impact of a test case using
> mkfile and not dd if=/dev/zero.
sadly, I have no Solaris in my environment, all test were done on Linux
>
> On Thu, Mar 6, 2014 at 11:17 PM, <devzero at web.de> wrote:
> > Hi Pavel,
> >
> > maybe that´s related to zfs compression ?
> >
> > on compressed zfs filesystem, zeroes are not written to disk.
compression was not turned on on the volume (unless this is enabled even if
compression is set to off)
> >
> > # dd if=/dev/zero of=test.dat bs=1024k count=100
> >
> > /zfspool # ls -la
> > total 8
> > drwxr-xr-x 3 root root 4 Feb 26 10:18 .
> > drwxr-xr-x 27 root root 4096 Mar 29 2013 ..
> > drwxr-xr-x 25 root root 25 Mar 29 2013 backup
> > -rw-r--r-- 1 root root 104857600 Feb 26 10:18 test.dat
> >
> > /zfspool # du -k test.dat
> > 1 test.dat
> >
> > /zfspool # du -k --apparent-size test.dat
> > 102400 test.dat
> >
> > despite that, space calculation on compressed fs is a difficult thing...
> >
space was as reported by 'zfs list', on a volume that was created specifically
for this test. I would assume that is the most reliable way to get space usage
> The other Question that would be interested (both with and without -S)
> is when you use the dd if=/dev/urandom created file, but change some
> places with dd =/dev/zero (ie the reverse of the A test case, creatin
> with dd if=/dev/zero and changes with dd if=/dev/urandom)
I just rerun all the tests with --sparse and --inplace, results follow
(cleanup is done after each 'zfs list', not shown)
thanks
Pavel Herrmann
zero-inited file
remote runs:
# dd if=/dev/zero of=testfile bs=1024 count=102400
# dd if=/dev/urandom of=testfile count=1 bs=1024 seek=854 conv=notrunc
# dd if=/dev/urandom of=testfile count=1 bs=1024 seek=45368 conv=notrunc
# dd if=/dev/urandom of=testfile count=50 bs=1024 seek=9647 conv=notrunc
# rsync -aHAv --stats --delete --sparse root at remote:/test/ .
receiving incremental file list
./
testfile
Number of files: 2 (reg: 1, dir: 1)
Number of created files: 1 (reg: 1)
Number of regular files transferred: 1
Total file size: 104,857,600 bytes
Total transferred file size: 104,857,600 bytes
Literal data: 104,857,600 bytes
Matched data: 0 bytes
File list size: 50
File list generation time: 0.001 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 33
Total bytes received: 104,870,500
sent 33 bytes received 104,870,500 bytes 41,948,213.20 bytes/sec
total size is 104,857,600 speedup is 1.00
# zfs snapshot zraid/test at a
# rsync -aHAv --stats --delete --sparse root at remote:/test/ .
receiving incremental file list
testfile
Number of files: 2 (reg: 1, dir: 1)
Number of created files: 0
Number of regular files transferred: 1
Total file size: 104,857,600 bytes
Total transferred file size: 104,857,600 bytes
Literal data: 10,240 bytes
Matched data: 104,847,360 bytes
File list size: 50
File list generation time: 0.001 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 71,710
Total bytes received: 51,301
sent 71,710 bytes received 51,301 bytes 82,007.33 bytes/sec
total size is 104,857,600 speedup is 852.42
# zfs snapshot zraid/test at b
# rsync -aHAv --stats --delete --sparse root at remote:/test/ .
receiving incremental file list
testfile
Number of files: 2 (reg: 1, dir: 1)
Number of created files: 0
Number of regular files transferred: 1
Total file size: 104,857,600 bytes
Total transferred file size: 104,857,600 bytes
Literal data: 10,240 bytes
Matched data: 104,847,360 bytes
File list size: 50
File list generation time: 0.001 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 71,710
Total bytes received: 51,301
sent 71,710 bytes received 51,301 bytes 82,007.33 bytes/sec
total size is 104,857,600 speedup is 852.42
# zfs snapshot zraid/test at c
# rsync -aHAv --stats --delete --sparse root at remote:/test/ .
receiving incremental file list
testfile
Number of files: 2 (reg: 1, dir: 1)
Number of created files: 0
Number of regular files transferred: 1
Total file size: 104,857,600 bytes
Total transferred file size: 104,857,600 bytes
Literal data: 61,440 bytes
Matched data: 104,796,160 bytes
File list size: 50
File list generation time: 0.001 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 71,710
Total bytes received: 102,489
sent 71,710 bytes received 102,489 bytes 116,132.67 bytes/sec
total size is 104,857,600 speedup is 601.94
# zfs list
NAME USED AVAIL REFER MOUNTPOINT
<snip>
zraid/test 1.43M 3.44T 616K legacy
# rsync -aHAv --stats --delete --inplace root at remote:/test/ .
receiving incremental file list
testfile
Number of files: 2 (reg: 1, dir: 1)
Number of created files: 0
Number of regular files transferred: 1
Total file size: 104,857,600 bytes
Total transferred file size: 104,857,600 bytes
Literal data: 104,857,600 bytes
Matched data: 0 bytes
File list size: 50
File list generation time: 0.001 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 71,710
Total bytes received: 104,870,497
sent 71,710 bytes received 104,870,497 bytes 29,983,487.71 bytes/sec
total size is 104,857,600 speedup is 1.00
# zfs snapshot zraid/test at a
# rsync -aHAv --stats --delete --inplace root at remote:/test/ .
receiving incremental file list
testfile
Number of files: 2 (reg: 1, dir: 1)
Number of created files: 0
Number of regular files transferred: 1
Total file size: 104,857,600 bytes
Total transferred file size: 104,857,600 bytes
Literal data: 10,240 bytes
Matched data: 104,847,360 bytes
File list size: 50
File list generation time: 0.001 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 71,710
Total bytes received: 51,301
sent 71,710 bytes received 51,301 bytes 82,007.33 bytes/sec
total size is 104,857,600 speedup is 852.42
# zfs snapshot zraid/test at b
# rsync -aHAv --stats --delete --inplace root at remote:/test/ .
receiving incremental file list
testfile
Number of files: 2 (reg: 1, dir: 1)
Number of created files: 0
Number of regular files transferred: 1
Total file size: 104,857,600 bytes
Total transferred file size: 104,857,600 bytes
Literal data: 10,240 bytes
Matched data: 104,847,360 bytes
File list size: 50
File list generation time: 0.001 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 71,710
Total bytes received: 51,301
sent 71,710 bytes received 51,301 bytes 82,007.33 bytes/sec
total size is 104,857,600 speedup is 852.42
# zfs snapshot zraid/test at c
# rsync -aHAv --stats --delete --inplace root at remote:/test/ .
receiving incremental file list
testfile
Number of files: 2 (reg: 1, dir: 1)
Number of created files: 0
Number of regular files transferred: 1
Total file size: 104,857,600 bytes
Total transferred file size: 104,857,600 bytes
Literal data: 61,440 bytes
Matched data: 104,796,160 bytes
File list size: 50
File list generation time: 0.001 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 71,710
Total bytes received: 102,489
sent 71,710 bytes received 102,489 bytes 69,679.60 bytes/sec
total size is 104,857,600 speedup is 601.94
# zfs list
NAME USED AVAIL REFER MOUNTPOINT
<snip>
zraid/test 346M 3.44T 100M legacy
random-init
remote runs:
# dd if=/dev/urandom of=testfile count=102400 bs=1024
# dd if=/dev/urandom of=testfile count=1 bs=1024 seek=854 conv=notrunc
# dd if=/dev/urandom of=testfile count=1 bs=1024 seek=45368 conv=notrunc
# dd if=/dev/urandom of=testfile count=50 bs=1024 seek=9647 conv=notrunc
# rsync -aHAv --stats --delete --sparse root at remote:/test/ .
receiving incremental file list
./
testfile
Number of files: 2 (reg: 1, dir: 1)
Number of created files: 1 (reg: 1)
Number of regular files transferred: 1
Total file size: 104,857,600 bytes
Total transferred file size: 104,857,600 bytes
Literal data: 104,857,600 bytes
Matched data: 0 bytes
File list size: 50
File list generation time: 0.001 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 33
Total bytes received: 104,870,500
sent 33 bytes received 104,870,500 bytes 41,948,213.20 bytes/sec
total size is 104,857,600 speedup is 1.00
# zfs snapshot zraid/test at a
# rsync -aHAv --stats --delete --sparse root at remote:/test/ .
receiving incremental file list
testfile
Number of files: 2 (reg: 1, dir: 1)
Number of created files: 0
Number of regular files transferred: 1
Total file size: 104,857,600 bytes
Total transferred file size: 104,857,600 bytes
Literal data: 10,240 bytes
Matched data: 104,847,360 bytes
File list size: 50
File list generation time: 0.001 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 71,710
Total bytes received: 51,297
sent 71,710 bytes received 51,297 bytes 82,004.67 bytes/sec
total size is 104,857,600 speedup is 852.45
# zfs snapshot zraid/test at b
# rsync -aHAv --stats --delete --sparse root at remote:/test/ .
receiving incremental file list
testfile
Number of files: 2 (reg: 1, dir: 1)
Number of created files: 0
Number of regular files transferred: 1
Total file size: 104,857,600 bytes
Total transferred file size: 104,857,600 bytes
Literal data: 10,240 bytes
Matched data: 104,847,360 bytes
File list size: 50
File list generation time: 0.001 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 71,710
Total bytes received: 51,297
sent 71,710 bytes received 51,297 bytes 82,004.67 bytes/sec
total size is 104,857,600 speedup is 852.45
# zfs snapshot zraid/test at c
# rsync -aHAv --stats --delete --sparse root at remote:/test/ .
receiving incremental file list
testfile
Number of files: 2 (reg: 1, dir: 1)
Number of created files: 0
Number of regular files transferred: 1
Total file size: 104,857,600 bytes
Total transferred file size: 104,857,600 bytes
Literal data: 61,440 bytes
Matched data: 104,796,160 bytes
File list size: 50
File list generation time: 0.001 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 71,710
Total bytes received: 102,481
sent 71,710 bytes received 102,481 bytes 69,676.40 bytes/sec
total size is 104,857,600 speedup is 601.97
# zfs list
NAME USED AVAIL REFER MOUNTPOINT
<snip>
zraid/test 401M 3.44T 100M legacy
# rsync -aHAv --stats --delete --inplace root at remote:/test/ .
receiving incremental file list
./
testfile
Number of files: 2 (reg: 1, dir: 1)
Number of created files: 1 (reg: 1)
Number of regular files transferred: 1
Total file size: 104,857,600 bytes
Total transferred file size: 104,857,600 bytes
Literal data: 104,857,600 bytes
Matched data: 0 bytes
File list size: 50
File list generation time: 0.001 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 33
Total bytes received: 104,870,500
sent 33 bytes received 104,870,500 bytes 69,913,688.67 bytes/sec
total size is 104,857,600 speedup is 1.00
# zfs snapshot zraid/test at a
# rsync -aHAv --stats --delete --inplace root at test:/test/ .
receiving incremental file list
testfile
Number of files: 2 (reg: 1, dir: 1)
Number of created files: 0
Number of regular files transferred: 1
Total file size: 104,857,600 bytes
Total transferred file size: 104,857,600 bytes
Literal data: 10,240 bytes
Matched data: 104,847,360 bytes
File list size: 50
File list generation time: 0.001 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 71,710
Total bytes received: 51,297
sent 71,710 bytes received 51,297 bytes 82,004.67 bytes/sec
total size is 104,857,600 speedup is 852.45
# zfs snapshot zraid/test at b
# rsync -aHAv --stats --delete --inplace root at remote:/test/ .
receiving incremental file list
testfile
Number of files: 2 (reg: 1, dir: 1)
Number of created files: 0
Number of regular files transferred: 1
Total file size: 104,857,600 bytes
Total transferred file size: 104,857,600 bytes
Literal data: 10,240 bytes
Matched data: 104,847,360 bytes
File list size: 50
File list generation time: 0.001 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 71,710
Total bytes received: 51,297
sent 71,710 bytes received 51,297 bytes 82,004.67 bytes/sec
total size is 104,857,600 speedup is 852.45
# zfs snapshot zraid/test at c
# rsync -aHAv --stats --delete --inplace root at remote:/test/ .
receiving incremental file list
testfile
Number of files: 2 (reg: 1, dir: 1)
Number of created files: 0
Number of regular files transferred: 1
Total file size: 104,857,600 bytes
Total transferred file size: 104,857,600 bytes
Literal data: 61,440 bytes
Matched data: 104,796,160 bytes
File list size: 50
File list generation time: 0.001 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 71,710
Total bytes received: 102,481
sent 71,710 bytes received 102,481 bytes 116,127.33 bytes/sec
total size is 104,857,600 speedup is 601.97
# zfs list
NAME USED AVAIL REFER MOUNTPOINT
<snip>
zraid/test 101M 3.44T 100M legacy
random init, zeroed holes
remote runs:
# dd if=/dev/urandom of=testfile count=102400 bs=1024
# dd if=/dev/zero of=testfile count=20000 bs=1024 seek=10 conv=notrunc
# dd if=/dev/zero of=testfile count=20000 bs=1024 seek=20010 conv=notrunc
# dd if=/dev/zero of=testfile count=20000 bs=1024 seek=40010 conv=notrunc
# dd if=/dev/zero of=testfile count=20000 bs=1024 seek=60010 conv=notrunc
# dd if=/dev/zero of=testfile count=20000 bs=1024 seek=80010 conv=notrunc
# rsync -aHAv --stats --delete --sparse root at remote:/test/ .
receiving incremental file list
./
testfile
Number of files: 2 (reg: 1, dir: 1)
Number of created files: 1 (reg: 1)
Number of regular files transferred: 1
Total file size: 104,857,600 bytes
Total transferred file size: 104,857,600 bytes
Literal data: 104,857,600 bytes
Matched data: 0 bytes
File list size: 50
File list generation time: 0.001 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 33
Total bytes received: 104,870,500
sent 33 bytes received 104,870,500 bytes 29,963,009.43 bytes/sec
total size is 104,857,600 speedup is 1.00
# zfs snapshot zraid/test at a
# rsync -aHAv --stats --delete --sparse root at remote:/test/ .
receiving incremental file list
testfile
Number of files: 2 (reg: 1, dir: 1)
Number of created files: 0
Number of regular files transferred: 1
Total file size: 104,857,600 bytes
Total transferred file size: 104,857,600 bytes
Literal data: 20,480,000 bytes
Matched data: 84,377,600 bytes
File list size: 50
File list generation time: 0.001 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 71,710
Total bytes received: 20,515,557
sent 71,710 bytes received 20,515,557 bytes 8,234,906.80 bytes/sec
total size is 104,857,600 speedup is 5.09
# zfs snapshot zraid/test at b
# rsync -aHAv --stats --delete --sparse root at remote:/test/ .
receiving incremental file list
testfile
Number of files: 2 (reg: 1, dir: 1)
Number of created files: 0
Number of regular files transferred: 1
Total file size: 104,857,600 bytes
Total transferred file size: 104,857,600 bytes
Literal data: 0 bytes
Matched data: 104,857,600 bytes
File list size: 50
File list generation time: 0.001 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 71,710
Total bytes received: 41,057
sent 71,710 bytes received 41,057 bytes 25,059.33 bytes/sec
total size is 104,857,600 speedup is 929.86
# zfs snapshot zraid/test at c
# rsync -aHAv --stats --delete --sparse root at remote:/test/ .
receiving incremental file list
testfile
Number of files: 2 (reg: 1, dir: 1)
Number of created files: 0
Number of regular files transferred: 1
Total file size: 104,857,600 bytes
Total transferred file size: 104,857,600 bytes
Literal data: 0 bytes
Matched data: 104,857,600 bytes
File list size: 50
File list generation time: 0.001 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 71,710
Total bytes received: 41,057
sent 71,710 bytes received 41,057 bytes 45,106.80 bytes/sec
total size is 104,857,600 speedup is 929.86
# zfs snapshot zraid/test at d
# rsync -aHAv --stats --delete --sparse root at remote:/test/ .
receiving incremental file list
testfile
Number of files: 2 (reg: 1, dir: 1)
Number of created files: 0
Number of regular files transferred: 1
Total file size: 104,857,600 bytes
Total transferred file size: 104,857,600 bytes
Literal data: 0 bytes
Matched data: 104,857,600 bytes
File list size: 50
File list generation time: 0.001 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 71,710
Total bytes received: 41,057
sent 71,710 bytes received 41,057 bytes 75,178.00 bytes/sec
total size is 104,857,600 speedup is 929.86
# zfs snapshot zraid/test at e
# rsync -aHAv --stats --delete --sparse root at remote:/test/ .
receiving incremental file list
testfile
Number of files: 2 (reg: 1, dir: 1)
Number of created files: 0
Number of regular files transferred: 1
Total file size: 104,857,600 bytes
Total transferred file size: 104,857,600 bytes
Literal data: 0 bytes
Matched data: 104,857,600 bytes
File list size: 50
File list generation time: 0.001 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 71,710
Total bytes received: 41,057
sent 71,710 bytes received 41,057 bytes 75,178.00 bytes/sec
total size is 104,857,600 speedup is 929.86
# zfs list
NAME USED AVAIL REFER MOUNTPOINT
<snip>
zraid/test 309M 3.44T 2.72M legacy
# rsync -aHAv --stats --delete --inplace root at remote:/test/ .
receiving incremental file list
./
testfile
Number of files: 2 (reg: 1, dir: 1)
Number of created files: 1 (reg: 1)
Number of regular files transferred: 1
Total file size: 104,857,600 bytes
Total transferred file size: 104,857,600 bytes
Literal data: 104,857,600 bytes
Matched data: 0 bytes
File list size: 50
File list generation time: 0.001 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 33
Total bytes received: 104,870,500
sent 33 bytes received 104,870,500 bytes 29,963,009.43 bytes/sec
total size is 104,857,600 speedup is 1.00
# zfs snapshot zraid/test at a
# rsync -aHAv --stats --delete --inplace root at remote:/test/ .
receiving incremental file list
testfile
Number of files: 2 (reg: 1, dir: 1)
Number of created files: 0
Number of regular files transferred: 1
Total file size: 104,857,600 bytes
Total transferred file size: 104,857,600 bytes
Literal data: 20,480,000 bytes
Matched data: 84,377,600 bytes
File list size: 50
File list generation time: 0.001 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 71,710
Total bytes received: 20,515,557
sent 71,710 bytes received 20,515,557 bytes 13,724,844.67 bytes/sec
total size is 104,857,600 speedup is 5.09
# zfs snapshot zraid/test at b
# rsync -aHAv --stats --delete --inplace root at remote:/test/ .
receiving incremental file list
testfile
Number of files: 2 (reg: 1, dir: 1)
Number of created files: 0
Number of regular files transferred: 1
Total file size: 104,857,600 bytes
Total transferred file size: 104,857,600 bytes
Literal data: 0 bytes
Matched data: 104,857,600 bytes
File list size: 50
File list generation time: 0.001 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 71,710
Total bytes received: 41,057
sent 71,710 bytes received 41,057 bytes 45,106.80 bytes/sec
total size is 104,857,600 speedup is 929.86
# zfs snapshot zraid/test at c
# rsync -aHAv --stats --delete --inplace root at remote:/test/ .
receiving incremental file list
testfile
Number of files: 2 (reg: 1, dir: 1)
Number of created files: 0
Number of regular files transferred: 1
Total file size: 104,857,600 bytes
Total transferred file size: 104,857,600 bytes
Literal data: 0 bytes
Matched data: 104,857,600 bytes
File list size: 50
File list generation time: 0.001 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 71,710
Total bytes received: 41,057
sent 71,710 bytes received 41,057 bytes 45,106.80 bytes/sec
total size is 104,857,600 speedup is 929.86
# zfs snapshot zraid/test at d
# rsync -aHAv --stats --delete --inplace root at remote:/test/ .
receiving incremental file list
testfile
Number of files: 2 (reg: 1, dir: 1)
Number of created files: 0
Number of regular files transferred: 1
Total file size: 104,857,600 bytes
Total transferred file size: 104,857,600 bytes
Literal data: 0 bytes
Matched data: 104,857,600 bytes
File list size: 50
File list generation time: 0.001 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 71,710
Total bytes received: 41,057
sent 71,710 bytes received 41,057 bytes 45,106.80 bytes/sec
total size is 104,857,600 speedup is 929.86
# zfs snapshot zraid/test at e
# rsync -aHAv --stats --delete --inplace root at remote:/test/ .
receiving incremental file list
testfile
z
Number of files: 2 (reg: 1, dir: 1)
Number of created files: 0
Number of regular files transferred: 1
Total file size: 104,857,600 bytes
Total transferred file size: 104,857,600 bytes
Literal data: 0 bytes
Matched data: 104,857,600 bytes
File list size: 50
File list generation time: 0.001 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 71,710
Total bytes received: 41,057
sent 71,710 bytes received 41,057 bytes 45,106.80 bytes/sec
total size is 104,857,600 speedup is 929.86
zalohovadlo recover # zfs list
NAME USED AVAIL REFER MOUNTPOINT
<snip>
zraid/test 198M 3.44T 100M legacy
More information about the rsync
mailing list