measuring 2.6.1 improvements
Eric Whiting
ewhiting at amis.com
Tue Apr 27 20:14:10 GMT 2004
Wayne Davison wrote:
>
> On Tue, Apr 27, 2004 at 11:52:11AM -0600, Eric Whiting wrote:
> > ... but here is some representative data:
>
> Thanks for the confirming stats. Some questions/comments:
>
> You didn't mention what command you ran, so I'm curious if the 18M jump
> at the end was due to using --delete-after? (The way the delete options
> are written uses a lot of memory for large file sets.)
I'm using --delete, but not --delete-after. The command was something like this:
/usr/bin/rsync -a --rsync-path=/usr/bin/rsync --delete --partial -vv -e
'/usr/local/bin/ssh ' --log-format="%o %f %b %l" --timeout 7200 --force
--compress --exclude-from=/path --exclude 'pattern' /source/dir/from/nfs
remotehost:/remote/path
I was watching the memory usage with a bash command line while loop
while(true);do cat /proc/$pid/status |grep VmSize ;sleep 60;done
My last runs I changed the sleep to 5 at the end to get finer resolution on the
memory jump. I was not expecting to see that jump -- I was also curious. some
more cut/paste:
dest_host:/usr/bin # while(true);do cat /proc/24624/status |grep VmSize;sleep
60; done
VmSize: 6380 kB
VmSize: 8196 kB
VmSize: 9236 kB
VmSize: 10796 kB
VmSize: 12096 kB
VmSize: 14168 kB
VmSize: 15468 kB
VmSize: 16768 kB
VmSize: 18068 kB
VmSize: 20408 kB <- memory jump starts after this point (sender stays stable
at 22M)
VmSize: 26016 kB
(cntl-c to change the resolution)
dest_host:/usr/bin # while(true);do cat /proc/24624/status |grep VmSize;sleep
5; done
VmSize: 35884 kB
VmSize: 35884 kB
VmSize: 35884 kB
VmSize: 37964 kB
VmSize: 22412 kB
VmSize: 22412 kB
VmSize: 22412 kB
VmSize: 22412 kB
VmSize: 22412 kB
VmSize: 22412 kB
> The per-file memory improvement is probably the bulk of what you saw in
> your test. If you want to see an even larger improvement, compare the
> use of the -H option (--hard-link) between versions (and note how much
> less data is sent over the wire too).
I'm confused. I don't think I have hard links in this tree. This improvement
will not help me without hard links will it?
> I'm curious if you noticed an improvement in shared memory on the
> receiving side staying shared (that is, if your fork() uses copy-
> on-write shared memory in the first place).
Not sure about my fork() -- here is the sys info:
linux kernel 2.4.21
glibc-2.3.2-6
suse 8.2
eric
More information about the rsync
mailing list