measuring 2.6.1 improvements

Eric Whiting ewhiting at amis.com
Tue Apr 27 20:14:10 GMT 2004


Wayne Davison wrote:
> 
> On Tue, Apr 27, 2004 at 11:52:11AM -0600, Eric Whiting wrote:
> > ... but here is some representative data:
> 
> Thanks for the confirming stats.  Some questions/comments:
> 
> You didn't mention what command you ran, so I'm curious if the 18M jump
> at the end was due to using --delete-after?  (The way the delete options
> are written uses a lot of memory for large file sets.)

I'm using --delete, but not --delete-after. The command was something like this:

/usr/bin/rsync  -a --rsync-path=/usr/bin/rsync --delete --partial -vv -e
'/usr/local/bin/ssh ' --log-format="%o %f %b %l"  --timeout 7200   --force  
--compress   --exclude-from=/path  --exclude 'pattern' /source/dir/from/nfs
remotehost:/remote/path

I was watching the memory usage with a bash command line while loop 
while(true);do cat /proc/$pid/status |grep VmSize ;sleep 60;done

My last runs I changed the sleep to 5 at the end to get finer resolution on the
memory jump. I was not expecting to see that jump -- I was also curious.  some
more cut/paste:

dest_host:/usr/bin # while(true);do cat /proc/24624/status  |grep VmSize;sleep
60; done
VmSize:     6380 kB
VmSize:     8196 kB
VmSize:     9236 kB
VmSize:    10796 kB
VmSize:    12096 kB
VmSize:    14168 kB
VmSize:    15468 kB
VmSize:    16768 kB
VmSize:    18068 kB
VmSize:    20408 kB <- memory jump starts after this point (sender stays stable
at 22M)
VmSize:    26016 kB
(cntl-c  to change the resolution)

dest_host:/usr/bin # while(true);do cat /proc/24624/status  |grep VmSize;sleep
5; done
VmSize:    35884 kB
VmSize:    35884 kB
VmSize:    35884 kB
VmSize:    37964 kB
VmSize:    22412 kB
VmSize:    22412 kB
VmSize:    22412 kB
VmSize:    22412 kB
VmSize:    22412 kB
VmSize:    22412 kB


 
> The per-file memory improvement is probably the bulk of what you saw in
> your test.  If you want to see an even larger improvement, compare the
> use of the -H option (--hard-link) between versions (and note how much
> less data is sent over the wire too).

I'm confused. I don't think I have hard links in this tree. This improvement
will not help me without hard links will it?
 
> I'm curious if you noticed an improvement in shared memory on the
> receiving side staying shared (that is, if your fork() uses copy-
> on-write shared memory in the first place).

Not sure about my fork() -- here is the sys info:

linux kernel 2.4.21
glibc-2.3.2-6
suse 8.2

eric


More information about the rsync mailing list