Speed tests

[jw schultz]

On Wed, Nov 13, 2002 at 01:27:46PM +0200, Mozzi wrote:
> Hi all
> 
> I hope no one minds but I was asked to post my timetrial findings back 
> to the list.
> Hope it helps someone else as well, if you have any suggestions please 
> mention them as I need all the speed that I can get
> 
> The scenaro is as follows
> I have to switch mail servers and I need to copy all my mbox files over 
> to the new machine.
> As you may well know time if off the utmost importance so I need to do 
> things as quickly as possible, we are an ISP so we cannot have people 
> offline for to long.
> I have from 12:00 to 5:00 in the morning
> I conducted some tests to see what will be the most effecient to copy 
> the data over with and tried to simulate the enviroment as closely as 
> possible but on a smaller scale.
> Here are my results
> 
> Smaller test(not whole directory)
> All test were done with the same set of files that I put in a special 
> directory for the test
> there are 4593 files in the dir with a total size of 3.3G
> All tests were done from the same two machines in the same direction,I 
> just changed the options
> It was done on two Gigabit nic's with a crosover cable so network usage 
> and speed were effectively
> not a factor.
> I also tested tar but that took to long
> scp failed misserably
> I didn't test ftp at all as I cannot see that it will be faster

Stop using *  Aside from the time the shell will be spending
sorting the list, argument processing is generally
inefficient compared to directory walking.

Also, runing tar and writing to the same disk, controller or
even bus will bear no relationship to the network.
Better to do tar cf - .|rsh dest 'cd destdir; tar xf -'
The verbose option will kill performance, particularly if it
goes to a CRT. 


> 
> 
> time rsync -pogrve 'ssh -c arcfour' /var/spool/mqueue/testwind 
> root at 10.0.0.9:/var/spool/mail
> 
> wrote 3429040714 bytes  read 73492 bytes  5287762.85 bytes/sec
> total size is 3428390423  speedup is 1.00
> 
> real    10m48.081s
> user    3m1.530s
> sys     1m56.540s
> 
> -------------------------------------------------------------------------
> 
> time rsync -pogrve ssh  /var/spool/mqueue/testwind 
> root at 10.0.0.9:/var/spool/mail
> 
> wrote 3429040714 bytes  read 73492 bytes  3820740.06 bytes/sec
> total size is 3428390423  speedup is 1.00
> 
> real    14m56.454s
> user    4m58.250s
> sys     2m5.180s
> [root at ais-mail01 spool]#
> -------------------------------------------------------------------------
> 
> [root at ais-mail01 spool]# scp /var/spool/mqueue/testwind/* 
> 10.0.0.9:/var/spool/mail
> bash: /usr/bin/scp: Argument list too long
> You have new mail in /var/spool/mail/stefaans
> [root at ais-mail01 spool]#
> 
> -------------------------------------------------------------------------
> 
> time tar -cvzf sendmail.tgz *
> 
> real    34m13.435s
> user    14m17.790s
> sys     2m15.140s
> 
> -------------------------------------------------------------------------
> time rsync -arve ssh  /var/spool/mqueue/testwind 
> root at 10.0.0.9:/var/spool/mail
> 
> wrote 3429040714 bytes  read 73492 bytes  4410436.28 bytes/sec
> total size is 3428390423  speedup is 1.00
> 
> real    12m57.454s
> user    4m56.800s
> sys     1m56.860s
> 
> -------------------------------------------------------------------------
> 
> time rsync -arve 'ssh -c arcfour'  /var/spool/mqueue/testwind 
> root at 10.0.0.9:/var/spool/mail
> 
> real    9m51.771s
> user    2m55.240s
> sys     1m52.290s
> [root at ais-mail01 spool]#
> 
> Mozzi
> 
> -- 
>  _
> ( )	Ascii ribbon campaign against html email
>  X
> / \
> 
> -- 
> To unsubscribe or change options: 
> http://lists.samba.org/mailman/listinfo/rsync
> Before posting, read: http://www.tuxedo.org/~esr/faqs/smart-questions.html

-- 
________________________________________________________________
	J.W. Schultz            Pegasystems Technologies
	email address:		jw at pegasys.ws

		Remember Cernan and Schmitt