Linux and Solaris performance

[jw schultz]

On Wed, Jul 17, 2002 at 01:52:15PM -0400, Dave North wrote:
> We have an interesting quandry here.  When I'm rsync'ing my directory
> tree (100 directories each containing 1000 files) I see some strange
> results:
> 
> All of these machines are on their own network segment (100t) in our QA
> lab
> 
> Solaris->Solaris - time taken: 11m32s
> Solaris->RH Linux 7.2 - time taken: 206s
> RH Linux->Rn Linux - time taken 1m59s
> 
> In each instance I have rsync running as a server on the target machine
> (Linux in each case).  I just can't explain why the performance is just
> so AMAZING on Linux.  Any ideas?

I can immediately see three factors that may be affecting
this.

	1. Cache.

	I'm not certain of Solaris but linux uses almost all
	free memory for caching so there may be much less
	disk I/O in your situation on the linux boxes.  

	This will be affected by kernel version, amount of
	memory, other activity on system.

	2. I/O and network subsystems.  

	I can't remember the name but you may recall a
	notorious benchmark pitting Linux against NT a
	couple of years ago.  Linux has been undergoing
	focused effort to improve performance both disk I/O
	(esp 2.5) and networking(2.3).  Solaris has not been
	held under the cross-hairs and if you talk to an SGI
	fan he'll tell you Solaris has poor network
	performance. 1+1=2

	Rsync is a poor choice for unit and system
	benchmarks.   Other tools would be better for
	checking whether I/O or network are affecting things.

	3.  Filesystem design.

	You don't say what filesystem types are involved.
	I'm assuming ext2 on Linux and UFS on Solaris.

	This might not be a major factor but could be.  The
	design of a filesystem involving directory layout,
	block allocation and tracking schemes can affect the
	time taken to add, delete and grow files.  Witness
	early XFS slowness in mass file deletion.

I can immediately see FOUR factors.

	4.  Filesystem build options.

	Block and fragment size affect not only allocation
	frequency but the presence of indirect blocks
	affects speed of file access. Less important for
	speed would be bytes per inode and cylinders per
	group.  Historically Solaris defaults have been
	piss-poor for performance so i haven't used defaults
	on that platform in more than 10 years.


I can immediately see three, no five factors...  Damn, come
in again.
	5.  Synchronous directory updates.

	UNIX has traditionally done directory ops
	synchronously.  Linux, on the other hand,  does them
	asynchronously.  In English what this means is that
	on UNIX the system calls unlink(2), rename(2) or
	open(2) will not return until any change to the
	directory file has been written to disk; but linux
	will will return from these system calls as soon as
	the in-memory structures are updated. 

	Rsync is pretty heavy into making directory changes.
	Every file updated causes two directory changes.  On
	UNIX this means that you have to pause for the disk
	I/O to complete twice for every updated file.

	This may be what accounts for the big jump when you
	change the destination from Solaris to Linux.

	Mount options may affect significantly.
	A fairly good description of this can be found at:
http://marc.theaimsgroup.com/?l=linux-kernel&m=102676438229862&w=2


-- 
________________________________________________________________
	J.W. Schultz            Pegasystems Technologies
	email address:		jw at pegasys.ws

		Remember Cernan and Schmitt