Suggest Rsync Performance Improvements

[Damon Atkins]

1. Large 1 MB I/O, all reads and write to file systems
1MB (can use setvbuf to do this with out coding)
  e.g a awk programme doing 8K I/O to read 2GB file
took 16 min, a perl programme doing 1MB I/O took 16
seconds.

2. When doing rsync -a /dir1/dir2 /dir3/dir4
   Do not use pipe's, as they only read/write 5k at a
time this is extremly slow, check it out with Solaris
truss. Use sockets with large I/O or shared memory on
the same system.

To see the difference try
 timex dd if=xyz of=lll bs=5k
  vs
 timex dd if=xyz of=lll bs=1024k

3. When allocating memory which will be accessed all
the time use valloc() which is
memalign(sysconf(_SC_PAGESIZE),size)

It aligns memory to the page, so memory copies are
word/page align and therefore faster, ie. 4 bytes are
copied at once instead of a byte at once.

4. If you use GigaBit ethernet to get performance you
need to read/write 64Kbytes to the socket and let the
OS break it up into MTU sized packets, or better even
send 1Mbytes to the socket. 

I would change the rsync code, but could not find any
into on the variables, so that I could safely change
the code, and known it still works.

Damon.

http://mobile.yahoo.com.au - Yahoo! Messenger for SMS
- Now send & receive IMs on your mobile via SMS