Suggest Rsync Performance Improvements
damon_atkins at yahoo.com.au
Wed Sep 11 06:56:00 EST 2002
1. Large 1 MB I/O, all reads and write to file systems
1MB (can use setvbuf to do this with out coding)
e.g a awk programme doing 8K I/O to read 2GB file
took 16 min, a perl programme doing 1MB I/O took 16
2. When doing rsync -a /dir1/dir2 /dir3/dir4
Do not use pipe's, as they only read/write 5k at a
time this is extremly slow, check it out with Solaris
truss. Use sockets with large I/O or shared memory on
the same system.
To see the difference try
timex dd if=xyz of=lll bs=5k
timex dd if=xyz of=lll bs=1024k
3. When allocating memory which will be accessed all
the time use valloc() which is
It aligns memory to the page, so memory copies are
word/page align and therefore faster, ie. 4 bytes are
copied at once instead of a byte at once.
4. If you use GigaBit ethernet to get performance you
need to read/write 64Kbytes to the socket and let the
OS break it up into MTU sized packets, or better even
send 1Mbytes to the socket.
I would change the rsync code, but could not find any
into on the variables, so that I could safely change
the code, and known it still works.
http://mobile.yahoo.com.au - Yahoo! Messenger for SMS
- Now send & receive IMs on your mobile via SMS
More information about the rsync