if i want to delete a million files, can rsync be more faster than rm?

夏凯 walkerxk at gmail.com
Thu Jul 18 23:26:30 MDT 2013


i found two article said that rsync -a --delete empty/ a1/ will much 
more faster than rm when deleting millions files:
http://linuxnote.net/jianingy/en/linux/a-fast-way-to-remove-huge-number-of-files.html
http://www.quora.com/File-Systems/How-can-someone-rapidly-delete-400-000-files
but my test gets opposite result, and i think it's impossible on theory.
i use the command 'for i in {0..9};do nice -20 ionice -c3 touch 
${i}{0..99999}&done' to create one million files.
my script and command are as bellows:
walkerxk at tp:~/test.d$ ./test.sh &>test.log
walkerxk at tp:~/test.d$ grep . d[12].log
d1.log:1.89user 28.20system 13:40.39elapsed 3%CPU (0avgtext+0avgdata 
77632maxresident)k
d1.log:759808inputs+104outputs (0major+24631minor)pagefaults 0swaps
d2.log:3.68user 36.77system 17:54.18elapsed 3%CPU (0avgtext+0avgdata 
172128maxresident)k
d2.log:630416inputs+48outputs (4major+11255minor)pagefaults 0swaps
walkerxk at tp:~/test.d$ cat test.log
ls: cannot access d1: No such file or directory
0
0
walkerxk at tp:~/test.d$ cat test.sh
#!/bin/bash
set -e
cd ~/test.d
mkdir -p d1 d2 empty
for dir in d1 d2
do
         cd $dir
         if [[ `ls -U|wc -l` -ne 1000000 ]]
         then
                 for i in {0..9}
                 do
                         nice -20 ionice -c3 touch ${i}{0..99999}
         done
         fi
         cd ../
done
timeout 1200 time rm -fr d1 &>d1.log||echo rc=$?
ls -U d1|wc -l
timeout 1200 time rsync -a --delete empty/ d2/ &>d2.log||echo rc=$?
ls -U d2|wc -l
walkerxk at tp:~/test.d$


More information about the rsync mailing list