[clug] What's the fastest way of deleting files?

[Stephen Jenkin]

On Wed, 11 May 2005, Michael James wrote:

>  what's the fastest way of deleting files?
> 
> find -type f -print -exec rm {} \;	still chugging after hours

find ...|xargs /bin/rm --- should _always_ be quicker that -exec rm
(Invokes rm least number of times, versus once per file)

rm -rf should be fastest: invoke just once and it keeps track of where
it's up to...

> To get the hands on part over quickly
>  I make a TRASH directory in the same partition
>  and  "mv unwanted_dir  TRASH"

Let it run at low priority in background, possibly overnight?? [cron is
your friend]

> But then the TRASH needs to be emptied, trashd anyone?

Not possible to do quickly...
You have to:
- return all blocks to the free chain
- release all inodes [if the ref. count == 0]
- update the directories as each file is deleted

Most of this stuff is cached, so you can't really make it go faster.

Good questions to ask:
- are you keeping or deleting more files??
  - if deleting more, create a new partition, move 'wanted' there,
    zap old partition.
- How often do you do this??
  - If once in a blue moon - have to put up with the pain
  - Often?  Structure your data to support this horror stretch
    - many smaller partitions...  Pretty bad unless your using LVM's
    - a few files containing 'ext2/3' file systems "-o loop" mounted...
    - if they are write-once, read often, delete all in group,
      consider ISO filesys...
- Can you make it someone else's problem??
- Anyway to stop the system doing "sync's" while deleting??
  [Some Unixes were able to turn-off synchronous disk writes...]
  [A great way to scamble databases when disks fail]

There is no general concept of 'defrag' or 'optimise' in the world of
Unix/Linux filesystems...
Which means some tasks, backups/restores, mass-deletes, mass-creates, take
a very long time...


HTH
sj
> 
> 
> -- 
> Michael James			michael at james.st
> Network Programmer		work:   02 6246 5040
> 8 Brennan St			home:   02 6247 2556
> Hackett, ACT 2602		mobile: 04 1747 4065
> 
> Give the people control of media, they will use it.
> Don't give people control of media, and you will lose them.
> Jeff Jarvis
> -- 
> linux mailing list
> linux at lists.samba.org
> https://lists.samba.org/mailman/listinfo/linux
> 


Steve Jenkin, Unix Sys Admin
0412 786 915 (+61 412 786 915)
PO Box 48, Kippax ACT 2615, AUSTRALIA

sjenkin at canb.auug.org.au http://www.tip.net.au/~sjenkin