[clug] file reorgs: moving files & maintaining directory structures

[David Deaves]

>
> Questions:
>
> 1. Are there good DeDuplication tools you can recommend based on SHA1 or MD5? I had to invent my own :(
>
> 2. What tools other people used for this sort of work? [Reorganising and partitioning by date or size]
>

1. I have also found fdupes to be useful
2. I normally just copy it all and procrastinate cleaning it up!



This set of code could probably do what you seem to want in terms of moving a bunch
of files (oldest approximately half - I took your 100,000 as accurate)


find . -type f -printf '%T@ -*- %p\n'  |        # Get files with mtime in Seconds (and decimals)
 sort -n  |     # Sort from oldest to newest
  sed 's/^[0-9.]* -*- //' |     # Prune Seconds and delimiter from front
   head -50000 |        # Select oldest half - or send this to a file and prune where you want
    cpio -pvdm /mnt/some/new/location   |  # Copy them - stdout will be the list of successful copies
     xargs -d '\n' rm           # Clean up those copied


Break the pipeline where ever you want to give you the control you need

If you want to edit the initial list to find a logical brake you may want to add %t
to your find line to get a human readable time

find . -type f -printf '%T@ %t -*- %p\n'  |     # Get files with mtime in Seconds (and decimals)
 sort -n  > /tmp/temp_file      # Sort from oldest to newest


Dave !