[clug] Dependency resolution for non-source code

[David Schoen]

On 6 February 2011 22:24, Paul Wayper <paulway at mabula.net> wrote:
> Basically I produce .wav files, which then get transcoded into .mp3 and .ogg
> files, possibly more than one each (e.g. high bitrate, low bitrate), and then
> torrent files produced for the resultant .mp3 and .ogg files.  Occasionally I
> might need to regenerate all the files, if some bit of metadata has changed,
> or just some of them - if I add a new tracker to the torrent files, for
> instance.  The names too obey logical patterns but are not the same throughout.

Hi Paul,

I started on the same track for maintaining multiple versions of my
music collection (mostly FLAC from my audio CDs) for iPod, car, phone
etc...

At first I started off just trying to do everything in a Makefile with
a two stage process that initially split out the metadata to a
separate text file and then had the final file simply depend on those
two. But I found all the different types of metadata too painful to
deal with.

I ended up modifying Perl Audio Converter (
http://pacpl.sourceforge.net/ ) to support some basic logic to support
running over the same structure multiple times. PACPL tries to some of
what you wanted, but fails to do it in such a way that makes sense if
you're exporting the same directory again and again.

First thing I changed was to check the source and destination
timestamps and only output a file if the destination doesn't exist or
is older than the source file. The second thing I changed was to
output via temporary files, e.g. ".foo.tmp.mp3", that way if something
goes wrong, the script can adequately clean up and restore a
consistent state automatically on subsequent runs. I also added a
--cleandest argument, which is roughly similar to rsync's --delete
argument.

Changing the above means I can update metadata, insert new files or
delete files I no longer want and they will just appear in the
destination "cached" formats I have chosen.

Given this currently takes about 12s (closer to 3s when everything's
already in FS cache) to scan 7457 source files, I haven't seen good
reason to improve the process beyond what it does now.

I can post my modifications to PACPL somewhere, but be warned I made
some pretty core changes and some of the output has become a little
dodgy since my changes. I had originally meant to complete a prototype
and then improve it so that I could either get a patch included
upstream or failing that post a forked project, but I sort of gave up
after the prototype worked (for me) so well.

I'm assuming once the audio formats have been output it should be
trivial to automate the generation of .torrent files in a separate
script?


Cheers,
Dave