[clug] Two (perhaps simple) shell questions.

[Brad Hards]

On Tuesday 18 August 2009 11:33:01 Andrew Janke wrote:
> 2) Suppose I am doing this:
>
> for i in `ls *.mnc`;
> do
>    <program> $i $i.out
> done
>
> In this case <program> is not dependant on any of the other results
> and I often have a large number of them to do. I also happen to have a
> machine with 8 CPU's. So is there a simple C/L function/program/thing
> that will allow me to do this:
>
> for i in `ls *.mnc`;
> do
>    do8ofthisatonce <program> $i $i.out
> done
>
> Currently for "real" things I use Sun Grid Engine so do8ofthisatonce
> is qbatch, where qbatch is my own little dirty perl interface to qsub
> (http://mavis.anu.edu.au/scripts/qbatch) so that you can use SGE like
> the old UCSF batch. Still this is not quite what I want as I'd like to
> know if there is something that will just work on the localhost for
> quick and dirty C/L hacking.
I think a python script would be better (see the multiprocessing stuff), but if 
you want to do this with posix / shell commands, perhaps what you really want 
is to set atd to run at a load of 7.5 or similar, and then submit the whole 
lot with batch.

> My only thought was to use make -j8 or the likes...
If you want something more than batch, perhaps Generic NQS would be worth 
installing.

Brad