[clug] command to reverse 'xxd -b'??
Hal Ashburner
hal at ashburner.info
Mon Oct 28 23:10:48 MDT 2013
Hey Steve,
I'd just use a lookup table of size 16 and treat it nibble by nibble
Can do this in bash if you don't want to go the C route.
#!/bin/bash
set -e
set -u
declare -a table
table["0000"]="0";
table["0001"]="1";
table["0010"]="2";
table["0011"]="3";
table["0100"]="4";
table["0101"]="5";
table["0110"]="6";
table["0111"]="7";
table["1000"]="8";
table["1001"]="9";
table["1010"]="a";
table["1011"]="b";
table["1100"]="c";
table["1101"]="d";
table["1110"]="e";
table["1111"]="f";
echo \${table[1010]} = ${table[1010]} == a
loop, incrementing in units of 4.
with something like:
echo ${input_line:$position:4}
and if it's not divisible by 4 you have a different problem ;)
(I think I'd go the C route myself unless I was wanting to play with a
different language - x86-64 asm using AVX instructions might be fun if
you've got a Sandy Bridge or newer core i7 CPU)
On 29 October 2013 15:50, steve jenkin <sjenkin at canb.auug.org.au> wrote:
> I'm trying to understand the von Neumann 'extractor', also called
> 'corrector'. It removes 'bias' (more 0's or 1's in a bit-stream) from a
> Random Number Generator output.
>
> It looks at pair of bits, discarding '00' and '11' (bits same).
> For '01', '10', it emits the first bit and discards the second.
>
> <https://en.wikipedia.org/wiki/Randomness_extractor#Von_Neumann_extractor>
>
> So I wrote a shell script... Nothing like reusing existing wheels.
>
> Only I hit a speed bump, reversing the output of 'xxd -b'.
> [the man page says "No, we dont do this."]
>
> 'xxd -r -p' will happily reverse hex values into binary.
> [v. neat, learned that today]
>
> 'xxd -b' dumps file in binary, strings of ascii '0' and '1'.
> But I couldn't find a command that would nicely rewrite binary back into
> hex...
>
> I ended up using 'awk' :(
>
> Questions:
>
> - anyone got a way to just use commands to do this?
> [I could've used 'sed' to convert '0000' .. '1111' to hex.]
> [was going to be long & tedious]
> Any other ideas?
>
> - PERL can probably do this trivially... example welcome!
>
> - Python may or may not be good for bit-banging like this.
> example welcome, if it's simple :)
>
> Thanks In Advance
> steve
>
>
> ============= sh/bash Script, commented ===========
>
> Pass input file(s) as command line parameters,
> script writes binary output to STDOUT
>
> call as:
> script input-file >new-bin-file
>
>
> xxd -b -g0 "$@"|\ # convert input to binary, with line nrs
> # & ascii dump. '-p' doesn't work with '-b'
>
> sed -e 's/^[0-9a-f][0-9a-f]*: //' -e 's/ .*//' -e 's/../&|/g'|\
> tr '|' '\n'|grep -v -e '^$' |\ # remove line nrs & ascii dump
> # break binary into two bits per line
>
> grep -v -e '^00' -e '^11'|sed -e 's/.$//'|\
> # von Neumann corrector
>
> (tr -d '\n';echo) |\ # join output back into one long line
>
> sed -e 's/......../&|/g'|tr '|' '\n' |\
> # reformat into 8-bits per line
>
> awk '{s = $1; n=0;
> for (i=1; i <= length(s); i++)
> { n=n*2; n += index("01", substr(s, i, 1))-1;}
> printf("%02X\n", n);
> }' |\ # not fancy. rewrite as Hex
>
> (tr -d '\n';echo)|\ # back to one long line
> sed -e 's/..../&|/g'|tr '|' '\n'|\ # 4hex (2bytes) per line for xxd
>
> xxd -r -p # use xxd to reverse hexdump
>
> ================================================
>
>
> ============= sh/bash Script, usable [watch for line-breaks] ===========
>
> xxd -b -g0 "$@"|\
> sed -e 's/^[0-9a-f][0-9a-f]*: //' -e 's/ .*//' -e 's/../&|/g'|tr '|'
> '\n'|grep -v -e '^$' |\
> grep -v -e '^00' -e '^11'|sed -e 's/.$//'|(tr -d '\n';echo) |\
> sed -e 's/......../&|/g'|tr '|' '\n' |\
> awk '{s = $1; n=0;
> for (i=1; i <= length(s); i++) { n=n*2; n += index("01", substr(s, i,
> 1))-1;}
> printf("%02X\n", n);}' |\
> (tr -d '\n';echo)|sed -e 's/..../&|/g'|tr '|' '\n'|\
> xxd -r -p
>
> ================================================
>
>
> --
> Steve Jenkin, Info Tech, Systems and Design Specialist.
> 0412 786 915 (+61 412 786 915)
> PO Box 48, Kippax ACT 2615, AUSTRALIA
>
> sjenkin at canb.auug.org.au http://members.tip.net.au/~sjenkin
> --
> linux mailing list
> linux at lists.samba.org
> https://lists.samba.org/mailman/listinfo/linux
>
More information about the linux
mailing list