[clug] command to reverse 'xxd -b'??

Ian Munsie darkstarsword at gmail.com
Sun Nov 3 22:51:29 MST 2013

>  - Python may or may not be good for bit-banging like this.
>    example welcome, if it's simple :)

In python to convert a binary string to hex string you would do this:

hex(int("100101011...", 2))

The 2 says the string is a base two number. If you don't like the fact
that hex() puts a "0x" at the start of the output string you can
always use the printf style format strings instead, like:
'%x' % int("10101010101...", 2)

The challenge with using Python here is more trying to execute a one
liner Python from a shell script, which isn't really what Python is
intended for and quickly leads to pretty ugly code. So, here's the
ugly code I came up that could replace your awk line ;-)

python -c 'import sys; print "\n".join( [ "%.2x" % int(line, 2) for
line in sys.stdin.readlines() ])' \;

Of course, I would do the whole thing in Python, but then I'm strange
like that :-p

I've used generators ("yield" keyword) below to avoid reading the
entire input at any stage, so you can run this over input that never
ends like /dev/urandom (something in your shell pipeline is blocking
that possibility).

#!/usr/bin/env python

def byte_reader(fp):
    while True:
        byte = fp.read(1)
        if not byte:
        yield ord(byte)

def bit_pairs(input):
    for byte in input:
        for i in reversed(range(0, 8, 2)):
            yield (byte >> i) & 0b11

def bit_corrector(input):
    for bits in bit_pairs(input):
        if bits in (0b01, 0b10):
            yield bits >> 1

def byte_assembler(input):
    idx = out = 0
    for bit in input:
        out |= bit << (7 - idx)
        idx += 1
        if idx == 8:
            yield out
            idx = out = 0
    # Could output an incomplete byte at the end of the stream, but
    # probably better to just discard it.
    # if idx:
    #     yield out

if __name__ == '__main__':
    import sys
    fp = sys.stdin
    if len(sys.argv) > 2:
        print 'usage: %s [ file ]' % sys.argv[0]
    if len(sys.argv) > 1:
        fp = open(sys.argv[1], 'rb')

    for out in byte_assembler(bit_corrector(bit_pairs(byte_reader(fp)))):


Please avoid sending me Word or PowerPoint attachments.
See http://www.gnu.org/philosophy/no-word-attachments.html

More information about the linux mailing list