[clug] using 'dd' in a pipeline

[John Mills]

On Mon, Nov 24, 2014 at 01:56:45PM +1100, steve jenkin wrote:
> Date: Mon, 24 Nov 2014 13:56:45 +1100
> From: steve jenkin <sjenkin at canb.auug.org.au>
> Subject: [clug] using 'dd' in a pipeline
> To: CLUG List <linux at lists.samba.org>
> X-Mailer: Apple Mail (2.1878.6)
> 
> I’ve been looking at deliberate MD5 collisions and went to write a simple script based on ‘dd’, only to fall flat on my face - detecting ‘end of file’ from STDIN.
> 
> ‘dd’ doesn’t consider EOF as an error and returns “success”.
> 
> I could use dd’s “iseek”, but that ties me to named files in the file system :(
> I’d prefer to use a pipeline - current script below, but it requires I first know the size of the file, exactly the same issue as ‘iseek’.
> 
> One option is to catch the output of ‘dd’ in a temporary file, then test for its size, ending when the temp file is zero length.
> 
> I tried using the bash ‘read’ with "-n 0”, but that reads an entire line [not good for a binary file]
> ‘IFS=“” read -r -n 1’ runs, but consumes a byte of the input and discards binary ‘\0’ bytes :(
> 
> The stat() and read() system calls either don’t report EOF, or return zero-length at EOF.
> 
> I could test if a program that does a read() of zero-length, then uses ‘select’ with a short timeout, would reliably detect EOF on STDIN if it was a pipe or file.
> It doesn’t seem reliable & portable to me, but it may be.
> 
> ioctl() and fcntl() didn’t seem to have
> 
> Anyone got ideas that I can use in a shell script? I’d prefer not to have to write code :)
> There may already be extended versions of ‘dd’ that do this (haven’t checked these: dc3dd dcfldd dd_rescue ddd)
> 
> Or suggestion on existing tools or how to implement a simple test for EOF on STDIN (pipe or file) in PERL, PHP or other scripting languages.
> 
> Thanks in Advance
> steve
> 
> =====================
> Two files with same MD5 hash (e06723d4961a0a3f950e7786f3766338)
> <http://natmchugh.blogspot.co.at/2014/10/how-i-created-two-images-with-same-md5.html>
> 
> Files:
> brown.jpg <http://www.fishtrap.co.uk/james.jpg.coll>
> white.jpg <http://www.fishtrap.co.uk/barry.jpg.coll>
> =====================
> 
> #!/bin/bash
> 
> bs=4096
> count=1
> 
>  for file in white.jpg brown.jpg
> do
>   echo "$file blocksize=${bs}"
>   size=$(ls -l $file | awk -v bs="$bs" '{print $5 / bs}')
> 
>   cat $file|\
>   while [[ $size -gt 0 ]]
>   do
>     echo -n "$size "
>     dd bs=$bs count=$blocks 2>/dev/null |md5sum -
>     size=$(( $size - $blocks ))
>   done
>   echo
> done
> 
> --
> Steve Jenkin, IT Systems and Design 
> 0412 786 915 (+61 412 786 915)
> PO Box 48, Kippax ACT 2615, AUSTRALIA
> 
> mailto:sjenkin at canb.auug.org.au http://members.tip.net.au/~sjenkin
> 
> -- 
> linux mailing list
> linux at lists.samba.org
> https://lists.samba.org/mailman/listinfo/linux

Steve
Can you point out where the variable 'blocks' is set in your script?

Thanks
john Mills