[clug] 'dd' command detecting EOF: an example script

[steve jenkin]

This ‘feature’ of ‘dd’ (or lack of) has bugged me for a while…

The only way to detect EOF I found on-line was to look at its output…
Messy and probably fails on multiple edge conditions.

Why do I want to do this?

I’d like to run a command on each 32k block of a file, the only way I could think of, without writing some code, was to use ‘dd’.
[Saw this done once to break a tar file into pieces to fit on floppy disks. Hadn’t remembered how the loop terminated.]

If anyone has a better, or newer, way to do this, I’d appreciate suggestions.

steve

==============

Refs

Learn The DD Command Revised
https://www.linuxquestions.org/questions/linux-newbie-8/learn-the-dd-command-362506/

Advanced Bash-Scripting Guide
	An in-depth exploration of the art of shell scripting
http://www.tldp.org/LDP/abs/html/abs-guide.html

==============
--
Steve Jenkin, IT Systems and Design 
0412 786 915 (+61 412 786 915)
PO Box 38, Kippax ACT 2615, AUSTRALIA

mailto:sjenkin at canb.auug.org.au http://members.tip.net.au/~sjenkin

———————————oooOooo———————————

#!/bin/sh
# ~/bin/dd_eof	- EXAMPLE deblocking a file with 'dd', running it through a command and detecting EOF
#		I consider 'dd' behaviour in error - reading from an empty or closed pipe, it should exit with an error, not copy zero bytes, infinitely
#		currently 'dd' only reports IO errors (disk full, read/write error, ...)
# Steve Jenkin Wed 15 May 2019 08:06:52 AEST

# NOTE: assumes you want to output to 'STDOUT'.

DEBUG=2		# turn debugging ON, more verbose
DEBUG=1		# turn debugging ON
DEBUG=		# zero or unset, turn debugging OFF

Tmp_EOF=$(mktemp)				# takes a little time, but should be safe, writable and free of side-effects
trap "[ -n "$DEBUG" ] && echo ${Tmp_EOF}; /bin/rm ${Tmp_EOF}" 0 HUP TERM		# attempt to cleanup temp file.

# No 'usage' or 'help' function included, as a live script should have... 

######### setup script variables #########

# NO error checking done
# in a live script, *extreme* danger from running unchecked 'Cmd' variable.

Ifile="${1:-t.0}"
Cmd="${2:-/bin/cat}"
Blk_sz="${3:-32k}"
Count="${4:-1}"

######### start script proper #########

if [[ ! -f "${Ifile}" ]] || [[ ! -r "${Ifile}" ]]
then
    echo "Can't read ${Ifile}" >&2
    exit 1
fi

[[ "$DEBUG" -gt 1 ]] && set -x	# turn on debugging for the loop
set -o pipefail			# ask shell to report _any_ non-zero exit from pipelines, not just last executed cmd (right most)
				# 'dd' _does_ exit with IO errors, want to catch them on input or output, though 'STDOUT' is redirected outside this script

# the 'dd' feeding the file into 'while' isn't checked for errors (a limitation in production)
# 'dd' is used because 'cat' doesn't always push large blocks, confusing the EOF test and ending on 1st read.
# if rewriting this script for pipeline use, remove 'Ifile' and the 'dd' feeding the while loop.

n=0
dd bs="${Blk_sz}" if="${Ifile}" 2>/dev/null |\
    while dd count="${Count}" bs="${Blk_sz}" 2>"${Tmp_EOF}" | ${Cmd} 2>/dev/null 
    do
	x=$?			# for debugging, remember pipeline return status.
	n=$(( $n + 1 ))		# count blocks
	[[ -n "$DEBUG" ]] && echo "status: $x blk: $n" >&2	# or '>/dev/null' for nothing

	# check Temp file for indicator of EOF.
	# first extract single line - could've been 'head -1', but that assumes nothing went wrong & added other lines to STDERR
	# then check if a full block was found
	if grep -e ' records in$' "${Tmp_EOF}" | grep -q -v -e "^${Count}+0 records in"
	then
	    [[ -n "$DEBUG" ]] && echo "dd exit status $?" >&2
	    break
	fi
    done			# writes to 'STDOUT', could be captured.

exit

set +x	# turn off debugging after the loop

##################################3

Refs

Learn The DD Command Revised
https://www.linuxquestions.org/questions/linux-newbie-8/learn-the-dd-command-362506/

Advanced Bash-Scripting Guide
	An in-depth exploration of the art of shell scripting
http://www.tldp.org/LDP/abs/html/abs-guide.html


———————————oooOooo———————————