[clug] 'dd' command detecting EOF: an example script

[David Deaves]

Using  iflag=direct  seems to do what you want, as long as the input is a file (doesn't work
for a pipe, not sure about networked file...).

(
  i=0;
  while dd count=32 bs=1k of=part_$i iflag=direct
  do
    let i++
  done
) < ~/somefile


Dave !

> This ‘feature’ of ‘dd’ (or lack of) has bugged me for a while…
>
> The only way to detect EOF I found on-line was to look at its output…
> Messy and probably fails on multiple edge conditions.
>
> Why do I want to do this?
>
> I’d like to run a command on each 32k block of a file, the only way I could think of, without writing some code, was to use ‘dd’.
> [Saw this done once to break a tar file into pieces to fit on floppy disks. Hadn’t remembered how the loop terminated.]
>
> If anyone has a better, or newer, way to do this, I’d appreciate suggestions.
>
> steve
>
> ==============
>
> Refs
>
> Learn The DD Command Revised
> https://www.linuxquestions.org/questions/linux-newbie-8/learn-the-dd-command-362506/
>
> Advanced Bash-Scripting Guide
> 	An in-depth exploration of the art of shell scripting
> http://www.tldp.org/LDP/abs/html/abs-guide.html
>
> ==============
> --
> Steve Jenkin, IT Systems and Design
> 0412 786 915 (+61 412 786 915)
> PO Box 38, Kippax ACT 2615, AUSTRALIA
>
> mailto:sjenkin at canb.auug.org.au http://members.tip.net.au/~sjenkin
>
> ———————————oooOooo———————————
>
> #!/bin/sh
> # ~/bin/dd_eof	- EXAMPLE deblocking a file with 'dd', running it through a command and detecting EOF
> #		I consider 'dd' behaviour in error - reading from an empty or closed pipe, it should exit with an error, not copy zero bytes, infinitely
> #		currently 'dd' only reports IO errors (disk full, read/write error, ...)
> # Steve Jenkin Wed 15 May 2019 08:06:52 AEST
>
> # NOTE: assumes you want to output to 'STDOUT'.
>
> DEBUG=2		# turn debugging ON, more verbose
> DEBUG=1		# turn debugging ON
> DEBUG=		# zero or unset, turn debugging OFF
>
> Tmp_EOF=$(mktemp)				# takes a little time, but should be safe, writable and free of side-effects
> trap "[ -n "$DEBUG" ] && echo ${Tmp_EOF}; /bin/rm ${Tmp_EOF}" 0 HUP TERM		# attempt to cleanup temp file.
>
> # No 'usage' or 'help' function included, as a live script should have...
>
> ######### setup script variables #########
>
> # NO error checking done
> # in a live script, *extreme* danger from running unchecked 'Cmd' variable.
>
> Ifile="${1:-t.0}"
> Cmd="${2:-/bin/cat}"
> Blk_sz="${3:-32k}"
> Count="${4:-1}"
>
> ######### start script proper #########
>
> if [[ ! -f "${Ifile}" ]] || [[ ! -r "${Ifile}" ]]
> then
>     echo "Can't read ${Ifile}" >&2
>     exit 1
> fi
>
> [[ "$DEBUG" -gt 1 ]] && set -x	# turn on debugging for the loop
> set -o pipefail			# ask shell to report _any_ non-zero exit from pipelines, not just last executed cmd (right most)
> 				# 'dd' _does_ exit with IO errors, want to catch them on input or output, though 'STDOUT' is redirected outside this script
>
> # the 'dd' feeding the file into 'while' isn't checked for errors (a limitation in production)
> # 'dd' is used because 'cat' doesn't always push large blocks, confusing the EOF test and ending on 1st read.
> # if rewriting this script for pipeline use, remove 'Ifile' and the 'dd' feeding the while loop.
>
> n=0
> dd bs="${Blk_sz}" if="${Ifile}" 2>/dev/null |\
>     while dd count="${Count}" bs="${Blk_sz}" 2>"${Tmp_EOF}" | ${Cmd} 2>/dev/null
>     do
> 	x=$?			# for debugging, remember pipeline return status.
> 	n=$(( $n + 1 ))		# count blocks
> 	[[ -n "$DEBUG" ]] && echo "status: $x blk: $n" >&2	# or '>/dev/null' for nothing
>
> 	# check Temp file for indicator of EOF.
> 	# first extract single line - could've been 'head -1', but that assumes nothing went wrong & added other lines to STDERR
> 	# then check if a full block was found
> 	if grep -e ' records in$' "${Tmp_EOF}" | grep -q -v -e "^${Count}+0 records in"
> 	then
> 	    [[ -n "$DEBUG" ]] && echo "dd exit status $?" >&2
> 	    break
> 	fi
>     done			# writes to 'STDOUT', could be captured.
>
> exit
>
> set +x	# turn off debugging after the loop
>
> ##################################3
>
> Refs
>
> Learn The DD Command Revised
> https://www.linuxquestions.org/questions/linux-newbie-8/learn-the-dd-command-362506/
>
> Advanced Bash-Scripting Guide
> 	An in-depth exploration of the art of shell scripting
> http://www.tldp.org/LDP/abs/html/abs-guide.html
>
>
> ———————————oooOooo———————————
>
>
>
> --
> linux mailing list
> linux at lists.samba.org
> https://lists.samba.org/mailman/listinfo/linux
>