[clug] Command line (Bash) URL encoding & decode. Both easy and hard

[steve jenkin]

Turns out _decoding_ URL  strings is easy, handled with variable substitution and ‘echo -e’ or ‘printf' [1st example] (e.g. ‘%20’ is space, the hexadecimal value of its ASCII code)

Encoding, not so easy, there isn’t a ‘printf’ option.

There’s an issue with ‘sed’:
 “%” needs to be encoded first (to “%25”). if substitutions are done out of order, you’d get space becoming %20 then %2520.

=======

Decode URL encodings

<http://askubuntu.com/questions/53770/how-can-i-encode-and-decode-percent-encoded-strings-on-the-command-line>

Pure bash solution for decoding only:

a='%C3%A6ndr%C3%BCk'
echo -e "${a//%/\\x}”
———
ændrük

printf "${a//%/\\x}” # same result as 'echo -e’, but without trailing newline

=========

from the man page:

 ${parameter/pattern/string}

where ‘//‘ before the pattern is ‘replace globally'

=======

Encode URL's

<http://stackoverflow.com/questions/296536/urlencode-from-a-bash-script>
encoded=""
  
for (( pos=0 ; pos<strlen ; pos++ )); do
  c=${string:$pos:1}
  case "$c” in
    [-_.~a-zA-Z0-9] ) o="${c}" ;;
    * )               printf -v o '%%%02x' "'$c"
  esac

 encoded+="${o}"
 done

——
OR
—— 

URLenc=$( echo “${string}” | sed -f URL.sed) # you _could_ do this in a single long line

where URL.sed is:

# sed url escaping
s:%:%25:g
s: :%20:g
s:<:%3C:g
s:>:%3E:g
s:#:%23:g
s:{:%7B:g
s:}:%7D:g
s:|:%7C:g
s:\\:%5C:g
s:\^:%5E:g
s:~:%7E:g
s:\[:%5B:g
s:\]:%5D:g
s:`:%60:g
s:;:%3B:g
s:/:%2F:g
s:?:%3F:g
s^:^%3A^g
s:@:%40:g
s:=:%3D:g
s:&:%26:g
s:\$:%24:g
s:\!:%21:g
s:\*:%2A:g


--
Steve Jenkin, IT Systems and Design 
0412 786 915 (+61 412 786 915)
PO Box 48, Kippax ACT 2615, AUSTRALIA

mailto:sjenkin at canb.auug.org.au http://members.tip.net.au/~sjenkin