0
Summary

How to convert a single string a "b" 'c d' $'e\nf' into separate arguments, respecting quotes and preserving whitespaces and newlines?

Question

I'm trying to read and process the output of a script that exports value lists, one per line, in form of quoted strings (as created by printf %q). The values within a list are space-separated, may be quoted, may include spaces and newlines, and are of unknown count.

Example: a "b" 'c d' $'e\nf'

I'm looking for a way to restore the values in a POSIX shell script and handle them as arguments to a shell function.

Desired solutions, in descending priority:

  1. POSIX shell commands and core utilities like xargs, printf and the like; no eval
  2. Bash
  3. A full-blown programming language like perl
Skeleton for the shell script process.sh:
#!/bin/sh
set -eu

process() {
  printf '>%s<\n' "${@}"
}

main() {
  value_list="${1}"
  # split value list here and set positional parameters
  set -- ???
  process "${@}"
}

main "${@}"
Desired behaviour
$ process.sh "a \"b\" 'c d' $'e\nf'"
>a<
>b<
>c d<
>e
f<
What happened so far
  • I tried different combinations of printf (incl. %q), xargs, while read loops, changing IFS, separation of args with a null byte, subshell invocations, heredocs.
  • xargs allows unquoting, but only for spaces, not for newlines, and only without argument -d.
  • A while read loop can parse args, but does not allow to call a shell function when input is read from a pipe
1

2 Answers 2

0

This answer is based on a comment by ilkkachu and uses zsh.

Shell script process.sh:
#!/bin/sh
set -eu
export script="$(readlink -f "${0}")"

split() {
  zsh -c 'printf "%s\0" "${(Q@)${(z)1}}"' "${script}/split" "${@}"
}

process() {
  printf '>%s<\n' "${@}"
}

main() {
  value_list="${1}"
  split "${value_list}" | xargs -0 sh -c 'init() { . "${script}"; } && init && process "${@}"' "${script}/main"
}

[ "${#}" -eq 0 ] || main "${@}"
Benefits
  • It works
Drawbacks
  • Script must be sourced to call process() form xargs, a little awkward wrapped in a function to strip the arguments and prevent an endless loop
  • Requirement for zsh
0

A simplified variant of this zsh-based answer:

#!/bin/zsh
set -eu

process() {
  printf '>%s<\n' "${@}"
}

main() {
  value_list="${1}"
  process "${(Q@)${(z)value_list}}"
}

main "${@}"
Benefits
  • It works
Drawbacks
  • Requirement for zsh

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.