Bash: converting a string with both spaces and quotes to an array

Question

I have a function (not created by me) that outputs a series of strings inside of quotes:

command <args>

“Foo”
“FooBar”
“Foo Bar”
“FooBar/Foo Bar”

When I try to assign it to an array (Bash; BSD/Mac), instead of 4 elements, I get 7. For example, for ${array[2]} I should get “Foo Bar”, but instead, I get ”Foo with the next element being Bar”. Any element without the space works correctly (i.e. ${array[0]} = “Foo”)

How can assign each of these elements between the quote including the space to an array that the elements are separated by spaces(?) themselves?

Right now, I am thinking of using sed/awk to “strip” out the quotes, but I think there should be a better and more efficient way.

Currently, I am assigning the output of the command (looks exactly like the output above including the quotes) to a temporary variable then assigning it to an array.

_tempvar=“$(command <args>)”

declare -a _array=(${_tempvar})

Answer 1

In bash, you'd use readarray to read the lines of a file or from the output of some command into an array; however that was only added in version 4.0 released in 2009, but macos still comes with bash 3.2.

macos comes with zsh though which is a much better shell.

To get the non-empty lines of the output of a command you'd split it with the f parameter expansion flag (which splits on linefeed), and to delete the " (U+0022), “ (U+0201C) and ” (U+201D) characters, use the ${var//pattern[/replacement]} operator for instance:

#! /bin/zsh -
array=( ${(f)${"$(cmd)"//['"“”']}} ) || exit

If those are strings quoted with the U+0022 ASCII character and the quoting is compatible with the way quotes work in the zsh language, you can also use its z/Z flag (to tokenise text the same way as the language parser does) and Q flag (to remove quotes) instead of splitting by line (which would assume quoted strings can't span several lines).

#! /bin/zsh -
array=( ${(Q)${(Z[n])"$(cmd)"}} ) || exit

---

Your

declare -a array=(${tempvar})

in bash uses the split+glob operator which is invoked when an expansion is left (usually unintentionally) unquoted. It splits the output on characters of the special $IFS parameter (which by default in bash contains space, tab and newline) using a complex algorithm and the resulting words subject to globbing aka filename generation (which is hardly ever desirable).

Here, split+glob could be used to get you the non-empty lines of the output of your command, but you'd need to tune it first:

IFS=$'\n' # split on newline only:
set -o noglob # disable the glob part which we don't want
array=( $(cmd) ) # split+glob

Then you can remove the "“” with ${var//pattern[/replacement]} as well but in bash that has to be done in subsequently as it can't cumulate parameter expansion operators and the syntax (inherited from ksh93) is a bit more awkward:

array=( "${array[@]//['"“”']}" )

Note that contrary to the zsh approach, that won't handle things like "foo \"bar\" and \\backslash".

Answer 2

You get 7 elements because word splitting is occuring, caused by the spaces.

Set IFS=$'\n' before adding the strings to the array then you'll get 4 elements but with double quotes.

Example:

IFS=$'\n'

arr=($(command <args>))

If you want 4 elements without quotes do this:

IFS=$'\n'

arr=($(command <args> | sed s'#"##'g))

Full example:

IFS=$'\n'

# tst.txt has your strings:
arr=($(cat tst.txt | sed s'#"##'g))

declare -p arr

Output:

declare -a arr=([0]="Foo" [1]="FooBar" [2]="Foo Bar" [3]="FooBar/Foo Bar")

Answer 3

readarray -t array <<< $(echo $'"a a"\n"b   b"\n"c   c"')
declare -p array
declare -a array=([0]="\"a a\"" [1]="\"b   b\"" [2]="\"c   c\"")

readarray -t array <<< $(command <args>)