Using the grep command with an array of words

Question

I've got a text file with some lines and saved the text file into an array using

readarray -t array < Textfile

The Text file contains e.g. the following:

123
456
789

Now I'd like to use the grep command to look for the results in another text file using the array, so print out lines where "123", "456" or "789" appear. To test the array thing, I tried to make grep look for "123", "456" and "789" the same text file and output the matching lines, if at least one of the three character series appears.

I tried

grep "${array[*]}" Textfile

but this doesn't show any results. What am I doing wrong?

Answer 1

First of all, you could just use the file itself. That is far simpler than trying to use a shell array:

grep -f file1.txt file2.txt

That will print any lines from file2.txt that match any lines in file1.txt.

If you have to use an array for some reason, things are more complicated. You can't just do grep "${array[*]}" Textfile because "${array[*]}" will expand to the list of elements in the array separated by a space:

$ array=("foo" "bar" "baz")
$ echo "${array[*]}"
foo bar baz

Which means that your grep command would become:

grep 'foo bar baz' file

And that means "grep for foo bar baz in the file file". What you want to do is to grep for foo, or bar, or baz. This can be done using the -E option of grep and joining the patterns you want to search for with |:

grep -E 'foo|bar|baz' file

To do this, you will need to do something complicated like:

grep -E "$(printf '%s|' "${array[@]}" | sed 's/|$//')" file

Or perhaps:

grep -E "$(export IFS="|"; echo "${array[*]}")" file

On the whole, it's simply better, quicker and easier to use the file and forget about the array.

Answer 2

grep "${array[*]}" Textfile

Will work as long as you set IFS to newline (or anything that starts with newline), and use -- or -e to make sure it still works even if the first element starts with -.

"${array[*]}" in Korn-like shells, like "$*" in POSIX shells expands to the list of elements joined with the first character of $IFS. The default value of $IFS is <SPC><TAB><NL> (<SPC><TAB><NL><NUL> in zsh), so by default, you'd get the element joined with SPC characters. For grep, you'd need the different regexps to be newline separated for grep to loop for each one in turn.

IFS=$'\n'
grep -e "${array[*]}" file

In zsh, a cleaner approach would be grep -e "${(F)array}" file (where the F parameter expansion flag, short for pj[\n], explicitly joins with newline (aka lineFeed) instead of modifying $IFS globally), or grep -e$^array file (which expands the array in fish/rc-style, where that becomes grep -efirst -esecond file which is another way to supply several patterns to grep.

Another option is to do:

printf '%s\n' "${array[@]}" | grep -f - file

This time passing the newline-delimited list of patterns over grep's stdin rather than via an argument.

Those approaches would work whether your patterns are fixed strings (with -F), extended regexps (with -E) or basic regexps (default).

You may want to make sure the list of patterns is not empty:

(( ${#array[@]} > 0 )) && grep ...

Calling grep with an empty pattern yields different results depending on the grep implementation and generally not what you want.