1

In Bash, how can I compute and capture the result from an intermediate node in a pipeline?

For example, given a list of integers and a command or function maximum, using a single Bash pipeline, how can I capture the maximum value of the list of integers in a variable, but also output the list to standard output?

In the following Bash script, the idea is to change function capture_maximum_and_print_numbers so that it is a single pipeline.

function maximum {
  awk 'NR==1||$0>x{x=$0}END{print x}'
}

function capture_maximum_and_print_numbers {
  local numbers="3\n6\n1\n4\n2\n9\n5\n"
  maximum=$(printf $numbers | maximum)
  printf $numbers
}

capture_maximum_and_print_numbers
printf "maximum=$maximum\n"
Improve this question

2 Answers 2

Reset to default
2

If you are strictly looking to both assign the result to a variable AND output the list of numbers to standard output in a single line then the following will give you what you want (I changed the name of your maximum function to correspond to what it does, with the variable $maximum corresponding to the stored result itself):

function calc_max {
  awk 'NR==1||$0>x{x=$0}END{print x}'
}

function capture_maximum_and_print_numbers {
  local numbers="3\n6\n1\n4\n2\n9\n5\n"
  export maximum=$(printf $numbers | tee /dev/tty | calc_max)
 

}

capture_maximum_and_print_numbers 
echo max_result="$maximum"
#You have stored the max_result variable and can pass it to whatever you want.

tee can be used in the middle of long command piping to store the "current" result in different files, output to stdout etc. It splits the single pipe into multiple copies with individual destinations, while still allowing your "primary" pipe to process further down the chain.

tee dev/tty corresponds to piping to the current tty terminal and /dev/tty will be present on every *nix system

Improve this answer
3
  • 1
    Is it possible to use tee to store the result directly into a shell variable in the same context in which it is run? Commented Aug 3, 2021 at 17:52
  • Why do you export the maximum variable? By sending the output to /dev/tty, you make it impossible (or very difficult) to redirect the output of this script to e.g. a file or some other processing step. It also requires a TTY, which would not be available if the script was run in a non-interactive session (e.g. via a cron job). Commented Aug 5, 2021 at 20:45
  • Exporting the variable was done to fulfill the capturing requirement of "how can I capture the maximum value of the list of integers in a variable, but also output the list to standard output?". The good thing about specifying export for the result variable is it allows you to "pull it out" into your current shell using source [script] or . [script] if needed. Also, you are absolutely right that /dev/tty would not work for cron jobs, but cron's stdout needs redirection to file by default or it goes to mail. Also, any cron job can be ran without it inside of a terminal. Commented Aug 8, 2021 at 4:34
1 
#!/bin/sh

generate_data () {
        shuf -i 0-1000 -n 10 -r
}

print_maximum () {
        awk 'NR == 1 || m < $0 { m = $0 } END { print m }'
}

exec 3>&1
maximum=$(
    generate_data |
    tee /dev/fd/3 |
    print_maximum
)
exec 3>&-

printf 'maximum = %d\n' "$maximum"

This passes some random numbers through tee, duplicating them to file descriptor 3 and to the standard output of tee. The standard output is read by print_maximum, which finds and outputs the largest number among the ones generated.

File descriptor 3 is initially opened as copy of the shell's standard output stream (with exec 3>&1), which means that when tee writes to it (by writing to /dev/fd/3), the data appears on the shell's standard output. That file descriptor is later closed (with exec 3>&-).

Note that the script uses /bin/sh as it does not need to use any bash-isms. With bash though, you'd be guaranteed that /dev/fd/3 was usable as file descriptor 3, and not reliant on that path being provided by the system.

Example run:

$ sh script
918
787
290
998
737
1000
845
374
497
809
maximum = 1000

To show that all output is generated on the script's standard output, I can show that it's possible to grep the generated data:

$ sh script | grep -F 0
100
807
maximum = 807

$ sh script | grep -F 0
440
207
Improve this answer
2
  • If file descriptor 3 is a copy of standard output, why doesn't tee /dev/fd/3 output the input data twice to standard output, once to /dev/fd/3 and then to standard output attached to file descriptor 1? Commented Aug 23, 2021 at 18:59
  • I think I understand why there is no duplication. You copy standard output to file descriptor 3 in the parent shell and tee /dev/fd/3 writes to standard output in the child shell. Two different standard outputs! Commented Aug 23, 2021 at 19:02

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.