5

I want to run a sequence of command pipelines with pv on each one. Here's an example:

for p in 1 2 3
do
  cat /dev/zero | pv -N $p | dd of=/dev/null &
done

The actual commands in the pipe don't matter (cat/dd are just an example)...

The goal being 4 concurrently running pipelines, each with their own pv output. However when I try to background the commands like this, pv stops and all I get are 4 stopped jobs. I've tried with {...|pv|...}&, bash -c "...|pv|..." & all with the same result.

How can I run multiple pv command pipelines concurrently?

Improve this question

2 Answers 2

Reset to default
8

Found that I can do this with xargs and the -P option:

josh@subdivisions:/# seq 1 10 | xargs -P 4 -I {} bash -c "dd if=/dev/zero bs=1024 count=10000000 | pv -c -N {} | dd of=/dev/null"
        3: 7.35GiB 0:00:29 [ 280MiB/s] [                                                                                         <=>                                                                 ]
        1: 7.88GiB 0:00:29 [ 312MiB/s] [                                                                                         <=>                                                                 ]
        4: 7.83GiB 0:00:29 [ 258MiB/s] [                                                                                         <=>                                                                 ]
        2: 6.55GiB 0:00:29 [ 238MiB/s] [                                                                                         <=>                                                                 ]

Send output of the array to iterate over into stdin of xargs; To run all commands simultaneously, use -P 0

Improve this answer
1
  • 1
    Also note that it's essential to add the options -c -N to pv like in the example command given here. Otherwise all 4 lines are printed one on to the other Commented Aug 3, 2020 at 6:31
4

pv cannot be started in background.

As you can see in the src/main/main.c file from pv's source code, they're setting the TOSTOP flag on the terminal with tcsetattr() (in the c.c_lflag of the termios structure). They're doing that in order to receive a SIGTTOU when trying to write to the terminal when not in foreground, catch it with a signal handler, and redirect the output to /dev/null in order not to "mess up" the terminal.

/*
 * Set terminal option TOSTOP so we get signal SIGTTOU if we try to
 * write to the terminal while backgrounded.
 *
 * Also, save the current terminal attributes for later restoration.
 */
memset(&t, 0, sizeof(t));
tcgetattr(STDERR_FILENO, &t);
t_save = t;
t.c_lflag |= TOSTOP;
tcsetattr(STDERR_FILENO, TCSANOW, &t);

This is of course gross, because it doesn't set that flag just for itself, but for all the programs using the terminal.

But that's not all. As explained in the glibc manual:

Function: int tcsetattr (int filedes, int when, const struct termios *termios-p)

If this function is called from a background process on its controlling terminal, normally all processes in the process group are sent a SIGTTOU signal, in the same way as if the process were trying to write to the terminal. The exception is if the calling process itself is ignoring or blocking SIGTTOU signals, in which case the operation is performed and no signal is sent. See Job Control.

They're not blocking or ignoring the SIGTTOU. And they're also not checking the return value of tcsetattr() (which would return -1 and set errno to EINTR if they had set the SIGTTOU signal handler before).

So the process is stopped. If it receives a SIGCONT (as from the bg command), it will stop again when trying to complete the tcsetattr().

So I guess you should consider it a feature ;-)

Improve this answer
1
  • Thanks! This is really useful to understand the behavior I was seeing :) Commented Sep 28, 2018 at 21:25

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.