Linux shell command pasting issue

Question

I frequently need to run a script and provide it several lines of STDIN. When doing this, I'll typically prepare the invocations, and then copy & paste them from one terminal window to another. Ideally, I'd like to be able to paste blocks of text such as:

<script name> <script args>
<STDIN line 1>
<STDIN line 2>
...

after which I'll hit control-d to indicate to the script that STDIN has reached the end of input.

If I do this in bash (as a single paste), the script executes, none of the STDIN echos in the session, an extra <cr> must be hit before ^d is handled, and the first line of the script's STDIN is lost. So, I typically paste the script invocation as its own paste, then all the lines of STDIN as a second paste. Doing it this way, the lines pasted echo, and I can hit control-d to indicate end of input.

If I do this under zsh, most of the STDIN lines don't echo (if there are a lot of them, some of the last lines will echo, the first of which may only echo a trailing fragment of the line), and again, an extra <cr> is needed before ^d is recognized.

It doesn't matter if the script in question is python or perl...both behave the same, which leads me to believe this is a shell issue.

The question is, what's the issue, and is there a fix that would allow doing each execution as a single paste?

The content of the script being run really doesn't matter. In testing, I've used scripts as simple as a while loop reading STDIN line by line into a buffer, then outside the while loop, print the buffer. By <cr>, I mean the enter or return key...depending on what it is on your keyboard.

Answer 1

Try using a heredoc, they're pretty easy to cut and paste:

script.sh arg arg arg <<'END_INPUT'
line1
line2
line3
END_INPUT

Answer 2

Most likely, what you are observing depends on line editing and the mode your shell sets the terminal in.

When you paste the text

cat
a
b

(including a newline after "b") to the command line of an interactive Bash session, by default it finds the terminal set in raw mode — as it is when line editing is provided by the GNU Readline library; it means that, among other things, input is sent to the shell character by character (not line by line), the conversion of carriage return characters (\r, what the terminal sees when Enter is pressed or a newline is pasted to the command line) into line feed characters (\n) is disabled and typed characters are echoed to the command line by the shell itself (not by the terminal).
The shell then reads the first line, finds it is a complete simple command (cat), and runs it. While doing that it also disables its own line editing, setting the terminal in cooked mode and enabling the \r → \n conversion. At that point, though, the input buffer already contains a\rb\r, which is what is read by cat and printed on the screen. If you then press Ctrl+D (^D), cat exits and a new prompt is printed, overwriting the (only) printed line (because it ends in a carriage return).

Note that:

• If, in place of cat, you invoke a program that reads from standard input in a loop, equivalent to the script

  while IFS= read -r foo
  do
    # Accumulate the content of foo
  done
  

  it will read nothing until you hit Enter, because there is no \n character in the input buffer (unless the pasted data exceeds the terminal's input buffer size, see below).

• After pasting, the text (b, in this case) you see on the line that follows the command is not the echo of the pasted content, but the output of cat. You can easily verify this, for instance, by pasting

  od -An -tc
  a
  b
  

  Pasting into a shell run as strace -f -e read,write bash may also help in understanding what is going on.

• If the pasted text meant to be read from standard input is "enough" (on my system, more than 4095 characters; see "Canonical and noncanonical mode" in man 3 termios), what written above only applies to the first 4095 byte-sized read — what the terminal puts in its input buffer when in raw mode, at most. Subsequent reads happen in cooked mode, \r → \n conversion takes place and that part of the pasted text is echoed to the command line (and likely intermingles with the command's output).

In principle, this behavior can be changed by disabling Bash's line editing:

$ set +o emacs
$ set +o vi
$ cat         # Pasting the above snippet here
a
b
a
b
$             # ^D makes cat exit and brings the prompt back

Note, though, that lengthy input will still intermingle with the output because the terminal has no way to tell the shell to wait until the paste operation is over before starting its processing, unless "bracketed paste" is enabled (see below).

---

zsh seems to be a bit simpler. The only pitfall: "bracketed paste" is probably enabled by default. It allows you to paste multi-line commands to the command line, executing them as if they were a script while ensuring that special characters (e.g. tabs, newlines) are properly handled. This also means that, when given

cat
a
b

zsh reads the three lines and executes cat, which waits for new input from the terminal; when cat exits, the shell tries running a (which fails, unless such a command exists on your system), then b.

Disabling bracketed paste allows for the behavior you seem to be looking for (but, compared to bash with no line editing, by default zsh does not echo the pasted content, except for the part that exceeds the first 4095 bytes (see the bulleted list above)):

% unset zle_bracketed_paste
% cat         # Pasting the above snippet here
a             # cat prints these two lines
b
%             # ^D makes cat exit and brings the prompt back