Timeline for What are the shell's control and redirection operators?
Current License: CC BY-SA 4.0
14 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Nov 27, 2020 at 11:29 | comment | added | user482745 | Is this true also on copy on write file systems like zfs? (Asking just for completeness sake) | |
| May 28, 2020 at 18:49 | history | edited | Kusalananda♦ | CC BY-SA 4.0 |
Fix order of options and operands in sort command
|
| Nov 14, 2019 at 18:48 | comment | added | Scott - Слава Україні | @Manav: I don’t understand what you’re saying — how is your command “in the same vein as the dd”? — and I wonder how well you understand my answer. But I see now that my answer was incomplete — it doesn’t mention that some commands that attempt to work around the problem will work sometimes, at random. Your command is an example of that — it will work sometimes, but not always. | |
| Nov 14, 2019 at 8:19 | comment | added | Manav |
In the same vein as the dd, and probably not handling as many caveats, but conceptually simpler - cat a | wc | tee a.
|
|
| Apr 13, 2017 at 12:36 | history | edited | CommunityBot |
replaced http://unix.stackexchange.com/ with https://unix.stackexchange.com/
|
|
| Feb 22, 2015 at 0:09 | comment | added | mikeserv |
Commands also buffer writes, and sometimes do so by line rather than by byte count (though admittedly, usually only when stdout is a tty - but that is not always the case). In those cases, given variable line lengths, the buffers could overrun one another. It's just not a safe way to approach writing to a file - not without (at least) an intermediate ring-buffer and/or guaranteed safe blocking factor of some kind. And there's the race involved between the shell's opening 1<> and the command's opening the named file. In one of the links I used deleted/here-doc files as ring-buffers.
|
|
| Feb 21, 2015 at 23:58 | comment | added | Scott - Слава Україні |
I don't understand. If a program is buffering reads, then, at any given time, it has probably read more data than it has processed; i.e., the read pointer (at the kernel level) is leading the read pointer at the stdio level, where the processing is occurring. And, conversely, it has written less data than it has processed; i.e., the kernel-level write pointer is trailing the stdio write pointer. Therefore, unless you're doing something that bloats the data (e.g., cat -n), the write pointer will always lag behind the read pointer.
|
|
| Feb 21, 2015 at 23:45 | comment | added | mikeserv |
Well, because most commands will buffer anyway, you run that risk regardless, or, if not that, then a neverending loop of reading the output as it is written. The dd conv=notrunc is more risky where the pipe is concerned - it isn't reading the input file, but the pipe, and, in that scenario, not even common input file is same as output file safeguards will prohibit the loop mentioned. In general, unless you are completely certain of how a filesystem and a command's i/o lib will behave together, you can never safely overwrite a file in place without first assuring your own buffer.
|
|
| Feb 21, 2015 at 23:32 | comment | added | Scott - Слава Україні |
@mikeserv: Well, the dd … conv=notrunc and the 1<> answers never truncate the output file, so, if the output of the command is less than the input (e.g., grep), there will be some bytes of the original left over at the end of the file. And, if the output is larger than the input (e.g., cat -n, nl, or (potentially) grep -n), there's a risk of overwriting old data before you've read it.
|
|
| Feb 21, 2015 at 22:15 | comment | added | mikeserv |
Well, the thing @syntaxerror said is doubly strange because, as I think, dash would be the default recovery shell on Ubuntu and it not only does not understand a <<< herestring, but it also gets anonymous pipes for << heredocuments and doesn't mess with ${TMPDIR:-/tmp} for that purpose at all. See this or this for demos on here-document handling. Also why the same amount of output or less warning?
|
|
| Feb 21, 2015 at 22:09 | comment | added | Scott - Слава Україні |
@mikeserv: Yeah, (1) I'm quoting syntaxerror, and (2) I said I was surprised. I thought that, if anything would be read-write, it would be /tmp.
|
|
| Feb 21, 2015 at 22:05 | comment | added | mikeserv | /tmp A directory made available for applications that need a place to create temporary files. Applications shall be allowed to create files in this directory, but shall not assume that such files are preserved between invocations of the application. | |
| Feb 21, 2015 at 20:43 | comment | added | Scott - Слава Україні | Since this question keeps on coming up, I thought I’d try my hand at writing a “canonical answer”. Should I post it here (and maybe link to it from some of the other more heavily trafficked questions), or should I move it to one of the questions that actually raises this issue? Also, is this perhaps a situation where questions should be merged? | |
| Feb 21, 2015 at 20:42 | history | answered | Scott - Слава Україні | CC BY-SA 3.0 |