The only way i've found how to read a subprocess' output in a streaming fashion (while also capturing it in a variable) in Python (for multiple output streams, i.e. both stdout and stderr) is by passing the subprocess a named temporary file to write to and then opening the same temporary file in a separate reading handle.
Note: this is for Python 3
stdout_write = tempfile.NamedTemporaryFile()
stdout_read = io.open(stdout_write.name, "r")
stderr_write = tempfile.NamedTemporaryFile()
stderr_read = io.open(stderr_write.name, "r")
stdout_captured = ""
stderr_captured = ""
proc = subprocess.Popen(["command"], stdout=stdout_write, stderr=stderr_write)
while True:
proc_done: bool = cli_process.poll() is not None
while True:
content = stdout_read.read(1024)
sys.stdout.write(content)
stdout_captured += content
if len(content) < 1024:
break
while True:
content = stderr_read.read(1024)
sys.stderr.write(content)
stdout_captured += content
if len(content) < 1024:
break
if proc_done:
break
time.sleep(0.1)
stdout_write.close()
stdout_read.close()
stderr_write.close()
stderr_read.close()
However, if you don't need to capture the output, then you can simply pass sys.stdout and sys.stderr streams from your Python script to the called subprocess, as xaav suggested in his answer :
subprocess.Popen(["command"], stdout=sys.stdout, stderr=sys.stderr)
