1

I want to redirect both output and stderr to a log file. Easy enough, right?

python3 /home/user/Utilities/gpu.py L1.py &> log_L1.py.txt &

However, when instead of calling the command directly I use a system call, I do not get the output and the stderr in the files, but rather they are returned to my screen, and the output files are not created.

staggered_runner.py:

import time
import os

scripts=['L1.py','L2.py','L3.py','L4.py','L3_2D.py','L4_2D.py']

waiting=1200

for s in scripts:
    command='python3 /home/user/Utilities/gpu.py '+s+' &> log_'+s+'.txt &'
    print (command)
    os.system (command)
    time.sleep(waiting)

Then I run

python3 staggered_runner.py

I expected to receive only the direct outputs of staggered_runner.py, that is, those of each run of print (command), and have the rest directed to the to appropriate files.

How can I do this, while still using the wrapper?

Improve this question
1
  • I suspect the reason it doesn't behave as expected is that os.system uses /bin/sh - which doesn't understand &> as a redirection: see for example What shell does os.system use in Python?. However IMHO you should really be using subprocess.popen where you can set the command's output file explicitly. Commented Oct 3, 2019 at 16:33

1 Answer 1

Reset to default
1 
import subprocess, sys, shlex

try:
    process = (subprocess.Popen(shlex.split(command), 
            stdout=subprocess.PIPE, 
            stderr=subprocess.PIPE, 
            shell=True, cwd="Pass desired directory in as string variable")

  stdout, stderr = process.communicate()

use the shlex to build your command, and subprocess to execute it in the command prompt.

subprocess.Popen Allows you to set the current working directory and set a pipe up for IPC.

Improve this answer

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.