Using variable in subprocess command (Python)

Question

Currently calling:

f = open('/tmp/list.txt','w')
f.write(list)
f.close()  # Make sure to close the file before call sub-process.
           # Otherwise, file content will not visible to sub-process.

process = subprocess.Popen('oscommand --file={}'.format(f.name),
                           shell=True, stdout=subprocess.PIPE)

However need to use a variable as an argument which has been generated using [ShortId][1]. Need something like:

u=ShortId()
process = subprocess.Popen('oscommand --label "The unique id is "'+u' --file={}'.format(f.name),
                               shell=True, stdout=subprocess.PIPE)

How is the escaping best handled ?

You are passing a string to the Popen command. You can format the string and save it as a variable before you pass it in. — James
– James, Commented Mar 29, 2016 at 13:23

ShadowRanger · Accepted Answer · 2016-03-29 13:33:02Z

4

This is actually easier if you stop trying to make str commands with shell=True and just use the safer, faster list based command with shell=False (the default):

u=ShortId()
cmd = ['oscommand', '--label', 'The unique id is {}'.format(u), '--file={}'.format(f.name)]
process = subprocess.Popen(cmd, stdout=subprocess.PIPE)

This avoids the risk of either the id or the file name containing shell metacharacters (say, a space) causing the command to be parsed incorrectly (or dangerously, if the string is actively malicious, e.g. a file named foo;sudo rm -rf /*).

Also note that there are better ways to do temporary files, whether or not the file is supposed to last after the Popen command finishes:

import tempfile

with tempfile.NamedTemporaryFile('w+t', delete=False) as f:
    f.write(mylist)
... rest of code here ...

Alternatively, if the file should be cleaned up automatically after use:

with tempfile.NamedTemporaryFile('w+t') as f:
    f.write(mylist)
    f.flush()
    ... rest of code using temp file ...

edited Mar 29, 2016 at 13:33

answered Mar 29, 2016 at 13:28

ShadowRanger

158k12 gold badges221 silver badges315 bronze badges

Sign up to request clarification or add additional context in comments.

8 Comments

chepner Over a year ago

Even the last argument can be split as two: [..., '--file', f.name]. The --arg=value form is intended for situations (not of which I can think of at the moment) where it is more convenient to think of the option and its argument as a single shell word.

ShadowRanger Over a year ago

@chepner: That would depend on the argument parsing library in use, but I agree that most would treat it equivalently.

dross Over a year ago

Tried to switch over to using tempfile before proceding however following error when implementing as shown above by ShadowRanger "Problem reading file '/tmp/tmpULLmrW' "

ShadowRanger Over a year ago

@dross: Did you close or otherwise delete the Python reference to the tempfile before the Popen-ed program finished? If it doesn't open the file until after Python deletes it, that would cause problems.

dross Over a year ago

@ShadowRange: If I try: with tempfile.NamedTemporaryFile('w+t') as f: f.write(subnets) f.seek(0) for line in f: print line.rstrip() f.close() Then I get only a partial output of lines until "Problem reading file /tmp/xxx" appears. The file has around 15,000 lines and it would appear to be getting discarded early.

|

Collectives™ on Stack Overflow

Using variable in subprocess command (Python)

1 Answer 1

8 Comments

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

8 Comments

Related