how to store the output of ubuntu command line with python

Question

I need to run the following command line (in Linux Ubuntu) from a Python script and store the results into a variable

I am using the following code

dir = 'labels.txt'
for dir in subdirs:
    args = "tail -1 %s"%(dir)
    output = subprocess.check_output(args, shell = True)

I have the following error message: No such file or directory. I can't figure out what I'm doing wrong. Thank you for your help Fred

The assignment to dir serves no purpose, since dir is then used as a loop variable in the for loop. You might as well remove that line from the post. What's relevant is subdirs, which is unknown. — Tom Karzes
– Tom Karzes, Commented Apr 2, 2021 at 9:25
See my answer that provides with a subprocess wrapper and also a correct for loop that walks over '.txt' files in a given directory. Does it work for you? — Orsiris de Jong
– Orsiris de Jong, Commented Apr 2, 2021 at 9:58

noob_coder · Accepted Answer · 2021-04-02 09:35:39Z

Assuming you mean subprocess.check_output, you're supplying relative directory. When you run a command with subprocess.check_output(args, shell=True), it is executed in your home directory - /home/<username>/. There, it is trying to access the file dir, which may not exist.

If you try to run the same command in shell, you'd get this output tail: cannot open '<filename>' for reading: No such file or directory

To counter this, I'd suggest you use absolute paths in your program. An example would be as follows

import os
import subprocess


directory_to_iterate_through = "/absolute/path/to/directory"

for file in os.listdir(directory_to_iterate_through):
    filepath = os.path.join(directory_to_iterate_through, file)
    args = "tail -1 %s" % (filepath)
    output = subprocess.check_output(args, shell=True)

Dharman · Accepted Answer · 2021-04-02 09:58:37Z

You could use package command_runner which will deal with timeouts, encoding etc on Unix and Windows platforms and provides the exit code so you know when the command fails.

Grab it with python -m pip install command_runner

Use it with

from command_runner import command_runner

all_output = []
file= 'labels.txt'

command = "tail -1 {}".format(file)
exit_code, output = command_runner(command, shell=True, timeout=30)
if exit_code == 0:
   all_output.append(output)

print(all_output)

If you happen to need to iter over files in a given directory, you might use ofunctions.file_utils.

Grab it with python -m pip install ofunctions.file_utils

Use it like:

from ofunctions.file_utils import get_files_recursive

files = get_files_recursive('/my/folder', ext_include_list='.txt')
for file in files:
    exit_code, output = command_runner('tail -1 {}'.format(file), timeout=30)
    if exit_code == 0:
        print(output)

DISCLAIMER: I am the author of command_runner package.

Collectives™ on Stack Overflow

how to store the output of ubuntu command line with python

2 Answers 2

Comments

Comments

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Related